public final class SentenceTokenizer
extends org.apache.lucene.analysis.Tokenizer
The output tokens can then be broken into words with WordTokenFilter
Constructor and Description |
---|
SentenceTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
Reader reader) |
SentenceTokenizer(org.apache.lucene.util.AttributeSource source,
Reader reader) |
SentenceTokenizer(Reader reader) |
Modifier and Type | Method and Description |
---|---|
void |
end() |
boolean |
incrementToken() |
void |
reset() |
void |
reset(Reader input) |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public SentenceTokenizer(Reader reader)
public SentenceTokenizer(org.apache.lucene.util.AttributeSource source, Reader reader)
public SentenceTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory, Reader reader)
public boolean incrementToken() throws IOException
incrementToken
in class org.apache.lucene.analysis.TokenStream
IOException
public void reset() throws IOException
reset
in class org.apache.lucene.analysis.TokenStream
IOException
public void reset(Reader input) throws IOException
reset
in class org.apache.lucene.analysis.Tokenizer
IOException
public void end() throws IOException
end
in class org.apache.lucene.analysis.TokenStream
IOException