public final class SentenceTokenizer extends Tokenizer
The output tokens can then be broken into words with WordTokenFilter
AttributeSource.AttributeFactory, AttributeSource.State
Constructor and Description |
---|
SentenceTokenizer(AttributeSource.AttributeFactory factory,
Reader reader) |
SentenceTokenizer(AttributeSource source,
Reader reader) |
SentenceTokenizer(Reader reader) |
Modifier and Type | Method and Description |
---|---|
void |
end() |
boolean |
incrementToken() |
void |
reset() |
close, correctOffset, setReader
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState
public SentenceTokenizer(Reader reader)
public SentenceTokenizer(AttributeSource source, Reader reader)
public SentenceTokenizer(AttributeSource.AttributeFactory factory, Reader reader)
public boolean incrementToken() throws IOException
incrementToken
in class TokenStream
IOException
public void reset() throws IOException
reset
in class TokenStream
IOException
public void end()
end
in class TokenStream
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.