HMMChineseTokenizer
instead.@Deprecated public final class WordTokenFilter extends TokenFilter
TokenFilter
that breaks sentences into words.AttributeSource.State
input
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
Constructor and Description |
---|
WordTokenFilter(TokenStream in)
Deprecated.
Construct a new WordTokenizer.
|
Modifier and Type | Method and Description |
---|---|
boolean |
incrementToken()
Deprecated.
|
void |
reset()
Deprecated.
|
close, end
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public WordTokenFilter(TokenStream in)
in
- TokenStream
of sentencespublic boolean incrementToken() throws IOException
incrementToken
in class TokenStream
IOException
public void reset() throws IOException
reset
in class TokenFilter
IOException
Copyright © 2000-2015 Apache Software Foundation. All Rights Reserved.