Class DelimitedTermFrequencyTokenFilter
- java.lang.Object
-
- org.apache.lucene.util.AttributeSource
-
- org.apache.lucene.analysis.TokenStream
-
- org.apache.lucene.analysis.TokenFilter
-
- org.apache.lucene.analysis.miscellaneous.DelimitedTermFrequencyTokenFilter
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Unwrappable<TokenStream>
public final class DelimitedTermFrequencyTokenFilter extends TokenFilter
Characters before the delimiter are the "token", the textual integer after is the term frequency. To use thisTokenFilter
the field must be indexed withIndexOptions.DOCS_AND_FREQS
but no positions or offsets.For example, if the delimiter is '|', then for the string "foo|5", "foo" is the token and "5" is a term frequency. If there is no delimiter, the TokenFilter does not modify the term frequency.
Note make sure your Tokenizer doesn't split on the delimiter, or this won't work
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.State
-
-
Field Summary
Fields Modifier and Type Field Description static char
DEFAULT_DELIMITER
-
Fields inherited from class org.apache.lucene.analysis.TokenFilter
input
-
Fields inherited from class org.apache.lucene.analysis.TokenStream
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
-
-
Constructor Summary
Constructors Constructor Description DelimitedTermFrequencyTokenFilter(TokenStream input)
DelimitedTermFrequencyTokenFilter(TokenStream input, char delimiter)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
incrementToken()
-
Methods inherited from class org.apache.lucene.analysis.TokenFilter
close, end, reset, unwrap
-
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
-
-
-
-
Field Detail
-
DEFAULT_DELIMITER
public static final char DEFAULT_DELIMITER
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
DelimitedTermFrequencyTokenFilter
public DelimitedTermFrequencyTokenFilter(TokenStream input)
-
DelimitedTermFrequencyTokenFilter
public DelimitedTermFrequencyTokenFilter(TokenStream input, char delimiter)
-
-
Method Detail
-
incrementToken
public boolean incrementToken() throws IOException
- Specified by:
incrementToken
in classTokenStream
- Throws:
IOException
-
-