Class LimitTokenPositionFilter
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.miscellaneous.LimitTokenPositionFilter
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Unwrappable<TokenStream>
This TokenFilter limits its emitted tokens to those with positions that are not greater than the
configured limit.
By default, this filter ignores any tokens in the wrapped TokenStream
once the limit
has been exceeded, which can result in reset()
being called prior to
incrementToken()
returning false
. For most TokenStream
implementations this
should be acceptable, and faster then consuming the full stream. If you are wrapping a
TokenStream
which requires that the full stream of tokens be exhausted in order to function
properly, use the consumeAllTokens
option.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.State
-
Field Summary
Fields inherited from class org.apache.lucene.analysis.TokenFilter
input
Fields inherited from class org.apache.lucene.analysis.TokenStream
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
-
Constructor Summary
ConstructorDescriptionLimitTokenPositionFilter
(TokenStream in, int maxTokenPosition) Build a filter that only accepts tokens up to and including the given maximum position.LimitTokenPositionFilter
(TokenStream in, int maxTokenPosition, boolean consumeAllTokens) Build a filter that limits the maximum position of tokens to emit. -
Method Summary
Methods inherited from class org.apache.lucene.analysis.TokenFilter
close, end, unwrap
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
-
Constructor Details
-
LimitTokenPositionFilter
Build a filter that only accepts tokens up to and including the given maximum position. This filter will not consume any tokens with position greater than the maxTokenPosition limit.- Parameters:
in
- the stream to wrapmaxTokenPosition
- max position of tokens to produce (1st token always has position 1)- See Also:
-
LimitTokenPositionFilter
Build a filter that limits the maximum position of tokens to emit.- Parameters:
in
- the stream to wrapmaxTokenPosition
- max position of tokens to produce (1st token always has position 1)consumeAllTokens
- whether all tokens from the wrapped input stream must be consumed even if maxTokenPosition is exceeded.
-
-
Method Details
-
incrementToken
- Specified by:
incrementToken
in classTokenStream
- Throws:
IOException
-
reset
- Overrides:
reset
in classTokenFilter
- Throws:
IOException
-