public final class LimitTokenPositionFilter extends TokenFilter
By default, this filter ignores any tokens in the wrapped TokenStream
once the limit has been exceeded, which can result in reset()
being
called prior to incrementToken()
returning false
. For most
TokenStream
implementations this should be acceptable, and faster
then consuming the full stream. If you are wrapping a TokenStream
which requires that the full stream of tokens be exhausted in order to
function properly, use the
consumeAllTokens
option.
AttributeSource.State
input
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
DEFAULT_ATTRIBUTE_FACTORY
Constructor and Description |
---|
LimitTokenPositionFilter(TokenStream in,
int maxTokenPosition)
Build a filter that only accepts tokens up to and including the given maximum position.
|
LimitTokenPositionFilter(TokenStream in,
int maxTokenPosition,
boolean consumeAllTokens)
Build a filter that limits the maximum position of tokens to emit.
|
Modifier and Type | Method and Description |
---|---|
boolean |
incrementToken() |
void |
reset() |
close, end
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public LimitTokenPositionFilter(TokenStream in, int maxTokenPosition)
in
- the stream to wrapmaxTokenPosition
- max position of tokens to produce (1st token always has position 1)LimitTokenPositionFilter(TokenStream,int,boolean)
public LimitTokenPositionFilter(TokenStream in, int maxTokenPosition, boolean consumeAllTokens)
in
- the stream to wrapmaxTokenPosition
- max position of tokens to produce (1st token always has position 1)consumeAllTokens
- whether all tokens from the wrapped input stream must be consumed
even if maxTokenPosition is exceeded.public boolean incrementToken() throws IOException
incrementToken
in class TokenStream
IOException
public void reset() throws IOException
reset
in class TokenFilter
IOException
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.