org.apache.lucene.analysis.miscellaneous
Class LimitTokenCountFilter
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.miscellaneous.LimitTokenCountFilter
- All Implemented Interfaces:
- Closeable
public final class LimitTokenCountFilter
- extends TokenFilter
This TokenFilter limits the number of tokens while indexing. It is
a replacement for the maximum field length setting inside IndexWriter
.
By default, this filter ignores any tokens in the wrapped TokenStream
once the limit has been reached, which can result in reset()
being
called prior to incrementToken()
returning false
. For most
TokenStream
implementations this should be acceptable, and faster
then consuming the full stream. If you are wrapping a TokenStream
which requires that the full stream of tokens be exhausted in order to
function properly, use the
consumeAllTokens
option.
Methods inherited from class org.apache.lucene.util.AttributeSource |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState |
LimitTokenCountFilter
public LimitTokenCountFilter(TokenStream in,
int maxTokenCount)
- Build a filter that only accepts tokens up to a maximum number.
This filter will not consume any tokens beyond the maxTokenCount limit
- See Also:
LimitTokenCountFilter(TokenStream,int,boolean)
LimitTokenCountFilter
public LimitTokenCountFilter(TokenStream in,
int maxTokenCount,
boolean consumeAllTokens)
- Build an filter that limits the maximum number of tokens per field.
- Parameters:
in
- the stream to wrapmaxTokenCount
- max number of tokens to produceconsumeAllTokens
- whether all tokens from the input must be consumed even if maxTokenCount is reached.
incrementToken
public boolean incrementToken()
throws IOException
- Specified by:
incrementToken
in class TokenStream
- Throws:
IOException
reset
public void reset()
throws IOException
- Overrides:
reset
in class TokenFilter
- Throws:
IOException
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.