Class LimitTokenCountAnalyzer
- java.lang.Object
-
- org.apache.lucene.analysis.Analyzer
-
- org.apache.lucene.analysis.AnalyzerWrapper
-
- org.apache.lucene.analysis.miscellaneous.LimitTokenCountAnalyzer
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public final class LimitTokenCountAnalyzer extends AnalyzerWrapper
This Analyzer limits the number of tokens while indexing. It is a replacement for the maximum field length setting insideIndexWriter
.- Since:
- 3.1
- See Also:
LimitTokenCountFilter
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
-
-
Field Summary
-
Fields inherited from class org.apache.lucene.analysis.Analyzer
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
-
-
Constructor Summary
Constructors Constructor Description LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount)
Build an analyzer that limits the maximum number of tokens per field.LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount, boolean consumeAllTokens)
Build an analyzer that limits the maximum number of tokens per field.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected Analyzer
getWrappedAnalyzer(String fieldName)
String
toString()
protected Analyzer.TokenStreamComponents
wrapComponents(String fieldName, Analyzer.TokenStreamComponents components)
-
Methods inherited from class org.apache.lucene.analysis.AnalyzerWrapper
attributeFactory, createComponents, getOffsetGap, getPositionIncrementGap, initReader, initReaderForNormalization, normalize, wrapReader, wrapReaderForNormalization, wrapTokenStreamForNormalization
-
Methods inherited from class org.apache.lucene.analysis.Analyzer
close, getReuseStrategy, normalize, tokenStream, tokenStream
-
-
-
-
Constructor Detail
-
LimitTokenCountAnalyzer
public LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount)
Build an analyzer that limits the maximum number of tokens per field. This analyzer will not consume any tokens beyond the maxTokenCount limit
-
LimitTokenCountAnalyzer
public LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount, boolean consumeAllTokens)
Build an analyzer that limits the maximum number of tokens per field.- Parameters:
delegate
- the analyzer to wrapmaxTokenCount
- max number of tokens to produceconsumeAllTokens
- whether all tokens from the delegate should be consumed even if maxTokenCount is reached.
-
-
Method Detail
-
getWrappedAnalyzer
protected Analyzer getWrappedAnalyzer(String fieldName)
- Specified by:
getWrappedAnalyzer
in classAnalyzerWrapper
-
wrapComponents
protected Analyzer.TokenStreamComponents wrapComponents(String fieldName, Analyzer.TokenStreamComponents components)
- Overrides:
wrapComponents
in classAnalyzerWrapper
-
-