Class LimitTokenCountAnalyzer
java.lang.Object
org.apache.lucene.analysis.Analyzer
org.apache.lucene.analysis.AnalyzerWrapper
org.apache.lucene.analysis.miscellaneous.LimitTokenCountAnalyzer
- All Implemented Interfaces:
Closeable
,AutoCloseable
This Analyzer limits the number of tokens while indexing. It is a replacement for the maximum
field length setting inside
IndexWriter
.- Since:
- 3.1
- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
-
Field Summary
Fields inherited from class org.apache.lucene.analysis.Analyzer
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
-
Constructor Summary
ConstructorDescriptionLimitTokenCountAnalyzer
(Analyzer delegate, int maxTokenCount) Build an analyzer that limits the maximum number of tokens per field.LimitTokenCountAnalyzer
(Analyzer delegate, int maxTokenCount, boolean consumeAllTokens) Build an analyzer that limits the maximum number of tokens per field. -
Method Summary
Modifier and TypeMethodDescriptionprotected Analyzer
getWrappedAnalyzer
(String fieldName) toString()
protected Analyzer.TokenStreamComponents
wrapComponents
(String fieldName, Analyzer.TokenStreamComponents components) Methods inherited from class org.apache.lucene.analysis.AnalyzerWrapper
attributeFactory, createComponents, getOffsetGap, getPositionIncrementGap, initReader, initReaderForNormalization, normalize, wrapReader, wrapReaderForNormalization, wrapTokenStreamForNormalization
Methods inherited from class org.apache.lucene.analysis.Analyzer
close, getReuseStrategy, normalize, tokenStream, tokenStream
-
Constructor Details
-
LimitTokenCountAnalyzer
Build an analyzer that limits the maximum number of tokens per field. This analyzer will not consume any tokens beyond the maxTokenCount limit- See Also:
-
LimitTokenCountAnalyzer
Build an analyzer that limits the maximum number of tokens per field.- Parameters:
delegate
- the analyzer to wrapmaxTokenCount
- max number of tokens to produceconsumeAllTokens
- whether all tokens from the delegate should be consumed even if maxTokenCount is reached.
-
-
Method Details
-
getWrappedAnalyzer
- Specified by:
getWrappedAnalyzer
in classAnalyzerWrapper
-
wrapComponents
protected Analyzer.TokenStreamComponents wrapComponents(String fieldName, Analyzer.TokenStreamComponents components) - Overrides:
wrapComponents
in classAnalyzerWrapper
-
toString
-