Package org.apache.lucene.analysis.miscellaneous

Class LimitTokenCountAnalyzer

java.lang.Object

org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.AnalyzerWrapper

org.apache.lucene.analysis.miscellaneous.LimitTokenCountAnalyzer

All Implemented Interfaces:

Closeable, AutoCloseable

public final class LimitTokenCountAnalyzer
extends AnalyzerWrapper

This Analyzer limits the number of tokens while indexing. It is a replacement for the maximum field length setting inside IndexWriter.

Since:

3.1

See Also:

• LimitTokenCountFilter

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents

Field Summary

Fields inherited from class org.apache.lucene.analysis.Analyzer

GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY

Constructor Summary

Constructors

Constructor	Description
LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount)	Build an analyzer that limits the maximum number of tokens per field.
LimitTokenCountAnalyzer(Analyzer delegate, int maxTokenCount, boolean consumeAllTokens)	Build an analyzer that limits the maximum number of tokens per field.

Method Summary

Modifier and Type	Method	Description
protected Analyzer	getWrappedAnalyzer(String fieldName)
String	toString()
protected Analyzer.TokenStreamComponents	wrapComponents(String fieldName, Analyzer.TokenStreamComponents components)

Methods inherited from class org.apache.lucene.analysis.AnalyzerWrapper

attributeFactory, createComponents, getOffsetGap, getPositionIncrementGap, initReader, initReaderForNormalization, normalize, wrapReader, wrapReaderForNormalization, wrapTokenStreamForNormalization

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getReuseStrategy, normalize, tokenStream, tokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

LimitTokenCountAnalyzer

public LimitTokenCountAnalyzer(Analyzer delegate,
 int maxTokenCount)

Build an analyzer that limits the maximum number of tokens per field. This analyzer will not consume any tokens beyond the maxTokenCount limit

See Also:

• LimitTokenCountAnalyzer(Analyzer,int,boolean)

LimitTokenCountAnalyzer

public LimitTokenCountAnalyzer(Analyzer delegate,
 int maxTokenCount,
 boolean consumeAllTokens)

Build an analyzer that limits the maximum number of tokens per field.

Parameters:

delegate - the analyzer to wrap

maxTokenCount - max number of tokens to produce

consumeAllTokens - whether all tokens from the delegate should be consumed even if maxTokenCount is reached.

Method Details

getWrappedAnalyzer

protected Analyzer getWrappedAnalyzer(String fieldName)

Specified by:

getWrappedAnalyzer in class AnalyzerWrapper

wrapComponents

protected Analyzer.TokenStreamComponents wrapComponents(String fieldName,
 Analyzer.TokenStreamComponents components)

Overrides:

wrapComponents in class AnalyzerWrapper

toString

public String toString()

Overrides:

toString in class Object