Uses of Class
org.apache.lucene.analysis.TokenFilter

Packages that use TokenFilter

org.apache.lucene.analysis	API and code to convert text into indexable/searchable tokens.
org.apache.lucene.analysis.ar	Analyzer for Arabic.
org.apache.lucene.analysis.bg	Analyzer for Bulgarian.
org.apache.lucene.analysis.br	Analyzer for Brazilian Portuguese.
org.apache.lucene.analysis.cn	Analyzer for Chinese, which indexes unigrams (individual chinese characters).
org.apache.lucene.analysis.cn.smart	Analyzer for Simplified Chinese, which indexes words.
org.apache.lucene.analysis.compound	A filter that decomposes compound words you find in many Germanic languages into the word parts.
org.apache.lucene.analysis.cz	Analyzer for Czech.
org.apache.lucene.analysis.de	Analyzer for German.
org.apache.lucene.analysis.el	Analyzer for Greek.
org.apache.lucene.analysis.en	Analyzer for English.
org.apache.lucene.analysis.es	Analyzer for Spanish.
org.apache.lucene.analysis.fa	Analyzer for Persian.
org.apache.lucene.analysis.fi	Analyzer for Finnish.
org.apache.lucene.analysis.fr	Analyzer for French.
org.apache.lucene.analysis.gl	Analyzer for Galician.
org.apache.lucene.analysis.hi	Analyzer for Hindi.
org.apache.lucene.analysis.hu	Analyzer for Hungarian.
org.apache.lucene.analysis.hunspell	Stemming TokenFilter using a Java implementation of the Hunspell stemming algorithm.
org.apache.lucene.analysis.icu	Analysis components based on ICU
org.apache.lucene.analysis.id	Analyzer for Indonesian.
org.apache.lucene.analysis.in	Analysis components for Indian languages.
org.apache.lucene.analysis.it	Analyzer for Italian.
org.apache.lucene.analysis.lv	Analyzer for Latvian.
org.apache.lucene.analysis.miscellaneous	Miscellaneous TokenStreams
org.apache.lucene.analysis.ngram	Character n-gram tokenizers and filters.
org.apache.lucene.analysis.nl	Analyzer for Dutch.
org.apache.lucene.analysis.payloads	Provides various convenience classes for creating payloads on Tokens.
org.apache.lucene.analysis.position	Filter for assigning position increments.
org.apache.lucene.analysis.pt	Analyzer for Portuguese.
org.apache.lucene.analysis.reverse	Filter to reverse token text.
org.apache.lucene.analysis.ru	Analyzer for Russian.
org.apache.lucene.analysis.shingle	Word n-gram filters
org.apache.lucene.analysis.snowball	TokenFilter and Analyzer implementations that use Snowball stemmers.
org.apache.lucene.analysis.standard	The org.apache.lucene.analysis.standard package contains three fast grammar-based tokenizers constructed with JFlex:
org.apache.lucene.analysis.stempel	Stempel: Algorithmic Stemmer
org.apache.lucene.analysis.sv	Analyzer for Swedish.
org.apache.lucene.analysis.synonym	Analysis components for Synonyms.
org.apache.lucene.analysis.th	Analyzer for Thai.
org.apache.lucene.analysis.tr	Analyzer for Turkish.
org.apache.lucene.collation	CollationKeyFilter converts each token into its binary CollationKey using the provided Collator, and then encode the CollationKey as a String using IndexableBinaryStringTools, to allow it to be stored as an index term.
org.apache.lucene.facet.enhancements	Enhanced category features
org.apache.lucene.facet.enhancements.association	Association category enhancements
org.apache.lucene.facet.index.streaming	Expert: attributes streaming definition for indexing facets
org.apache.lucene.search.highlight	The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages.

Uses of TokenFilter in org.apache.lucene.analysis

Subclasses of TokenFilter in org.apache.lucene.analysis

class	ASCIIFoldingFilter This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.
class	CachingTokenFilter This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.
class	FilteringTokenFilter Abstract base class for TokenFilters that may remove tokens.
class	ISOLatin1AccentFilter Deprecated. If you build a new index, use ASCIIFoldingFilter which covers a superset of Latin 1. This class is included for use with existing indexes and will be removed in a future release (possibly Lucene 4.0).
class	KeywordMarkerFilter Marks terms as keywords via the KeywordAttribute.
class	LengthFilter Removes words that are too long or too short from the stream.
class	LimitTokenCountFilter This TokenFilter limits the number of tokens while indexing.
class	LowerCaseFilter Normalizes token text to lower case.
class	PorterStemFilter Transforms the token stream as per the Porter stemming algorithm.
class	StopFilter Removes stop words from a token stream.
class	TeeSinkTokenFilter This TokenFilter provides the ability to set aside attribute states that have already been analyzed.

Uses of TokenFilter in org.apache.lucene.analysis.ar

Subclasses of TokenFilter in org.apache.lucene.analysis.ar

class	ArabicNormalizationFilter A TokenFilter that applies ArabicNormalizer to normalize the orthography.
class	ArabicStemFilter A TokenFilter that applies ArabicStemmer to stem Arabic words..

Uses of TokenFilter in org.apache.lucene.analysis.bg

Subclasses of TokenFilter in org.apache.lucene.analysis.bg

class

BulgarianStemFilter A TokenFilter that applies BulgarianStemmer to stem Bulgarian words.

Uses of TokenFilter in org.apache.lucene.analysis.br

Subclasses of TokenFilter in org.apache.lucene.analysis.br

class

BrazilianStemFilter A TokenFilter that applies BrazilianStemmer.

Uses of TokenFilter in org.apache.lucene.analysis.cn

Subclasses of TokenFilter in org.apache.lucene.analysis.cn

class

ChineseFilter Deprecated. Use StopFilter instead, which has the same functionality. This filter will be removed in Lucene 5.0

Uses of TokenFilter in org.apache.lucene.analysis.cn.smart

Subclasses of TokenFilter in org.apache.lucene.analysis.cn.smart

class

WordTokenFilter A TokenFilter that breaks sentences into words.

Uses of TokenFilter in org.apache.lucene.analysis.compound

Subclasses of TokenFilter in org.apache.lucene.analysis.compound

class	CompoundWordTokenFilterBase Base class for decomposition token filters.
class	DictionaryCompoundWordTokenFilter A TokenFilter that decomposes compound words found in many Germanic languages.
class	HyphenationCompoundWordTokenFilter A TokenFilter that decomposes compound words found in many Germanic languages.

Uses of TokenFilter in org.apache.lucene.analysis.cz

Subclasses of TokenFilter in org.apache.lucene.analysis.cz

class

CzechStemFilter A TokenFilter that applies CzechStemmer to stem Czech words.

Uses of TokenFilter in org.apache.lucene.analysis.de

Subclasses of TokenFilter in org.apache.lucene.analysis.de

class	GermanLightStemFilter A TokenFilter that applies GermanLightStemmer to stem German words.
class	GermanMinimalStemFilter A TokenFilter that applies GermanMinimalStemmer to stem German words.
class	GermanStemFilter A TokenFilter that stems German words.

Uses of TokenFilter in org.apache.lucene.analysis.el

Subclasses of TokenFilter in org.apache.lucene.analysis.el

class	GreekLowerCaseFilter Normalizes token text to lower case, removes some Greek diacritics, and standardizes final sigma to sigma.
class	GreekStemFilter A TokenFilter that applies GreekStemmer to stem Greek words.

Uses of TokenFilter in org.apache.lucene.analysis.en

Subclasses of TokenFilter in org.apache.lucene.analysis.en

class	EnglishMinimalStemFilter A TokenFilter that applies EnglishMinimalStemmer to stem English words.
class	EnglishPossessiveFilter TokenFilter that removes possessives (trailing 's) from words.
class	KStemFilter A high-performance kstem filter for english.

Uses of TokenFilter in org.apache.lucene.analysis.es

Subclasses of TokenFilter in org.apache.lucene.analysis.es

class

SpanishLightStemFilter A TokenFilter that applies SpanishLightStemmer to stem Spanish words.

Uses of TokenFilter in org.apache.lucene.analysis.fa

Subclasses of TokenFilter in org.apache.lucene.analysis.fa

class

PersianNormalizationFilter A TokenFilter that applies PersianNormalizer to normalize the orthography.

Uses of TokenFilter in org.apache.lucene.analysis.fi

Subclasses of TokenFilter in org.apache.lucene.analysis.fi

class

FinnishLightStemFilter A TokenFilter that applies FinnishLightStemmer to stem Finnish words.

Uses of TokenFilter in org.apache.lucene.analysis.fr

Subclasses of TokenFilter in org.apache.lucene.analysis.fr

class	ElisionFilter Removes elisions from a TokenStream.
class	FrenchLightStemFilter A TokenFilter that applies FrenchLightStemmer to stem French words.
class	FrenchMinimalStemFilter A TokenFilter that applies FrenchMinimalStemmer to stem French words.
class	FrenchStemFilter Deprecated. Use SnowballFilter with FrenchStemmer instead, which has the same functionality. This filter will be removed in Lucene 5.0

Uses of TokenFilter in org.apache.lucene.analysis.gl

Subclasses of TokenFilter in org.apache.lucene.analysis.gl

class

GalicianStemFilter A TokenFilter that applies GalicianStemmer to stem Galician words.

Uses of TokenFilter in org.apache.lucene.analysis.hi

Subclasses of TokenFilter in org.apache.lucene.analysis.hi

class	HindiNormalizationFilter A TokenFilter that applies HindiNormalizer to normalize the orthography.
class	HindiStemFilter A TokenFilter that applies HindiStemmer to stem Hindi words.

Uses of TokenFilter in org.apache.lucene.analysis.hu

Subclasses of TokenFilter in org.apache.lucene.analysis.hu

class

HungarianLightStemFilter A TokenFilter that applies HungarianLightStemmer to stem Hungarian words.

Uses of TokenFilter in org.apache.lucene.analysis.hunspell

Subclasses of TokenFilter in org.apache.lucene.analysis.hunspell

class

HunspellStemFilter TokenFilter that uses hunspell affix rules and words to stem tokens.

Uses of TokenFilter in org.apache.lucene.analysis.icu

Subclasses of TokenFilter in org.apache.lucene.analysis.icu

class	ICUFoldingFilter A TokenFilter that applies search term folding to Unicode text, applying foldings from UTR#30 Character Foldings.
class	ICUNormalizer2Filter Normalize token text with ICU's Normalizer2
class	ICUTransformFilter A TokenFilter that transforms text with ICU.

Uses of TokenFilter in org.apache.lucene.analysis.id

Subclasses of TokenFilter in org.apache.lucene.analysis.id

class

IndonesianStemFilter A TokenFilter that applies IndonesianStemmer to stem Indonesian words.

Uses of TokenFilter in org.apache.lucene.analysis.in

Subclasses of TokenFilter in org.apache.lucene.analysis.in

class

IndicNormalizationFilter A TokenFilter that applies IndicNormalizer to normalize text in Indian Languages.

Uses of TokenFilter in org.apache.lucene.analysis.it

Subclasses of TokenFilter in org.apache.lucene.analysis.it

class

ItalianLightStemFilter A TokenFilter that applies ItalianLightStemmer to stem Italian words.

Uses of TokenFilter in org.apache.lucene.analysis.lv

Subclasses of TokenFilter in org.apache.lucene.analysis.lv

class

LatvianStemFilter A TokenFilter that applies LatvianStemmer to stem Latvian words.

Uses of TokenFilter in org.apache.lucene.analysis.miscellaneous

Subclasses of TokenFilter in org.apache.lucene.analysis.miscellaneous

class

StemmerOverrideFilter Provides the ability to override any KeywordAttribute aware stemmer with custom dictionary-based stemming.

Uses of TokenFilter in org.apache.lucene.analysis.ngram

Subclasses of TokenFilter in org.apache.lucene.analysis.ngram

class	EdgeNGramTokenFilter Tokenizes the given token into n-grams of given size(s).
class	NGramTokenFilter Tokenizes the input into n-grams of the given size(s).

Uses of TokenFilter in org.apache.lucene.analysis.nl

Subclasses of TokenFilter in org.apache.lucene.analysis.nl

class

DutchStemFilter Deprecated. Use SnowballFilter with DutchStemmer instead, which has the same functionality. This filter will be removed in Lucene 5.0

Uses of TokenFilter in org.apache.lucene.analysis.payloads

Subclasses of TokenFilter in org.apache.lucene.analysis.payloads

class	DelimitedPayloadTokenFilter Characters before the delimiter are the "token", those after are the payload.
class	NumericPayloadTokenFilter Assigns a payload to a token based on the Token.type()
class	TokenOffsetPayloadTokenFilter Adds the Token.setStartOffset(int) and Token.setEndOffset(int) First 4 bytes are the start
class	TypeAsPayloadTokenFilter Makes the Token.type() a payload.

Uses of TokenFilter in org.apache.lucene.analysis.position

Subclasses of TokenFilter in org.apache.lucene.analysis.position

class

PositionFilter Set the positionIncrement of all tokens to the "positionIncrement", except the first return token which retains its original positionIncrement value.

Uses of TokenFilter in org.apache.lucene.analysis.pt

Subclasses of TokenFilter in org.apache.lucene.analysis.pt

class	PortugueseLightStemFilter A TokenFilter that applies PortugueseLightStemmer to stem Portuguese words.
class	PortugueseMinimalStemFilter A TokenFilter that applies PortugueseMinimalStemmer to stem Portuguese words.
class	PortugueseStemFilter A TokenFilter that applies PortugueseStemmer to stem Portuguese words.

Uses of TokenFilter in org.apache.lucene.analysis.reverse

Subclasses of TokenFilter in org.apache.lucene.analysis.reverse

class

ReverseStringFilter Reverse token string, for example "country" => "yrtnuoc".

Uses of TokenFilter in org.apache.lucene.analysis.ru

Subclasses of TokenFilter in org.apache.lucene.analysis.ru

class	RussianLightStemFilter A TokenFilter that applies RussianLightStemmer to stem Russian words.
class	RussianLowerCaseFilter Deprecated. Use LowerCaseFilter instead, which has the same functionality. This filter will be removed in Lucene 4.0
class	RussianStemFilter Deprecated. Use SnowballFilter with RussianStemmer instead, which has the same functionality. This filter will be removed in Lucene 4.0

Uses of TokenFilter in org.apache.lucene.analysis.shingle

Subclasses of TokenFilter in org.apache.lucene.analysis.shingle

class

ShingleFilter A ShingleFilter constructs shingles (token n-grams) from a token stream.

Uses of TokenFilter in org.apache.lucene.analysis.snowball

Subclasses of TokenFilter in org.apache.lucene.analysis.snowball

class

SnowballFilter A filter that stems words using a Snowball-generated stemmer.

Uses of TokenFilter in org.apache.lucene.analysis.standard

Subclasses of TokenFilter in org.apache.lucene.analysis.standard

class	ClassicFilter Normalizes tokens extracted with ClassicTokenizer.
class	StandardFilter Normalizes tokens extracted with StandardTokenizer.

Uses of TokenFilter in org.apache.lucene.analysis.stempel

Subclasses of TokenFilter in org.apache.lucene.analysis.stempel

class

StempelFilter Transforms the token stream as per the stemming algorithm.

Uses of TokenFilter in org.apache.lucene.analysis.sv

Subclasses of TokenFilter in org.apache.lucene.analysis.sv

class

SwedishLightStemFilter A TokenFilter that applies SwedishLightStemmer to stem Swedish words.

Uses of TokenFilter in org.apache.lucene.analysis.synonym

Subclasses of TokenFilter in org.apache.lucene.analysis.synonym

class

SynonymFilter Matches single or multi word synonyms in a token stream.

Uses of TokenFilter in org.apache.lucene.analysis.th

Subclasses of TokenFilter in org.apache.lucene.analysis.th

class

ThaiWordFilter TokenFilter that use BreakIterator to break each Token that is Thai into separate Token(s) for each Thai word.

Uses of TokenFilter in org.apache.lucene.analysis.tr

Subclasses of TokenFilter in org.apache.lucene.analysis.tr

class

TurkishLowerCaseFilter Normalizes Turkish token text to lower case.

Uses of TokenFilter in org.apache.lucene.collation

Subclasses of TokenFilter in org.apache.lucene.collation

class	CollationKeyFilter Converts each token into its CollationKey, and then encodes the CollationKey with IndexableBinaryStringTools, to allow it to be stored as an index term.
class	ICUCollationKeyFilter Converts each token into its CollationKey, and then encodes the CollationKey with IndexableBinaryStringTools, to allow it to be stored as an index term.

Uses of TokenFilter in org.apache.lucene.facet.enhancements

Subclasses of TokenFilter in org.apache.lucene.facet.enhancements

class

EnhancementsCategoryTokenizer A tokenizer which adds to each category token payload according to the CategoryEnhancements defined in the given EnhancementsIndexingParams.

Uses of TokenFilter in org.apache.lucene.facet.enhancements.association

Subclasses of TokenFilter in org.apache.lucene.facet.enhancements.association

class

AssociationListTokenizer Tokenizer for associations of a category

Uses of TokenFilter in org.apache.lucene.facet.index.streaming

Subclasses of TokenFilter in org.apache.lucene.facet.index.streaming

class	CategoryListTokenizer A base class for category list tokenizers, which add category list tokens to category streams.
class	CategoryParentsStream This class adds parents to a CategoryAttributesStream.
class	CategoryTokenizer Basic class for setting the CharTermAttributes and PayloadAttributes of category tokens.
class	CategoryTokenizerBase A base class for all token filters which add term and payload attributes to tokens and are to be used in CategoryDocumentBuilder.
class	CountingListTokenizer CategoryListTokenizer for facet counting

Uses of TokenFilter in org.apache.lucene.search.highlight

Subclasses of TokenFilter in org.apache.lucene.search.highlight

class

OffsetLimitTokenFilter This TokenFilter limits the number of tokens while indexing by adding up the current offset.