|Class and Description|
This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.
A filter to apply normal capitalization rules to Tokens.
When the plain text is extracted from documents, we will often have many words hyphenated and broken into two lines.
Marks terms as keywords via the
Removes words that are too long or too short from the stream.
(4.0) use the pattern-based analysis in the analysis/pattern package instead.
A TokenFilter which filters out Tokens at the same position and Term text as the previous token in the stream.
A read-only 4-byte FST backed map that allows fast case-insensitive key value lookups for
Trims leading and trailing whitespace from Tokens in the stream.
Splits words into subwords and performs optional transformations on subword groups.
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.