Uses of Class
org.apache.lucene.analysis.Tokenizer

Packages that use Tokenizer

Package	Description
org.apache.lucene.analysis	Text analysis.
org.apache.lucene.analysis.standard	Fast, general-purpose grammar-based tokenizer StandardTokenizer implements the Word Break rules from the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29.

Uses of Tokenizer in org.apache.lucene.analysis

Constructors in org.apache.lucene.analysis with parameters of type Tokenizer

Constructor and Description
TokenStreamComponents(Tokenizer tokenizer) Creates a new Analyzer.TokenStreamComponents from a Tokenizer
TokenStreamComponents(Tokenizer tokenizer, TokenStream result) Creates a new Analyzer.TokenStreamComponents instance

Uses of Tokenizer in org.apache.lucene.analysis.standard

Subclasses of Tokenizer in org.apache.lucene.analysis.standard

Modifier and Type	Class and Description
class	StandardTokenizer A grammar-based tokenizer constructed with JFlex.