Uses of Class
org.apache.lucene.analysis.Tokenizer

Packages that use Tokenizer

Package	Description
org.apache.lucene.analysis	Text analysis.
org.apache.lucene.analysis.standard	Fast, general-purpose grammar-based tokenizer StandardTokenizer implements the Word Break rules from the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29.

Uses of Tokenizer in org.apache.lucene.analysis

Methods in org.apache.lucene.analysis that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	TokenizerFactory.create()	Creates a TokenStream of the specified input using the default attribute factory.
abstract Tokenizer	TokenizerFactory.create(AttributeFactory factory)	Creates a TokenStream of the specified input using the given AttributeFactory

Constructors in org.apache.lucene.analysis with parameters of type Tokenizer

Constructor	Description
TokenStreamComponents(Tokenizer tokenizer)	Creates a new Analyzer.TokenStreamComponents from a Tokenizer
TokenStreamComponents(Tokenizer tokenizer, TokenStream result)	Creates a new Analyzer.TokenStreamComponents instance

Uses of Tokenizer in org.apache.lucene.analysis.standard

Subclasses of Tokenizer in org.apache.lucene.analysis.standard

Modifier and Type	Class	Description
class	StandardTokenizer	A grammar-based tokenizer constructed with JFlex.