org.apache.lucene.analysis.util

Class TokenizerFactory

java.lang.Object

org.apache.lucene.analysis.util.AbstractAnalysisFactory

org.apache.lucene.analysis.util.TokenizerFactory

Direct Known Subclasses:

ClassicTokenizerFactory, EdgeNGramTokenizerFactory, KeywordTokenizerFactory, LetterTokenizerFactory, NGramTokenizerFactory, PathHierarchyTokenizerFactory, PatternTokenizerFactory, SimplePatternSplitTokenizerFactory, SimplePatternTokenizerFactory, StandardTokenizerFactory, ThaiTokenizerFactory, UAX29URLEmailTokenizerFactory, WhitespaceTokenizerFactory, WikipediaTokenizerFactory

public abstract class TokenizerFactory
extends AbstractAnalysisFactory

Abstract parent class for analysis factories that create Tokenizer instances.

Since:

3.1

Field Summary

Fields inherited from class org.apache.lucene.analysis.util.AbstractAnalysisFactory

LUCENE_MATCH_VERSION_PARAM, luceneMatchVersion

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	TokenizerFactory(Map<String,String> args) Initialize this factory via a set of key-value pairs.

Method Summary

Modifier and Type	Method and Description
static Set<String>	availableTokenizers() returns a list of all available tokenizer names from context classpath
Tokenizer	create() Creates a TokenStream of the specified input using the default attribute factory.
abstract Tokenizer	create(AttributeFactory factory) Creates a TokenStream of the specified input using the given AttributeFactory
static String	findSPIName(Class<? extends TokenizerFactory> serviceClass) looks up a SPI name for the specified tokenizer factory
static TokenizerFactory	forName(String name, Map<String,String> args) looks up a tokenizer by name from context classpath
static Class<? extends TokenizerFactory>	lookupClass(String name) looks up a tokenizer class by name from context classpath
static void	reloadTokenizers(ClassLoader classloader) Reloads the factory list from the given ClassLoader.

Methods inherited from class org.apache.lucene.analysis.util.AbstractAnalysisFactory

get, get, get, get, get, getBoolean, getChar, getClassArg, getFloat, getInt, getLines, getLuceneMatchVersion, getOriginalArgs, getPattern, getSet, getSnowballWordSet, getWordSet, isExplicitLuceneMatchVersion, require, require, require, requireBoolean, requireChar, requireFloat, requireInt, setExplicitLuceneMatchVersion, splitAt, splitFileNames

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenizerFactory

protected TokenizerFactory(Map<String,String> args)

Initialize this factory via a set of key-value pairs.

Method Detail

forName

public static TokenizerFactory forName(String name,
                                       Map<String,String> args)

looks up a tokenizer by name from context classpath

lookupClass

public static Class<? extends TokenizerFactory> lookupClass(String name)

looks up a tokenizer class by name from context classpath

availableTokenizers

public static Set<String> availableTokenizers()

returns a list of all available tokenizer names from context classpath

findSPIName

public static String findSPIName(Class<? extends TokenizerFactory> serviceClass)

looks up a SPI name for the specified tokenizer factory

reloadTokenizers

public static void reloadTokenizers(ClassLoader classloader)

Reloads the factory list from the given ClassLoader. Changes to the factories are visible after the method ends, all iterators (availableTokenizers(),...) stay consistent.

NOTE: Only new factories are added, existing ones are never removed or replaced.

This method is expensive and should only be called for discovery of new factories on the given classpath/classloader!

create

public final Tokenizer create()

Creates a TokenStream of the specified input using the default attribute factory.

create

public abstract Tokenizer create(AttributeFactory factory)

Creates a TokenStream of the specified input using the given AttributeFactory