Package org.apache.lucene.analysis.util
Class TokenizerFactory
- java.lang.Object
-
- org.apache.lucene.analysis.util.AbstractAnalysisFactory
-
- org.apache.lucene.analysis.util.TokenizerFactory
-
- Direct Known Subclasses:
ClassicTokenizerFactory
,EdgeNGramTokenizerFactory
,KeywordTokenizerFactory
,LetterTokenizerFactory
,LowerCaseTokenizerFactory
,NGramTokenizerFactory
,PathHierarchyTokenizerFactory
,PatternTokenizerFactory
,SimplePatternSplitTokenizerFactory
,SimplePatternTokenizerFactory
,StandardTokenizerFactory
,ThaiTokenizerFactory
,UAX29URLEmailTokenizerFactory
,WhitespaceTokenizerFactory
,WikipediaTokenizerFactory
public abstract class TokenizerFactory extends AbstractAnalysisFactory
Abstract parent class for analysis factories that createTokenizer
instances.- Since:
- 3.1
-
-
Field Summary
-
Fields inherited from class org.apache.lucene.analysis.util.AbstractAnalysisFactory
LUCENE_MATCH_VERSION_PARAM, luceneMatchVersion
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
TokenizerFactory(Map<String,String> args)
Initialize this factory via a set of key-value pairs.
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description static Set<String>
availableTokenizers()
returns a list of all available tokenizer names from context classpathTokenizer
create()
Creates a TokenStream of the specified input using the default attribute factory.abstract Tokenizer
create(AttributeFactory factory)
Creates a TokenStream of the specified input using the given AttributeFactorystatic TokenizerFactory
forName(String name, Map<String,String> args)
looks up a tokenizer by name from context classpathstatic Class<? extends TokenizerFactory>
lookupClass(String name)
looks up a tokenizer class by name from context classpathstatic void
reloadTokenizers(ClassLoader classloader)
Reloads the factory list from the givenClassLoader
.-
Methods inherited from class org.apache.lucene.analysis.util.AbstractAnalysisFactory
get, get, get, get, get, getBoolean, getChar, getClassArg, getFloat, getInt, getLines, getLuceneMatchVersion, getOriginalArgs, getPattern, getSet, getSnowballWordSet, getWordSet, isExplicitLuceneMatchVersion, require, require, require, requireBoolean, requireChar, requireFloat, requireInt, setExplicitLuceneMatchVersion, splitAt, splitFileNames
-
-
-
-
Method Detail
-
forName
public static TokenizerFactory forName(String name, Map<String,String> args)
looks up a tokenizer by name from context classpath
-
lookupClass
public static Class<? extends TokenizerFactory> lookupClass(String name)
looks up a tokenizer class by name from context classpath
-
availableTokenizers
public static Set<String> availableTokenizers()
returns a list of all available tokenizer names from context classpath
-
reloadTokenizers
public static void reloadTokenizers(ClassLoader classloader)
Reloads the factory list from the givenClassLoader
. Changes to the factories are visible after the method ends, all iterators (availableTokenizers()
,...) stay consistent.NOTE: Only new factories are added, existing ones are never removed or replaced.
This method is expensive and should only be called for discovery of new factories on the given classpath/classloader!
-
create
public final Tokenizer create()
Creates a TokenStream of the specified input using the default attribute factory.
-
create
public abstract Tokenizer create(AttributeFactory factory)
Creates a TokenStream of the specified input using the given AttributeFactory
-
-