Class TokenizerFactory

Direct Known Subclasses:

public abstract class TokenizerFactory extends AbstractAnalysisFactory
Abstract parent class for analysis factories that create Tokenizer instances.
  • Constructor Details

    • TokenizerFactory

      protected TokenizerFactory()
      Default ctor for compatibility with SPI
    • TokenizerFactory

      protected TokenizerFactory(Map<String,String> args)
      Initialize this factory via a set of key-value pairs.
  • Method Details

    • forName

      public static TokenizerFactory forName(String name, Map<String,String> args)
      looks up a tokenizer by name from context classpath
    • lookupClass

      public static Class<? extends TokenizerFactory> lookupClass(String name)
      looks up a tokenizer class by name from context classpath
    • availableTokenizers

      public static Set<String> availableTokenizers()
      returns a list of all available tokenizer names from context classpath
    • findSPIName

      public static String findSPIName(Class<? extends TokenizerFactory> serviceClass)
      looks up a SPI name for the specified tokenizer factory
    • reloadTokenizers

      public static void reloadTokenizers(ClassLoader classloader)
      Reloads the factory list from the given ClassLoader. Changes to the factories are visible after the method ends, all iterators (availableTokenizers(),...) stay consistent.

      NOTE: Only new factories are added, existing ones are never removed or replaced.

      This method is expensive and should only be called for discovery of new factories on the given classpath/classloader!

    • create

      public final Tokenizer create()
      Creates a TokenStream of the specified input using the default attribute factory.
    • create

      public abstract Tokenizer create(AttributeFactory factory)
      Creates a TokenStream of the specified input using the given AttributeFactory