Class LuceneCarrot2TokenizerFactory

  • All Implemented Interfaces:
    org.carrot2.text.linguistic.ITokenizerFactory

    public class LuceneCarrot2TokenizerFactory
    extends Object
    implements org.carrot2.text.linguistic.ITokenizerFactory
    An implementation of Carrot2's ITokenizerFactory based on Lucene's Smart Chinese tokenizer. If Smart Chinese tokenizer is not available in classpath at runtime, the default Carrot2's tokenizer is used. Should the Lucene APIs need to change, the changes can be made in this class.
    WARNING: This API is experimental and might change in incompatible ways in the next release.
    • Constructor Detail

      • LuceneCarrot2TokenizerFactory

        public LuceneCarrot2TokenizerFactory()
    • Method Detail

      • getTokenizer

        public org.carrot2.text.analysis.ITokenizer getTokenizer​(org.carrot2.core.LanguageCode language)
        Specified by:
        getTokenizer in interface org.carrot2.text.linguistic.ITokenizerFactory