public class LuceneCarrot2TokenizerFactory
An implementation of Carrot2's ITokenizerFactory based on Lucene's
Smart Chinese tokenizer. If Smart Chinese tokenizer is not available in
classpath at runtime, the default Carrot2's tokenizer is used. Should the
Lucene APIs need to change, the changes can be made in this class.