Class CJKAnalyzer

All Implemented Interfaces:
Closeable, AutoCloseable

public final class CJKAnalyzer extends StopwordAnalyzerBase
An Analyzer that tokenizes text with StandardTokenizer, normalizes content with CJKWidthFilter, folds case with LowerCaseFilter, forms bigrams of CJK with CJKBigramFilter, and filters stopwords with StopFilter
Since:
3.1
  • Field Details

    • DEFAULT_STOPWORD_FILE

      public static final String DEFAULT_STOPWORD_FILE
      File containing default CJK stopwords.

      Currently it contains some common English words that are not usually useful for searching and some double-byte interpunctions.

      See Also:
  • Constructor Details

    • CJKAnalyzer

      public CJKAnalyzer()
      Builds an analyzer which removes words in getDefaultStopSet().
    • CJKAnalyzer

      public CJKAnalyzer(CharArraySet stopwords)
      Builds an analyzer with the given stop words
      Parameters:
      stopwords - a stopword set
  • Method Details