org.apache.lucene.analysis.cjk
Class CJKAnalyzer

java.lang.Object
  org.apache.lucene.analysis.Analyzer
      org.apache.lucene.analysis.cjk.CJKAnalyzer

public class CJKAnalyzer
extends org.apache.lucene.analysis.Analyzer

An Analyzer that tokenizes text with CJKTokenizer and filters with StopFilter

Field Summary

static String[]

STOP_WORDS An array containing some common English words that are not usually useful for searching and some double-byte interpunctions.

Fields inherited from class org.apache.lucene.analysis.Analyzer

overridesTokenStreamMethod

Constructor Summary

CJKAnalyzer()
Deprecated. Use CJKAnalyzer(Version) instead

CJKAnalyzer(String[] stopWords)
Deprecated. Use CJKAnalyzer(Version, String[]) instead

CJKAnalyzer(org.apache.lucene.util.Version matchVersion)
Builds an analyzer which removes words in STOP_WORDS.

CJKAnalyzer(org.apache.lucene.util.Version matchVersion, String[] stopWords)
Builds an analyzer which removes words in the provided array.

Method Summary

org.apache.lucene.analysis.TokenStream	reusableTokenStream(String fieldName, Reader reader) Returns a (possibly reused) TokenStream which tokenizes all the text in the provided Reader.
org.apache.lucene.analysis.TokenStream	tokenStream(String fieldName, Reader reader) Creates a TokenStream which tokenizes all the text in the provided Reader.

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getOffsetGap, getPositionIncrementGap, getPreviousTokenStream, setOverridesTokenStreamMethod, setPreviousTokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STOP_WORDS

public static final String[] STOP_WORDS

An array containing some common English words that are not usually useful for searching and some double-byte interpunctions.

Constructor Detail

CJKAnalyzer

public CJKAnalyzer()

Deprecated. Use CJKAnalyzer(Version) instead

Builds an analyzer which removes words in STOP_WORDS.

CJKAnalyzer

public CJKAnalyzer(org.apache.lucene.util.Version matchVersion)

Builds an analyzer which removes words in STOP_WORDS.

CJKAnalyzer

public CJKAnalyzer(String[] stopWords)

Deprecated. Use CJKAnalyzer(Version, String[]) instead

Builds an analyzer which removes words in the provided array.

Parameters:

stopWords - stop word array

CJKAnalyzer

public CJKAnalyzer(org.apache.lucene.util.Version matchVersion,
                   String[] stopWords)

Builds an analyzer which removes words in the provided array.

Parameters:

stopWords - stop word array

Method Detail

tokenStream

public final org.apache.lucene.analysis.TokenStream tokenStream(String fieldName,
                                                                Reader reader)

Creates a TokenStream which tokenizes all the text in the provided Reader.

Specified by:

tokenStream in class org.apache.lucene.analysis.Analyzer

Parameters:

fieldName - lucene field name

reader - input Reader

Returns:

A TokenStream built from CJKTokenizer, filtered with StopFilter

reusableTokenStream

public final org.apache.lucene.analysis.TokenStream reusableTokenStream(String fieldName,
                                                                        Reader reader)
                                                                 throws IOException

Returns a (possibly reused) TokenStream which tokenizes all the text in the provided Reader.

Overrides:

reusableTokenStream in class org.apache.lucene.analysis.Analyzer

Parameters:

fieldName - lucene field name

reader - Input Reader

Returns:

A TokenStream built from CJKTokenizer, filtered with StopFilter

Throws:

IOException