Package org.apache.lucene.analysis.bn
Class BengaliAnalyzer
- java.lang.Object
-
- org.apache.lucene.analysis.Analyzer
-
- org.apache.lucene.analysis.StopwordAnalyzerBase
-
- org.apache.lucene.analysis.bn.BengaliAnalyzer
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public final class BengaliAnalyzer extends StopwordAnalyzerBase
Analyzer for Bengali.- Since:
- 7.1.0
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
-
-
Field Summary
Fields Modifier and Type Field Description static String
DEFAULT_STOPWORD_FILE
File containing default Bengali stopwords.-
Fields inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
stopwords
-
Fields inherited from class org.apache.lucene.analysis.Analyzer
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
-
-
Constructor Summary
Constructors Constructor Description BengaliAnalyzer()
Builds an analyzer with the default stop words:DEFAULT_STOPWORD_FILE
.BengaliAnalyzer(CharArraySet stopwords)
Builds an analyzer with the given stop wordsBengaliAnalyzer(CharArraySet stopwords, CharArraySet stemExclusionSet)
Builds an analyzer with the given stop words
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected Analyzer.TokenStreamComponents
createComponents(String fieldName)
CreatesAnalyzer.TokenStreamComponents
used to tokenize all the text in the providedReader
.static CharArraySet
getDefaultStopSet()
Returns an unmodifiable instance of the default stop-words set.protected TokenStream
normalize(String fieldName, TokenStream in)
-
Methods inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
getStopwordSet, loadStopwordSet, loadStopwordSet, loadStopwordSet
-
Methods inherited from class org.apache.lucene.analysis.Analyzer
attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, initReader, initReaderForNormalization, normalize, tokenStream, tokenStream
-
-
-
-
Field Detail
-
DEFAULT_STOPWORD_FILE
public static final String DEFAULT_STOPWORD_FILE
File containing default Bengali stopwords.Default stopword list is from http://members.unine.ch/jacques.savoy/clef/bengaliST.txt The stopword list is BSD-Licensed.
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
BengaliAnalyzer
public BengaliAnalyzer(CharArraySet stopwords, CharArraySet stemExclusionSet)
Builds an analyzer with the given stop words- Parameters:
stopwords
- a stopword setstemExclusionSet
- a stemming exclusion set
-
BengaliAnalyzer
public BengaliAnalyzer(CharArraySet stopwords)
Builds an analyzer with the given stop words- Parameters:
stopwords
- a stopword set
-
BengaliAnalyzer
public BengaliAnalyzer()
Builds an analyzer with the default stop words:DEFAULT_STOPWORD_FILE
.
-
-
Method Detail
-
getDefaultStopSet
public static CharArraySet getDefaultStopSet()
Returns an unmodifiable instance of the default stop-words set.- Returns:
- an unmodifiable instance of the default stop-words set.
-
createComponents
protected Analyzer.TokenStreamComponents createComponents(String fieldName)
CreatesAnalyzer.TokenStreamComponents
used to tokenize all the text in the providedReader
.- Specified by:
createComponents
in classAnalyzer
- Returns:
Analyzer.TokenStreamComponents
built from aStandardTokenizer
filtered withLowerCaseFilter
,DecimalDigitFilter
,IndicNormalizationFilter
,BengaliNormalizationFilter
,SetKeywordMarkerFilter
if a stem exclusion set is provided,BengaliStemFilter
, and Bengali Stop words
-
normalize
protected TokenStream normalize(String fieldName, TokenStream in)
-
-