Package org.apache.lucene.analysis.ar
Class ArabicAnalyzer
java.lang.Object
org.apache.lucene.analysis.Analyzer
org.apache.lucene.analysis.StopwordAnalyzerBase
org.apache.lucene.analysis.ar.ArabicAnalyzer
- All Implemented Interfaces:
Closeable
,AutoCloseable
Analyzer
for Arabic.
This analyzer implements light-stemming as specified by: Light Stemming for Arabic Information Retrieval http://www.mtholyoke.edu/~lballest/Pubs/arab_stem05.pdf
The analysis package contains three primary components:
ArabicNormalizationFilter
: Arabic orthographic normalization.ArabicStemFilter
: Arabic light stemming- Arabic stop words file: a set of default Arabic stop words.
- Since:
- 3.1
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
-
Field Summary
Fields inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
stopwords
Fields inherited from class org.apache.lucene.analysis.Analyzer
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
-
Constructor Summary
ConstructorDescriptionBuilds an analyzer with the default stop words:DEFAULT_STOPWORD_FILE
.ArabicAnalyzer
(CharArraySet stopwords) Builds an analyzer with the given stop wordsArabicAnalyzer
(CharArraySet stopwords, CharArraySet stemExclusionSet) Builds an analyzer with the given stop word. -
Method Summary
Modifier and TypeMethodDescriptionprotected Analyzer.TokenStreamComponents
createComponents
(String fieldName) CreatesAnalyzer.TokenStreamComponents
used to tokenize all the text in the providedReader
.static CharArraySet
Returns an unmodifiable instance of the default stop-words set.protected TokenStream
normalize
(String fieldName, TokenStream in) Methods inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
getStopwordSet, loadStopwordSet, loadStopwordSet, loadStopwordSet
Methods inherited from class org.apache.lucene.analysis.Analyzer
attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, initReader, initReaderForNormalization, normalize, tokenStream, tokenStream
-
Field Details
-
DEFAULT_STOPWORD_FILE
File containing default Arabic stopwords.Default stopword list is from http://members.unine.ch/jacques.savoy/clef/index.html The stopword list is BSD-Licensed.
- See Also:
-
-
Constructor Details
-
ArabicAnalyzer
public ArabicAnalyzer()Builds an analyzer with the default stop words:DEFAULT_STOPWORD_FILE
. -
ArabicAnalyzer
Builds an analyzer with the given stop words- Parameters:
stopwords
- a stopword set
-
ArabicAnalyzer
Builds an analyzer with the given stop word. If a none-empty stem exclusion set is provided this analyzer will add aSetKeywordMarkerFilter
beforeArabicStemFilter
.- Parameters:
stopwords
- a stopword setstemExclusionSet
- a set of terms not to be stemmed
-
-
Method Details
-
getDefaultStopSet
Returns an unmodifiable instance of the default stop-words set.- Returns:
- an unmodifiable instance of the default stop-words set.
-
createComponents
CreatesAnalyzer.TokenStreamComponents
used to tokenize all the text in the providedReader
.- Specified by:
createComponents
in classAnalyzer
- Returns:
Analyzer.TokenStreamComponents
built from anStandardTokenizer
filtered withLowerCaseFilter
,DecimalDigitFilter
,StopFilter
,ArabicNormalizationFilter
,SetKeywordMarkerFilter
if a stem exclusion set is provided andArabicStemFilter
.
-
normalize
-