Package org.apache.lucene.analysis.nl
Class DutchAnalyzer
java.lang.Object
org.apache.lucene.analysis.Analyzer
org.apache.lucene.analysis.nl.DutchAnalyzer
- All Implemented Interfaces:
Closeable
,AutoCloseable
Analyzer
for Dutch language.
Supports an external list of stopwords (words that will not be indexed at all), an external list of exclusions (word that will not be stemmed, but indexed) and an external list of word-stem pairs that overrule the algorithm (dictionary stemming). A default set of stopwords is used unless an alternative list is specified, but the exclusion list is empty by default.
- Since:
- 3.1
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
-
Field Summary
Fields inherited from class org.apache.lucene.analysis.Analyzer
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
-
Constructor Summary
ConstructorDescriptionBuilds an analyzer with the default stop words (getDefaultStopSet()
) and a few default entries for the stem exclusion table.DutchAnalyzer
(CharArraySet stopwords) DutchAnalyzer
(CharArraySet stopwords, CharArraySet stemExclusionTable) DutchAnalyzer
(CharArraySet stopwords, CharArraySet stemExclusionTable, CharArrayMap<String> stemOverrideDict) -
Method Summary
Modifier and TypeMethodDescriptionprotected Analyzer.TokenStreamComponents
createComponents
(String fieldName) Returns a (possibly reused)TokenStream
which tokenizes all the text in the providedReader
.static CharArraySet
Returns an unmodifiable instance of the default stop-words set.protected TokenStream
normalize
(String fieldName, TokenStream in) Methods inherited from class org.apache.lucene.analysis.Analyzer
attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, initReader, initReaderForNormalization, normalize, tokenStream, tokenStream
-
Field Details
-
DEFAULT_STOPWORD_FILE
File containing default Dutch stopwords.- See Also:
-
-
Constructor Details
-
DutchAnalyzer
public DutchAnalyzer()Builds an analyzer with the default stop words (getDefaultStopSet()
) and a few default entries for the stem exclusion table. -
DutchAnalyzer
-
DutchAnalyzer
-
DutchAnalyzer
public DutchAnalyzer(CharArraySet stopwords, CharArraySet stemExclusionTable, CharArrayMap<String> stemOverrideDict)
-
-
Method Details
-
getDefaultStopSet
Returns an unmodifiable instance of the default stop-words set.- Returns:
- an unmodifiable instance of the default stop-words set.
-
createComponents
Returns a (possibly reused)TokenStream
which tokenizes all the text in the providedReader
.- Specified by:
createComponents
in classAnalyzer
- Returns:
- A
TokenStream
built from aStandardTokenizer
filtered withLowerCaseFilter
,StopFilter
,SetKeywordMarkerFilter
if a stem exclusion set is provided,StemmerOverrideFilter
, andSnowballFilter
-
normalize
-