Class DictionaryBuilder
java.lang.Object
org.apache.lucene.analysis.ja.util.DictionaryBuilder
Tool to build dictionaries. Usage:
java -cp [lucene classpath] org.apache.lucene.analysis.ja.util.DictionaryBuilder \ ${inputDir} ${outputDir} ${encoding} ${normalizeEntry}
The input directory is expected to include unk.def, matrix.def, plus any number of .csv files, roughly following the conventions of IPADIC. JapaneseTokenizer uses dictionaries built with this tool. Note that the input files required by this build generally must be generated from a corpus of real text using tools that are not part of Lucene.
The normalizeEntry option is a Boolean value.
If true, check a surface form (first column in csv) is NFC Normalized. If it isn't, NFC
normalized contents will be added to the TokenInfoDictionary in addition to the original form.
This option is false for pre-built dictionary in the Lucene.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic enum
Format of the dictionary. -
Method Summary
-
Method Details
-
build
public static void build(DictionaryBuilder.DictionaryFormat format, Path inputDir, Path outputDir, String encoding, boolean normalizeEntry) throws IOException - Throws:
IOException
-
main
- Throws:
IOException
-