Class DictionaryBuilder

  • public class DictionaryBuilder
    extends Object
    Tool to build dictionaries. Usage:
        java -cp [lucene classpath] org.apache.lucene.analysis.ja.util.DictionaryBuilder \
              ${inputDir} ${outputDir} ${encoding} ${normalizeEntry}

    The input directory is expected to include unk.def, matrix.def, plus any number of .csv files, roughly following the conventions of IPADIC. JapaneseTokenizer uses dictionaries built with this tool. Note that the input files required by this build generally must be generated from a corpus of real text using tools that are not part of Lucene.

    The normalizeEntry option is a Boolean value.
    If true, check a surface form (first column in csv) is NFC Normalized. If it isn't, NFC normalized contents will be added to the TokenInfoDictionary in addition to the original form.
    This option is false for pre-built dictionary in the Lucene.

    WARNING: This API is experimental and might change in incompatible ways in the next release.