Class KoreanTokenizer

  • All Implemented Interfaces:
    Closeable, AutoCloseable

    public final class KoreanTokenizer
    extends Tokenizer
    Tokenizer for Korean that uses morphological analysis.

    This tokenizer sets a number of additional attributes:

    This tokenizer uses a rolling Viterbi search to find the least cost segmentation (path) of the incoming characters.

    WARNING: This API is experimental and might change in incompatible ways in the next release.