Interface Fragmenter

  • All Known Implementing Classes:
    NullFragmenter, SimpleFragmenter, SimpleSpanFragmenter

    public interface Fragmenter
    Implements the policy for breaking text into multiple fragments for consideration by the Highlighter class. A sophisticated implementation may do this on the basis of detecting end of sentences in the text.
    • Method Detail

      • start

        void start​(String originalText,
                   TokenStream tokenStream)
        Initializes the Fragmenter. You can grab references to the Attributes you are interested in from tokenStream and then access the values in isNewFragment().
        Parameters:
        originalText - the original source text
        tokenStream - the TokenStream to be fragmented
      • isNewFragment

        boolean isNewFragment()
        Test to see if this token from the stream should be held in a new TextFragment. Every time this is called, the TokenStream passed to start(String, TokenStream) will have been incremented.