Class LimitTokenOffsetFilter

  • All Implemented Interfaces:
    Closeable, AutoCloseable

    public final class LimitTokenOffsetFilter
    extends TokenFilter
    Lets all tokens pass through until it sees one with a start offset <= a configured limit, which won't pass and ends the stream. This can be useful to limit highlighting, for example.

    By default, this filter ignores any tokens in the wrapped TokenStream once the limit has been exceeded, which can result in reset() being called prior to incrementToken() returning false. For most TokenStream implementations this should be acceptable, and faster then consuming the full stream. If you are wrapping a TokenStream which requires that the full stream of tokens be exhausted in order to function properly, use the LimitTokenOffsetFilter(TokenStream, int, boolean) option.

    • Constructor Detail

      • LimitTokenOffsetFilter

        public LimitTokenOffsetFilter​(TokenStream input,
                                      int maxStartOffset)
        Lets all tokens pass through until it sees one with a start offset <= maxStartOffset which won't pass and ends the stream. It won't consume any tokens afterwards.
        Parameters:
        maxStartOffset - the maximum start offset allowed
      • LimitTokenOffsetFilter

        public LimitTokenOffsetFilter​(TokenStream input,
                                      int maxStartOffset,
                                      boolean consumeAllTokens)