org.apache.lucene.analysis
Class SinkTokenizer

java.lang.Object
  org.apache.lucene.util.AttributeSource
      org.apache.lucene.analysis.TokenStream
          org.apache.lucene.analysis.Tokenizer
              org.apache.lucene.analysis.SinkTokenizer

Direct Known Subclasses:

DateRecognizerSinkTokenizer, TokenRangeSinkTokenizer, TokenTypeSinkTokenizer

Deprecated. Use TeeSinkTokenFilter instead

public class SinkTokenizer
extends Tokenizer

A SinkTokenizer can be used to cache Tokens for use in an Analyzer

WARNING: TeeTokenFilter and SinkTokenizer only work with the old TokenStream API. If you switch to the new API, you need to use TeeSinkTokenFilter instead, which offers the same functionality.

See Also:

TeeTokenFilter

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

AttributeSource.AttributeFactory, AttributeSource.State

Field Summary

protected Iterator	iter Deprecated.
protected List	lst Deprecated.

Fields inherited from class org.apache.lucene.analysis.Tokenizer

input

Constructor Summary

SinkTokenizer()
Deprecated.

SinkTokenizer(int initCap)
Deprecated.

SinkTokenizer(List input)
Deprecated.

Method Summary

void	add(Token t) Deprecated. Override this method to cache only certain tokens, or new tokens based on the old tokens.
void	close() Deprecated. By default, closes the input Reader.
List	getTokens() Deprecated. Get the tokens in the internal List.
Token	next(Token reusableToken) Deprecated. Returns the next token out of the list of cached tokens
void	reset() Deprecated. Reset the internal data structures to the start at the front of the list of tokens.

Methods inherited from class org.apache.lucene.analysis.Tokenizer

correctOffset, reset

Methods inherited from class org.apache.lucene.analysis.TokenStream

end, getOnlyUseNewAPI, incrementToken, next, setOnlyUseNewAPI

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

lst

protected List lst

Deprecated.

iter

protected Iterator iter

Deprecated.

Constructor Detail

SinkTokenizer

public SinkTokenizer(List input)

Deprecated.

SinkTokenizer

public SinkTokenizer()

Deprecated.

SinkTokenizer

public SinkTokenizer(int initCap)

Deprecated.

Method Detail

getTokens

public List getTokens()

Deprecated.

Get the tokens in the internal List.

WARNING: Adding tokens to this list requires the reset() method to be called in order for them to be made available. Also, this Tokenizer does nothing to protect against ConcurrentModificationExceptions in the case of adds happening while next(org.apache.lucene.analysis.Token) is being called.

WARNING: Since this SinkTokenizer can be reset and the cached tokens made available again, do not modify them. Modify clones instead.

Returns:

A List of Tokens

next

public Token next(Token reusableToken)
           throws IOException

Deprecated.

Returns the next token out of the list of cached tokens

Overrides:

next in class TokenStream

Parameters:

reusableToken - a Token that may or may not be used to return; this parameter should never be null (the callee is not required to check for null before using it, but it is a good idea to assert that it is not null.)

Returns:

The next Token in the Sink.

Throws:

IOException

add

public void add(Token t)

Deprecated.

Override this method to cache only certain tokens, or new tokens based on the old tokens.

Parameters:

t - The Token to add to the sink

close

public void close()
           throws IOException

Deprecated.

Description copied from class: Tokenizer

By default, closes the input Reader.

Overrides:

close in class Tokenizer

Throws:

IOException

reset

public void reset()
           throws IOException

Deprecated.

Reset the internal data structures to the start at the front of the list of tokens. Should be called if tokens were added to the list after an invocation of next(Token)

Overrides:

reset in class TokenStream

Throws:

IOException