AttributeSource.captureState()
and AttributeSource.restoreState(State)
which support all attributes.@Deprecated public abstract class BufferedTokenStream extends org.apache.lucene.analysis.TokenFilter
// Example of a class implementing the rule "A" "B" => "Q" "B" class MyTokenStream extends BufferedTokenStream { public MyTokenStream(TokenStream input) {super(input);} protected Token process(Token t) throws IOException { if ("A".equals(t.termText())) { Token t2 = read(); if (t2!=null && "B".equals(t2.termText())) t.setTermText("Q"); if (t2!=null) pushBack(t2); } return t; } } // Example of a class implementing "A" "B" => "A" "A" "B" class MyTokenStream extends BufferedTokenStream { public MyTokenStream(TokenStream input) {super(input);} protected Token process(Token t) throws IOException { if ("A".equals(t.termText()) && "B".equals(peek(1).termText())) write((Token)t.clone()); return t; } }NOTE: BufferedTokenStream does not clone() any Tokens. This is instead the responsibility of the implementing subclass. In the "A" "B" => "A" "A" "B" example above, the subclass must clone the additional "A" it creates.
Constructor and Description |
---|
BufferedTokenStream(org.apache.lucene.analysis.TokenStream input)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
boolean |
incrementToken()
Deprecated.
|
protected Iterable<org.apache.lucene.analysis.Token> |
output()
Deprecated.
Provides direct Iterator access to the buffered output stream.
|
protected org.apache.lucene.analysis.Token |
peek(int n)
Deprecated.
Peek n tokens ahead in the buffered input stream, without modifying
the stream.
|
protected abstract org.apache.lucene.analysis.Token |
process(org.apache.lucene.analysis.Token t)
Deprecated.
Process a token.
|
protected void |
pushBack(org.apache.lucene.analysis.Token t)
Deprecated.
Push a token back into the buffered input stream, such that it will
be returned by a future call to
read() |
protected org.apache.lucene.analysis.Token |
read()
Deprecated.
Read a token from the buffered input stream.
|
void |
reset()
Deprecated.
|
protected void |
write(org.apache.lucene.analysis.Token t)
Deprecated.
Write a token to the buffered output stream
|
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public BufferedTokenStream(org.apache.lucene.analysis.TokenStream input)
protected abstract org.apache.lucene.analysis.Token process(org.apache.lucene.analysis.Token t) throws IOException
IOException
public final boolean incrementToken() throws IOException
incrementToken
in class org.apache.lucene.analysis.TokenStream
IOException
protected org.apache.lucene.analysis.Token read() throws IOException
IOException
protected void pushBack(org.apache.lucene.analysis.Token t)
read()
protected org.apache.lucene.analysis.Token peek(int n) throws IOException
n
- Number of tokens into the input stream to peek, 1 based ...
0 is invalidread()
from the stream.IOException
protected void write(org.apache.lucene.analysis.Token t)
protected Iterable<org.apache.lucene.analysis.Token> output()
public void reset() throws IOException
reset
in class org.apache.lucene.analysis.TokenFilter
IOException