Package org.apache.lucene.tests.analysis
Class LookaheadTokenFilter<T extends LookaheadTokenFilter.Position>
- java.lang.Object
-
- org.apache.lucene.util.AttributeSource
-
- org.apache.lucene.analysis.TokenStream
-
- org.apache.lucene.analysis.TokenFilter
-
- org.apache.lucene.tests.analysis.LookaheadTokenFilter<T>
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Unwrappable<TokenStream>
- Direct Known Subclasses:
MockGraphTokenFilter
,MockRandomLookaheadTokenFilter
public abstract class LookaheadTokenFilter<T extends LookaheadTokenFilter.Position> extends TokenFilter
An abstract TokenFilter to make it easier to build graph token filters requiring some lookahead. This class handles the details of buffering up tokens, recording them by position, restoring them, providing access to them, etc.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
LookaheadTokenFilter.Position
Holds all state for a single position; subclass this to record other state at each position.-
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.State
-
-
Field Summary
Fields Modifier and Type Field Description protected boolean
end
protected int
inputPos
protected OffsetAttribute
offsetAtt
protected int
outputPos
protected PositionIncrementAttribute
posIncAtt
protected RollingBuffer<T>
positions
protected PositionLengthAttribute
posLenAtt
-
Fields inherited from class org.apache.lucene.analysis.TokenFilter
input
-
Fields inherited from class org.apache.lucene.analysis.TokenStream
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
LookaheadTokenFilter(TokenStream input)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected void
afterPosition()
This is called when all input tokens leaving a given position have been returned.protected void
insertToken()
Call this only from within afterPosition, to insert a new token.protected abstract T
newPosition()
protected boolean
nextToken()
Call this when you are done looking ahead; it will set the next token to return.protected boolean
peekToken()
Returns true if there is a new token.void
reset()
-
Methods inherited from class org.apache.lucene.analysis.TokenFilter
close, end, unwrap
-
Methods inherited from class org.apache.lucene.analysis.TokenStream
incrementToken
-
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
-
-
-
-
Field Detail
-
posIncAtt
protected final PositionIncrementAttribute posIncAtt
-
posLenAtt
protected final PositionLengthAttribute posLenAtt
-
offsetAtt
protected final OffsetAttribute offsetAtt
-
inputPos
protected int inputPos
-
outputPos
protected int outputPos
-
end
protected boolean end
-
positions
protected final RollingBuffer<T extends LookaheadTokenFilter.Position> positions
-
-
Constructor Detail
-
LookaheadTokenFilter
protected LookaheadTokenFilter(TokenStream input)
-
-
Method Detail
-
insertToken
protected void insertToken() throws IOException
Call this only from within afterPosition, to insert a new token. After calling this you should set any necessary token you need.- Throws:
IOException
-
afterPosition
protected void afterPosition() throws IOException
This is called when all input tokens leaving a given position have been returned. Override this and call insertToken and then set whichever token's attributes you want, if you want to inject a token starting from this position.- Throws:
IOException
-
newPosition
protected abstract T newPosition()
-
peekToken
protected boolean peekToken() throws IOException
Returns true if there is a new token.- Throws:
IOException
-
nextToken
protected boolean nextToken() throws IOException
Call this when you are done looking ahead; it will set the next token to return. Return the boolean back to the caller.- Throws:
IOException
-
reset
public void reset() throws IOException
- Overrides:
reset
in classTokenFilter
- Throws:
IOException
-
-