public final class FlattenGraphFilter extends TokenFilter
SynonymGraphFilter
, into a flat form so that
all nodes form a single linear chain with no side paths. Every
path through the graph touches every node. This is necessary
when indexing a graph token stream, because the index does not
save PositionLengthAttribute
and so it cannot
preserve the graph structure. However, at search time,
query parsers can correctly handle the graph and this token
filter should not be used.
If the graph was not already flat to start, this is likely a lossy process, i.e. it will often cause the graph to accept token sequences it should not, and to reject token sequences it should not.
However, when applying synonyms during indexing, this
is necessary because Lucene already does not index a graph
and so the indexing process is already lossy
(it ignores the PositionLengthAttribute
).
AttributeSource.State
input
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
Constructor and Description |
---|
FlattenGraphFilter(TokenStream in) |
Modifier and Type | Method and Description |
---|---|
void |
end() |
int |
getMaxLookaheadUsed()
For testing
|
boolean |
incrementToken() |
void |
reset() |
close
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
public FlattenGraphFilter(TokenStream in)
public boolean incrementToken() throws IOException
incrementToken
in class TokenStream
IOException
public void end() throws IOException
end
in class TokenFilter
IOException
public void reset() throws IOException
reset
in class TokenFilter
IOException
public int getMaxLookaheadUsed()
Copyright © 2000-2017 Apache Software Foundation. All Rights Reserved.