Package org.apache.lucene.analysis.core
Class FlattenGraphFilter
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.core.FlattenGraphFilter
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Unwrappable<TokenStream>
Converts an incoming graph token stream, such as one from
SynonymGraphFilter
, into a flat
form so that all nodes form a single linear chain with no side paths. Every path through the
graph touches every node. This is necessary when indexing a graph token stream, because the index
does not save PositionLengthAttribute
and so it cannot preserve the graph structure.
However, at search time, query parsers can correctly handle the graph and this token filter
should not be used.
If the graph was not already flat to start, this is likely a lossy process, i.e. it will often cause the graph to accept token sequences it should not, and to reject token sequences it should not.
However, when applying synonyms during indexing, this is necessary because Lucene already does
not index a graph and so the indexing process is already lossy (it ignores the PositionLengthAttribute
).
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.State
-
Field Summary
Fields inherited from class org.apache.lucene.analysis.TokenFilter
input
Fields inherited from class org.apache.lucene.analysis.TokenStream
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
-
Constructor Summary
-
Method Summary
Methods inherited from class org.apache.lucene.analysis.TokenFilter
close, unwrap
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
-
Constructor Details
-
FlattenGraphFilter
-
-
Method Details
-
incrementToken
- Specified by:
incrementToken
in classTokenStream
- Throws:
IOException
-
end
- Overrides:
end
in classTokenFilter
- Throws:
IOException
-
reset
- Overrides:
reset
in classTokenFilter
- Throws:
IOException
-
getMaxLookaheadUsed
public int getMaxLookaheadUsed()For testing
-