org.apache.lucene.index.memory
Class SynonymTokenFilter
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.index.memory.SynonymTokenFilter
public class SynonymTokenFilter
- extends org.apache.lucene.analysis.TokenFilter
Injects additional tokens for synonyms of token terms fetched from the
underlying child stream; the child stream must deliver lowercase tokens
for synonyms to be found.
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource |
org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State |
Fields inherited from class org.apache.lucene.analysis.TokenFilter |
input |
Constructor Summary |
SynonymTokenFilter(org.apache.lucene.analysis.TokenStream input,
SynonymMap synonyms,
int maxSynonyms)
Creates an instance for the given underlying stream and synonym table. |
Method Summary |
protected boolean |
createToken(String synonym,
org.apache.lucene.util.AttributeSource.State current)
Creates and returns a token for the given synonym of the current input
token; Override for custom (stateless or stateful) behavior, if desired. |
boolean |
incrementToken()
Returns the next token in the stream, or null at EOS. |
void |
reset()
|
Methods inherited from class org.apache.lucene.analysis.TokenFilter |
close, end |
Methods inherited from class org.apache.lucene.analysis.TokenStream |
getOnlyUseNewAPI, next, next, setOnlyUseNewAPI |
Methods inherited from class org.apache.lucene.util.AttributeSource |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString |
SYNONYM_TOKEN_TYPE
public static final String SYNONYM_TOKEN_TYPE
- The Token.type used to indicate a synonym to higher level filters.
- See Also:
- Constant Field Values
SynonymTokenFilter
public SynonymTokenFilter(org.apache.lucene.analysis.TokenStream input,
SynonymMap synonyms,
int maxSynonyms)
- Creates an instance for the given underlying stream and synonym table.
- Parameters:
input
- the underlying child token streamsynonyms
- the map used to extract synonyms for termsmaxSynonyms
- the maximum number of synonym tokens to return per underlying
token word (a value of Integer.MAX_VALUE indicates unlimited)
incrementToken
public final boolean incrementToken()
throws IOException
- Returns the next token in the stream, or null at EOS.
- Overrides:
incrementToken
in class org.apache.lucene.analysis.TokenStream
- Throws:
IOException
createToken
protected boolean createToken(String synonym,
org.apache.lucene.util.AttributeSource.State current)
- Creates and returns a token for the given synonym of the current input
token; Override for custom (stateless or stateful) behavior, if desired.
- Parameters:
synonym
- a synonym for the current token's termcurrent
- the current token from the underlying child stream
- Returns:
- a new token, or null to indicate that the given synonym should be
ignored
reset
public void reset()
throws IOException
- Overrides:
reset
in class org.apache.lucene.analysis.TokenFilter
- Throws:
IOException
Copyright © 2000-2010 Apache Software Foundation. All Rights Reserved.