public final class ICUCollationKeyFilter extends TokenFilter
Converts each token into its
then encodes the CollationKey with
allow it to be stored as an index term.
WARNING: Make sure you use exactly the same Collator at
index and query time -- CollationKeys are only comparable when produced by
the same Collator.
independently versioned, so it is safe to search against stored
CollationKeys if the following are exactly the same (best practice is
to store this information with the index and check that they remain the
same at query time):
CollationKeys generated by ICU Collators are not compatible with those
generated by java.text.Collators. Specifically, if you use
ICUCollationKeyFilter to generate index terms, do not use
CollationKeyFilter on the query side, or vice versa.
ICUCollationKeyFilter is significantly faster and generates significantly shorter keys than CollationKeyFilter. See http://site.icu-project.org/charts/collation-icu4j-sun for key generation timing and key length comparisons between ICU4J and java.text.Collator over several languages.
|Constructor and Description|
|Modifier and Type||Method and Description|
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public ICUCollationKeyFilter(TokenStream input, com.ibm.icu.text.Collator collator)
input- Source token stream
collator- CollationKey generator
public boolean incrementToken() throws IOException
IndexWriter) use this method to advance the stream to the next token. Implementing classes must implement this method and update the appropriate
AttributeImpls with the attributes of the next token.
The producer must make no assumptions about the attributes after the method
has been returned: the caller may arbitrarily change it. If the producer
needs to preserve the state for subsequent calls, it can use
AttributeSource.captureState() to create a copy of the current attribute state.
This method is called for every token of a document, so an efficient
implementation is crucial for good performance. To avoid calls to
references to all
AttributeImpls that this stream uses should be
retrieved during instantiation.
To ensure that filters and consumers know which attributes are available,
the attributes must be added during instantiation. Filters and consumers
are not required to check for availability of attributes in