org.apache.lucene.analysis.miscellaneous
Class ASCIIFoldingFilter
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.miscellaneous.ASCIIFoldingFilter
- All Implemented Interfaces:
- Closeable
public final class ASCIIFoldingFilter
- extends TokenFilter
This class converts alphabetic, numeric, and symbolic Unicode characters
which are not in the first 127 ASCII characters (the "Basic Latin" Unicode
block) into their ASCII equivalents, if one exists.
Characters from the following Unicode blocks are converted; however, only
those characters with reasonable ASCII alternatives are converted:
See: http://en.wikipedia.org/wiki/Latin_characters_in_Unicode
For example, 'à' will be replaced by 'a'.
Method Summary |
void |
foldToASCII(char[] input,
int length)
Converts characters above ASCII to their ASCII equivalents. |
static int |
foldToASCII(char[] input,
int inputPos,
char[] output,
int outputPos,
int length)
Converts characters above ASCII to their ASCII equivalents. |
boolean |
incrementToken()
|
boolean |
isPreserveOriginal()
Does the filter preserve the original tokens? |
void |
reset()
|
Methods inherited from class org.apache.lucene.util.AttributeSource |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString |
ASCIIFoldingFilter
public ASCIIFoldingFilter(TokenStream input)
ASCIIFoldingFilter
public ASCIIFoldingFilter(TokenStream input,
boolean preserveOriginal)
- Create a new
ASCIIFoldingFilter
.
- Parameters:
input
- TokenStream to filterpreserveOriginal
- should the original tokens be kept on the input stream with a 0 position increment
from the folded tokens?
isPreserveOriginal
public boolean isPreserveOriginal()
- Does the filter preserve the original tokens?
incrementToken
public boolean incrementToken()
throws IOException
- Specified by:
incrementToken
in class TokenStream
- Throws:
IOException
reset
public void reset()
throws IOException
- Overrides:
reset
in class TokenFilter
- Throws:
IOException
foldToASCII
public void foldToASCII(char[] input,
int length)
- Converts characters above ASCII to their ASCII equivalents. For example,
accents are removed from accented characters.
- Parameters:
input
- The string to foldlength
- The number of characters in the input string
foldToASCII
public static final int foldToASCII(char[] input,
int inputPos,
char[] output,
int outputPos,
int length)
- Converts characters above ASCII to their ASCII equivalents. For example,
accents are removed from accented characters.
- Parameters:
input
- The characters to foldinputPos
- Index of the first character to foldoutput
- The result of the folding. Should be of size >= length * 4
.outputPos
- Index of output where to put the result of the foldinglength
- The number of characters to fold
- Returns:
- length of output
- NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.