StandardTokenizer
instead, which has the same functionality.
This filter will be removed in Lucene 5.0@Deprecated public class RussianLetterTokenizer extends CharTokenizer
Tokenizer
that extends LetterTokenizer
by also allowing the basic Latin digits 0-9.
You must specify the required Version
compatibility when creating
RussianLetterTokenizer
:
CharTokenizer
uses an int based API to normalize and
detect token characters. See CharTokenizer.isTokenChar(int)
and
CharTokenizer.normalize(int)
for details.AttributeSource.State
DEFAULT_TOKEN_ATTRIBUTE_FACTORY
DEFAULT_ATTRIBUTE_FACTORY
Constructor and Description |
---|
RussianLetterTokenizer(Version matchVersion,
AttributeFactory factory,
Reader in)
Deprecated.
Construct a new RussianLetterTokenizer using a given
AttributeFactory . |
RussianLetterTokenizer(Version matchVersion,
Reader in)
Deprecated.
Construct a new RussianLetterTokenizer.
|
Modifier and Type | Method and Description |
---|---|
protected boolean |
isTokenChar(int c)
Deprecated.
Collects only characters which satisfy
Character.isLetter(int) . |
end, incrementToken, normalize, reset
close, correctOffset, setReader
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
public RussianLetterTokenizer(Version matchVersion, Reader in)
in
- the input to split up into tokenspublic RussianLetterTokenizer(Version matchVersion, AttributeFactory factory, Reader in)
AttributeFactory
. * @param
matchVersion Lucene version to match See
abovefactory
- the attribute factory to use for this Tokenizer
in
- the input to split up into tokensprotected boolean isTokenChar(int c)
Character.isLetter(int)
.isTokenChar
in class CharTokenizer
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.