|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use TokenStream | |
---|---|
org.apache.lucene.analysis | API and code to convert text into indexable/searchable tokens. |
org.apache.lucene.analysis.standard | A fast grammar-based tokenizer constructed with JFlex. |
org.apache.lucene.document | The logical representation of a Document for indexing and searching. |
Uses of TokenStream in org.apache.lucene.analysis |
---|
Subclasses of TokenStream in org.apache.lucene.analysis | |
---|---|
class |
ASCIIFoldingFilter
This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists. |
class |
CachingTokenFilter
This class can be used if the token attributes of a TokenStream are intended to be consumed more than once. |
class |
CharTokenizer
An abstract base class for simple, character-oriented tokenizers. |
class |
ISOLatin1AccentFilter
Deprecated. in favor of ASCIIFoldingFilter which covers a superset
of Latin 1. This class will be removed in Lucene 3.0. |
class |
KeywordTokenizer
Emits the entire input as a single token. |
class |
LengthFilter
Removes words that are too long or too short from the stream. |
class |
LetterTokenizer
A LetterTokenizer is a tokenizer that divides text at non-letters. |
class |
LowerCaseFilter
Normalizes token text to lower case. |
class |
LowerCaseTokenizer
LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. |
class |
NumericTokenStream
Expert: This class provides a TokenStream
for indexing numeric values that can be used by NumericRangeQuery or NumericRangeFilter . |
class |
PorterStemFilter
Transforms the token stream as per the Porter stemming algorithm. |
class |
SinkTokenizer
Deprecated. Use TeeSinkTokenFilter instead |
class |
StopFilter
Removes stop words from a token stream. |
class |
TeeSinkTokenFilter
This TokenFilter provides the ability to set aside attribute states that have already been analyzed. |
static class |
TeeSinkTokenFilter.SinkTokenStream
|
class |
TeeTokenFilter
Deprecated. Use TeeSinkTokenFilter instead |
class |
TokenFilter
A TokenFilter is a TokenStream whose input is another TokenStream. |
class |
Tokenizer
A Tokenizer is a TokenStream whose input is a Reader. |
class |
WhitespaceTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. |
Fields in org.apache.lucene.analysis declared as TokenStream | |
---|---|
protected TokenStream |
TokenFilter.input
The source of tokens for this filter. |
Methods in org.apache.lucene.analysis that return TokenStream | |
---|---|
TokenStream |
WhitespaceAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
StopAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
SimpleAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
PerFieldAnalyzerWrapper.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
KeywordAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
Analyzer.reusableTokenStream(String fieldName,
Reader reader)
Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method. |
TokenStream |
WhitespaceAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
StopAnalyzer.tokenStream(String fieldName,
Reader reader)
Filters LowerCaseTokenizer with StopFilter. |
TokenStream |
SimpleAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
PerFieldAnalyzerWrapper.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
KeywordAnalyzer.tokenStream(String fieldName,
Reader reader)
|
abstract TokenStream |
Analyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader. |
Constructors in org.apache.lucene.analysis with parameters of type TokenStream | |
---|---|
ASCIIFoldingFilter(TokenStream input)
|
|
CachingTokenFilter(TokenStream input)
|
|
ISOLatin1AccentFilter(TokenStream input)
Deprecated. |
|
LengthFilter(TokenStream in,
int min,
int max)
Build a filter that removes words that are too long or too short from the text. |
|
LowerCaseFilter(TokenStream in)
|
|
PorterStemFilter(TokenStream in)
|
|
StopFilter(boolean enablePositionIncrements,
TokenStream in,
Set stopWords)
Constructs a filter which removes words from the input TokenStream that are named in the Set. |
|
StopFilter(boolean enablePositionIncrements,
TokenStream input,
Set stopWords,
boolean ignoreCase)
Construct a token stream filtering the given input. |
|
StopFilter(boolean enablePositionIncrements,
TokenStream input,
String[] stopWords)
Deprecated. Use StopFilter.StopFilter(boolean, TokenStream, Set) instead. |
|
StopFilter(boolean enablePositionIncrements,
TokenStream in,
String[] stopWords,
boolean ignoreCase)
Deprecated. Use StopFilter.StopFilter(boolean, TokenStream, Set, boolean) instead. |
|
StopFilter(TokenStream in,
Set stopWords)
Deprecated. Use StopFilter.StopFilter(boolean, TokenStream, Set) instead |
|
StopFilter(TokenStream input,
Set stopWords,
boolean ignoreCase)
Deprecated. Use StopFilter.StopFilter(boolean, TokenStream, Set, boolean) instead |
|
StopFilter(TokenStream input,
String[] stopWords)
Deprecated. Use StopFilter.StopFilter(boolean, TokenStream, String[]) instead |
|
StopFilter(TokenStream in,
String[] stopWords,
boolean ignoreCase)
Deprecated. Use StopFilter.StopFilter(boolean, TokenStream, String[], boolean) instead |
|
TeeSinkTokenFilter(TokenStream input)
Instantiates a new TeeSinkTokenFilter. |
|
TeeTokenFilter(TokenStream input,
SinkTokenizer sink)
Deprecated. |
|
TokenFilter(TokenStream input)
Construct a token stream filtering the given input. |
Uses of TokenStream in org.apache.lucene.analysis.standard |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.standard | |
---|---|
class |
StandardFilter
Normalizes tokens extracted with StandardTokenizer . |
class |
StandardTokenizer
A grammar-based tokenizer constructed with JFlex |
Methods in org.apache.lucene.analysis.standard that return TokenStream | |
---|---|
TokenStream |
StandardAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Deprecated. Use StandardAnalyzer.tokenStream(java.lang.String, java.io.Reader) instead |
TokenStream |
StandardAnalyzer.tokenStream(String fieldName,
Reader reader)
Constructs a StandardTokenizer filtered by a StandardFilter , a LowerCaseFilter and a StopFilter . |
Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream | |
---|---|
StandardFilter(TokenStream in)
Construct filtering in. |
Uses of TokenStream in org.apache.lucene.document |
---|
Fields in org.apache.lucene.document declared as TokenStream | |
---|---|
protected TokenStream |
AbstractField.tokenStream
|
Methods in org.apache.lucene.document that return TokenStream | |
---|---|
TokenStream |
NumericField.tokenStreamValue()
Returns a NumericTokenStream for indexing the numeric value. |
TokenStream |
Field.tokenStreamValue()
The TokesStream for this field to be used when indexing, or null. |
TokenStream |
Fieldable.tokenStreamValue()
The TokenStream for this field to be used when indexing, or null. |
Methods in org.apache.lucene.document with parameters of type TokenStream | |
---|---|
void |
Field.setTokenStream(TokenStream tokenStream)
Expert: sets the token stream to be used for indexing and causes isIndexed() and isTokenized() to return true. |
void |
Field.setValue(TokenStream value)
Deprecated. use Field.setTokenStream(org.apache.lucene.analysis.TokenStream) |
Constructors in org.apache.lucene.document with parameters of type TokenStream | |
---|---|
Field(String name,
TokenStream tokenStream)
Create a tokenized and indexed field that is not stored. |
|
Field(String name,
TokenStream tokenStream,
Field.TermVector termVector)
Create a tokenized and indexed field that is not stored, optionally with storing term vectors. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |