|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use TokenStream | |
---|---|
org.apache.lucene.analysis | API and code to convert text into indexable/searchable tokens. |
org.apache.lucene.analysis.ar | Analyzer for Arabic. |
org.apache.lucene.analysis.br | Analyzer for Brazilian Portuguese. |
org.apache.lucene.analysis.cjk | Analyzer for Chinese, Japanese, and Korean, which indexes bigrams (overlapping groups of two adjacent Han characters). |
org.apache.lucene.analysis.cn | Analyzer for Chinese, which indexes unigrams (individual chinese characters). |
org.apache.lucene.analysis.cn.smart |
Analyzer for Simplified Chinese, which indexes words. |
org.apache.lucene.analysis.compound | A filter that decomposes compound words you find in many Germanic languages into the word parts. |
org.apache.lucene.analysis.cz | Analyzer for Czech. |
org.apache.lucene.analysis.de | Analyzer for German. |
org.apache.lucene.analysis.el | Analyzer for Greek. |
org.apache.lucene.analysis.fa | Analyzer for Persian. |
org.apache.lucene.analysis.fr | Analyzer for French. |
org.apache.lucene.analysis.miscellaneous | Miscellaneous TokenStreams |
org.apache.lucene.analysis.ngram | Character n-gram tokenizers and filters. |
org.apache.lucene.analysis.nl | Analyzer for Dutch. |
org.apache.lucene.analysis.payloads | Provides various convenience classes for creating payloads on Tokens. |
org.apache.lucene.analysis.position | Filter for assigning position increments. |
org.apache.lucene.analysis.query | Automatically filter high-frequency stopwords. |
org.apache.lucene.analysis.reverse | Filter to reverse token text. |
org.apache.lucene.analysis.ru | Analyzer for Russian. |
org.apache.lucene.analysis.shingle | Word n-gram filters |
org.apache.lucene.analysis.snowball | TokenFilter and Analyzer implementations that use Snowball
stemmers. |
org.apache.lucene.analysis.standard | A fast grammar-based tokenizer constructed with JFlex. |
org.apache.lucene.analysis.th | Analyzer for Thai. |
org.apache.lucene.collation |
CollationKeyFilter and ICUCollationKeyFilter
convert each token into its binary CollationKey using the
provided Collator , and then encode the CollationKey
as a String using
IndexableBinaryStringTools , to allow it to be
stored as an index term. |
org.apache.lucene.document | The logical representation of a Document for indexing and searching. |
org.apache.lucene.index.memory | High-performance single-document main memory Apache Lucene fulltext search index. |
org.apache.lucene.search.highlight | The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages. |
org.apache.lucene.wikipedia.analysis | Tokenizer that is aware of Wikipedia syntax. |
org.apache.lucene.wordnet | This package uses synonyms defined by WordNet. |
Uses of TokenStream in org.apache.lucene.analysis |
---|
Subclasses of TokenStream in org.apache.lucene.analysis | |
---|---|
class |
ASCIIFoldingFilter
This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists. |
class |
CachingTokenFilter
This class can be used if the token attributes of a TokenStream are intended to be consumed more than once. |
class |
CharTokenizer
An abstract base class for simple, character-oriented tokenizers. |
class |
ISOLatin1AccentFilter
Deprecated. If you build a new index, use ASCIIFoldingFilter
which covers a superset of Latin 1.
This class is included for use with existing
indexes and will be removed in a future release (possibly Lucene 4.0). |
class |
KeywordTokenizer
Emits the entire input as a single token. |
class |
LengthFilter
Removes words that are too long or too short from the stream. |
class |
LetterTokenizer
A LetterTokenizer is a tokenizer that divides text at non-letters. |
class |
LowerCaseFilter
Normalizes token text to lower case. |
class |
LowerCaseTokenizer
LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. |
class |
NumericTokenStream
Expert: This class provides a TokenStream
for indexing numeric values that can be used by NumericRangeQuery or NumericRangeFilter . |
class |
PorterStemFilter
Transforms the token stream as per the Porter stemming algorithm. |
class |
StopFilter
Removes stop words from a token stream. |
class |
TeeSinkTokenFilter
This TokenFilter provides the ability to set aside attribute states that have already been analyzed. |
static class |
TeeSinkTokenFilter.SinkTokenStream
|
class |
TokenFilter
A TokenFilter is a TokenStream whose input is another TokenStream. |
class |
Tokenizer
A Tokenizer is a TokenStream whose input is a Reader. |
class |
WhitespaceTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. |
Fields in org.apache.lucene.analysis declared as TokenStream | |
---|---|
protected TokenStream |
TokenFilter.input
The source of tokens for this filter. |
Methods in org.apache.lucene.analysis that return TokenStream | |
---|---|
TokenStream |
WhitespaceAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
StopAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
SimpleAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
PerFieldAnalyzerWrapper.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
KeywordAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
Analyzer.reusableTokenStream(String fieldName,
Reader reader)
Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method. |
TokenStream |
WhitespaceAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
StopAnalyzer.tokenStream(String fieldName,
Reader reader)
Filters LowerCaseTokenizer with StopFilter. |
TokenStream |
SimpleAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
PerFieldAnalyzerWrapper.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
KeywordAnalyzer.tokenStream(String fieldName,
Reader reader)
|
abstract TokenStream |
Analyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader. |
Constructors in org.apache.lucene.analysis with parameters of type TokenStream | |
---|---|
ASCIIFoldingFilter(TokenStream input)
|
|
CachingTokenFilter(TokenStream input)
|
|
ISOLatin1AccentFilter(TokenStream input)
Deprecated. |
|
LengthFilter(TokenStream in,
int min,
int max)
Build a filter that removes words that are too long or too short from the text. |
|
LowerCaseFilter(TokenStream in)
|
|
PorterStemFilter(TokenStream in)
|
|
StopFilter(boolean enablePositionIncrements,
TokenStream in,
Set<?> stopWords)
Constructs a filter which removes words from the input TokenStream that are named in the Set. |
|
StopFilter(boolean enablePositionIncrements,
TokenStream input,
Set<?> stopWords,
boolean ignoreCase)
Construct a token stream filtering the given input. |
|
TeeSinkTokenFilter(TokenStream input)
Instantiates a new TeeSinkTokenFilter. |
|
TokenFilter(TokenStream input)
Construct a token stream filtering the given input. |
Uses of TokenStream in org.apache.lucene.analysis.ar |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.ar | |
---|---|
class |
ArabicLetterTokenizer
Tokenizer that breaks text into runs of letters and diacritics. |
class |
ArabicNormalizationFilter
A TokenFilter that applies ArabicNormalizer to normalize the orthography. |
class |
ArabicStemFilter
A TokenFilter that applies ArabicStemmer to stem Arabic words.. |
Methods in org.apache.lucene.analysis.ar that return TokenStream | |
---|---|
TokenStream |
ArabicAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text
in the provided Reader . |
TokenStream |
ArabicAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader . |
Constructors in org.apache.lucene.analysis.ar with parameters of type TokenStream | |
---|---|
ArabicNormalizationFilter(TokenStream input)
|
|
ArabicStemFilter(TokenStream input)
|
Uses of TokenStream in org.apache.lucene.analysis.br |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.br | |
---|---|
class |
BrazilianStemFilter
A TokenFilter that applies BrazilianStemmer . |
Methods in org.apache.lucene.analysis.br that return TokenStream | |
---|---|
TokenStream |
BrazilianAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text
in the provided Reader . |
TokenStream |
BrazilianAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader . |
Constructors in org.apache.lucene.analysis.br with parameters of type TokenStream | |
---|---|
BrazilianStemFilter(TokenStream in)
|
|
BrazilianStemFilter(TokenStream in,
Set exclusiontable)
|
Uses of TokenStream in org.apache.lucene.analysis.cjk |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.cjk | |
---|---|
class |
CJKTokenizer
CJKTokenizer is designed for Chinese, Japanese, and Korean languages. |
Methods in org.apache.lucene.analysis.cjk that return TokenStream | |
---|---|
TokenStream |
CJKAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text
in the provided Reader . |
TokenStream |
CJKAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader . |
Uses of TokenStream in org.apache.lucene.analysis.cn |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.cn | |
---|---|
class |
ChineseFilter
A TokenFilter with a stop word table. |
class |
ChineseTokenizer
Tokenize Chinese text as individual chinese characters. |
Methods in org.apache.lucene.analysis.cn that return TokenStream | |
---|---|
TokenStream |
ChineseAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text in the
provided Reader . |
TokenStream |
ChineseAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader . |
Constructors in org.apache.lucene.analysis.cn with parameters of type TokenStream | |
---|---|
ChineseFilter(TokenStream in)
|
Uses of TokenStream in org.apache.lucene.analysis.cn.smart |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.cn.smart | |
---|---|
class |
SentenceTokenizer
Tokenizes input text into sentences. |
class |
WordTokenFilter
A TokenFilter that breaks sentences into words. |
Methods in org.apache.lucene.analysis.cn.smart that return TokenStream | |
---|---|
TokenStream |
SmartChineseAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
SmartChineseAnalyzer.tokenStream(String fieldName,
Reader reader)
|
Constructors in org.apache.lucene.analysis.cn.smart with parameters of type TokenStream | |
---|---|
WordTokenFilter(TokenStream in)
Construct a new WordTokenizer. |
Uses of TokenStream in org.apache.lucene.analysis.compound |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.compound | |
---|---|
class |
CompoundWordTokenFilterBase
Base class for decomposition token filters. |
class |
DictionaryCompoundWordTokenFilter
A TokenFilter that decomposes compound words found in many Germanic languages. |
class |
HyphenationCompoundWordTokenFilter
A TokenFilter that decomposes compound words found in many Germanic languages. |
Constructors in org.apache.lucene.analysis.compound with parameters of type TokenStream | |
---|---|
CompoundWordTokenFilterBase(TokenStream input,
Set dictionary)
|
|
CompoundWordTokenFilterBase(TokenStream input,
Set dictionary,
boolean onlyLongestMatch)
|
|
CompoundWordTokenFilterBase(TokenStream input,
Set dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch)
|
|
CompoundWordTokenFilterBase(TokenStream input,
String[] dictionary)
|
|
CompoundWordTokenFilterBase(TokenStream input,
String[] dictionary,
boolean onlyLongestMatch)
|
|
CompoundWordTokenFilterBase(TokenStream input,
String[] dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch)
|
|
DictionaryCompoundWordTokenFilter(TokenStream input,
Set dictionary)
|
|
DictionaryCompoundWordTokenFilter(TokenStream input,
Set dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch)
|
|
DictionaryCompoundWordTokenFilter(TokenStream input,
String[] dictionary)
|
|
DictionaryCompoundWordTokenFilter(TokenStream input,
String[] dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch)
|
|
HyphenationCompoundWordTokenFilter(TokenStream input,
HyphenationTree hyphenator,
Set dictionary)
|
|
HyphenationCompoundWordTokenFilter(TokenStream input,
HyphenationTree hyphenator,
Set dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch)
|
|
HyphenationCompoundWordTokenFilter(TokenStream input,
HyphenationTree hyphenator,
String[] dictionary)
|
|
HyphenationCompoundWordTokenFilter(TokenStream input,
HyphenationTree hyphenator,
String[] dictionary,
int minWordSize,
int minSubwordSize,
int maxSubwordSize,
boolean onlyLongestMatch)
|
Uses of TokenStream in org.apache.lucene.analysis.cz |
---|
Methods in org.apache.lucene.analysis.cz that return TokenStream | |
---|---|
TokenStream |
CzechAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text in
the provided Reader . |
TokenStream |
CzechAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader . |
Uses of TokenStream in org.apache.lucene.analysis.de |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.de | |
---|---|
class |
GermanStemFilter
A TokenFilter that stems German words. |
Methods in org.apache.lucene.analysis.de that return TokenStream | |
---|---|
TokenStream |
GermanAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text
in the provided Reader . |
TokenStream |
GermanAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader . |
Constructors in org.apache.lucene.analysis.de with parameters of type TokenStream | |
---|---|
GermanStemFilter(TokenStream in)
|
|
GermanStemFilter(TokenStream in,
Set exclusionSet)
Builds a GermanStemFilter that uses an exclusion table. |
Uses of TokenStream in org.apache.lucene.analysis.el |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.el | |
---|---|
class |
GreekLowerCaseFilter
Normalizes token text to lower case, removes some Greek diacritics, and standardizes final sigma to sigma. |
Methods in org.apache.lucene.analysis.el that return TokenStream | |
---|---|
TokenStream |
GreekAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text
in the provided Reader . |
TokenStream |
GreekAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader . |
Constructors in org.apache.lucene.analysis.el with parameters of type TokenStream | |
---|---|
GreekLowerCaseFilter(TokenStream in)
|
Uses of TokenStream in org.apache.lucene.analysis.fa |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.fa | |
---|---|
class |
PersianNormalizationFilter
A TokenFilter that applies PersianNormalizer to normalize the
orthography. |
Methods in org.apache.lucene.analysis.fa that return TokenStream | |
---|---|
TokenStream |
PersianAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text
in the provided Reader . |
TokenStream |
PersianAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided
Reader . |
Constructors in org.apache.lucene.analysis.fa with parameters of type TokenStream | |
---|---|
PersianNormalizationFilter(TokenStream input)
|
Uses of TokenStream in org.apache.lucene.analysis.fr |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.fr | |
---|---|
class |
ElisionFilter
Removes elisions from a TokenStream . |
class |
FrenchStemFilter
A TokenFilter that stems french words. |
Methods in org.apache.lucene.analysis.fr that return TokenStream | |
---|---|
TokenStream |
FrenchAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the
text in the provided Reader . |
TokenStream |
FrenchAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided
Reader . |
Constructors in org.apache.lucene.analysis.fr with parameters of type TokenStream | |
---|---|
ElisionFilter(TokenStream input)
Constructs an elision filter with standard stop words |
|
ElisionFilter(TokenStream input,
Set<?> articles)
Constructs an elision filter with a Set of stop words |
|
ElisionFilter(TokenStream input,
String[] articles)
Constructs an elision filter with an array of stop words |
|
FrenchStemFilter(TokenStream in)
|
|
FrenchStemFilter(TokenStream in,
Set exclusiontable)
|
Uses of TokenStream in org.apache.lucene.analysis.miscellaneous |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.miscellaneous | |
---|---|
class |
EmptyTokenStream
An always exhausted token stream. |
class |
PrefixAndSuffixAwareTokenFilter
Links two PrefixAwareTokenFilter . |
class |
PrefixAwareTokenFilter
Joins two token streams and leaves the last token of the first stream available to be used when updating the token values in the second stream based on that token. |
class |
SingleTokenTokenStream
A TokenStream containing a single token. |
Methods in org.apache.lucene.analysis.miscellaneous that return TokenStream | |
---|---|
TokenStream |
PrefixAwareTokenFilter.getPrefix()
|
TokenStream |
PrefixAwareTokenFilter.getSuffix()
|
TokenStream |
PatternAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a token stream that tokenizes all the text in the given Reader; This implementation forwards to tokenStream(String, String) and is
less efficient than tokenStream(String, String) . |
TokenStream |
PatternAnalyzer.tokenStream(String fieldName,
String text)
Creates a token stream that tokenizes the given string into token terms (aka words). |
Methods in org.apache.lucene.analysis.miscellaneous with parameters of type TokenStream | |
---|---|
void |
PrefixAwareTokenFilter.setPrefix(TokenStream prefix)
|
void |
PrefixAwareTokenFilter.setSuffix(TokenStream suffix)
|
Constructors in org.apache.lucene.analysis.miscellaneous with parameters of type TokenStream | |
---|---|
PrefixAndSuffixAwareTokenFilter(TokenStream prefix,
TokenStream input,
TokenStream suffix)
|
|
PrefixAwareTokenFilter(TokenStream prefix,
TokenStream suffix)
|
Uses of TokenStream in org.apache.lucene.analysis.ngram |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.ngram | |
---|---|
class |
EdgeNGramTokenFilter
Tokenizes the given token into n-grams of given size(s). |
class |
EdgeNGramTokenizer
Tokenizes the input from an edge into n-grams of given size(s). |
class |
NGramTokenFilter
Tokenizes the input into n-grams of the given size(s). |
class |
NGramTokenizer
Tokenizes the input into n-grams of the given size(s). |
Constructors in org.apache.lucene.analysis.ngram with parameters of type TokenStream | |
---|---|
EdgeNGramTokenFilter(TokenStream input)
|
|
EdgeNGramTokenFilter(TokenStream input,
EdgeNGramTokenFilter.Side side,
int minGram,
int maxGram)
Creates EdgeNGramTokenFilter that can generate n-grams in the sizes of the given range |
|
EdgeNGramTokenFilter(TokenStream input,
String sideLabel,
int minGram,
int maxGram)
Creates EdgeNGramTokenFilter that can generate n-grams in the sizes of the given range |
|
NGramTokenFilter(TokenStream input)
Creates NGramTokenFilter with default min and max n-grams. |
|
NGramTokenFilter(TokenStream input,
int minGram,
int maxGram)
Creates NGramTokenFilter with given min and max n-grams. |
Uses of TokenStream in org.apache.lucene.analysis.nl |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.nl | |
---|---|
class |
DutchStemFilter
A TokenFilter that stems Dutch words. |
Methods in org.apache.lucene.analysis.nl that return TokenStream | |
---|---|
TokenStream |
DutchAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the
text in the provided Reader . |
TokenStream |
DutchAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the
provided Reader . |
Constructors in org.apache.lucene.analysis.nl with parameters of type TokenStream | |
---|---|
DutchStemFilter(TokenStream _in)
|
|
DutchStemFilter(TokenStream _in,
Set exclusiontable)
Builds a DutchStemFilter that uses an exclusion table. |
|
DutchStemFilter(TokenStream _in,
Set exclusiontable,
Map stemdictionary)
|
Uses of TokenStream in org.apache.lucene.analysis.payloads |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.payloads | |
---|---|
class |
DelimitedPayloadTokenFilter
Characters before the delimiter are the "token", those after are the payload. |
class |
NumericPayloadTokenFilter
Assigns a payload to a token based on the Token.type() |
class |
TokenOffsetPayloadTokenFilter
Adds the Token.setStartOffset(int)
and Token.setEndOffset(int)
First 4 bytes are the start |
class |
TypeAsPayloadTokenFilter
Makes the Token.type() a payload. |
Constructors in org.apache.lucene.analysis.payloads with parameters of type TokenStream | |
---|---|
DelimitedPayloadTokenFilter(TokenStream input)
Construct a token stream filtering the given input. |
|
DelimitedPayloadTokenFilter(TokenStream input,
char delimiter,
PayloadEncoder encoder)
|
|
NumericPayloadTokenFilter(TokenStream input,
float payload,
String typeMatch)
|
|
TokenOffsetPayloadTokenFilter(TokenStream input)
|
|
TypeAsPayloadTokenFilter(TokenStream input)
|
Uses of TokenStream in org.apache.lucene.analysis.position |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.position | |
---|---|
class |
PositionFilter
Set the positionIncrement of all tokens to the "positionIncrement", except the first return token which retains its original positionIncrement value. |
Constructors in org.apache.lucene.analysis.position with parameters of type TokenStream | |
---|---|
PositionFilter(TokenStream input)
Constructs a PositionFilter that assigns a position increment of zero to all but the first token from the given input stream. |
|
PositionFilter(TokenStream input,
int positionIncrement)
Constructs a PositionFilter that assigns the given position increment to all but the first token from the given input stream. |
Uses of TokenStream in org.apache.lucene.analysis.query |
---|
Methods in org.apache.lucene.analysis.query that return TokenStream | |
---|---|
TokenStream |
QueryAutoStopWordAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
QueryAutoStopWordAnalyzer.tokenStream(String fieldName,
Reader reader)
|
Uses of TokenStream in org.apache.lucene.analysis.reverse |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.reverse | |
---|---|
class |
ReverseStringFilter
Reverse token string, for example "country" => "yrtnuoc". |
Constructors in org.apache.lucene.analysis.reverse with parameters of type TokenStream | |
---|---|
ReverseStringFilter(TokenStream in)
Create a new ReverseStringFilter that reverses all tokens in the supplied TokenStream . |
|
ReverseStringFilter(TokenStream in,
char marker)
Create a new ReverseStringFilter that reverses and marks all tokens in the supplied TokenStream . |
Uses of TokenStream in org.apache.lucene.analysis.ru |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.ru | |
---|---|
class |
RussianLetterTokenizer
A RussianLetterTokenizer is a Tokenizer that extends LetterTokenizer
by also allowing the basic latin digits 0-9. |
class |
RussianLowerCaseFilter
Deprecated. Use LowerCaseFilter instead, which has the same
functionality. This filter will be removed in Lucene 4.0 |
class |
RussianStemFilter
A TokenFilter that stems Russian words. |
Methods in org.apache.lucene.analysis.ru that return TokenStream | |
---|---|
TokenStream |
RussianAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) TokenStream which tokenizes all the text
in the provided Reader . |
TokenStream |
RussianAnalyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the
provided Reader . |
Constructors in org.apache.lucene.analysis.ru with parameters of type TokenStream | |
---|---|
RussianLowerCaseFilter(TokenStream in)
Deprecated. |
|
RussianStemFilter(TokenStream in)
|
Uses of TokenStream in org.apache.lucene.analysis.shingle |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.shingle | |
---|---|
class |
ShingleFilter
A ShingleFilter constructs shingles (token n-grams) from a token stream. |
class |
ShingleMatrixFilter
A ShingleMatrixFilter constructs shingles (token n-grams) from a token stream. |
Methods in org.apache.lucene.analysis.shingle that return TokenStream | |
---|---|
TokenStream |
ShingleAnalyzerWrapper.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
ShingleAnalyzerWrapper.tokenStream(String fieldName,
Reader reader)
|
Constructors in org.apache.lucene.analysis.shingle with parameters of type TokenStream | |
---|---|
ShingleFilter(TokenStream input)
Construct a ShingleFilter with default shingle size. |
|
ShingleFilter(TokenStream input,
int maxShingleSize)
Constructs a ShingleFilter with the specified single size from the TokenStream input |
|
ShingleFilter(TokenStream input,
String tokenType)
Construct a ShingleFilter with the specified token type for shingle tokens. |
|
ShingleMatrixFilter(TokenStream input,
int minimumShingleSize,
int maximumShingleSize)
Creates a shingle filter using default settings. |
|
ShingleMatrixFilter(TokenStream input,
int minimumShingleSize,
int maximumShingleSize,
Character spacerCharacter)
Creates a shingle filter using default settings. |
|
ShingleMatrixFilter(TokenStream input,
int minimumShingleSize,
int maximumShingleSize,
Character spacerCharacter,
boolean ignoringSinglePrefixOrSuffixShingle)
Creates a shingle filter using the default ShingleMatrixFilter.TokenSettingsCodec . |
|
ShingleMatrixFilter(TokenStream input,
int minimumShingleSize,
int maximumShingleSize,
Character spacerCharacter,
boolean ignoringSinglePrefixOrSuffixShingle,
ShingleMatrixFilter.TokenSettingsCodec settingsCodec)
Creates a shingle filter with ad hoc parameter settings. |
Uses of TokenStream in org.apache.lucene.analysis.snowball |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.snowball | |
---|---|
class |
SnowballFilter
A filter that stems words using a Snowball-generated stemmer. |
Methods in org.apache.lucene.analysis.snowball that return TokenStream | |
---|---|
TokenStream |
SnowballAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
Returns a (possibly reused) StandardTokenizer filtered by a
StandardFilter , a LowerCaseFilter ,
a StopFilter , and a SnowballFilter |
TokenStream |
SnowballAnalyzer.tokenStream(String fieldName,
Reader reader)
Constructs a StandardTokenizer filtered by a StandardFilter , a LowerCaseFilter , a StopFilter ,
and a SnowballFilter |
Constructors in org.apache.lucene.analysis.snowball with parameters of type TokenStream | |
---|---|
SnowballFilter(TokenStream input,
SnowballProgram stemmer)
|
|
SnowballFilter(TokenStream in,
String name)
Construct the named stemming filter. |
Uses of TokenStream in org.apache.lucene.analysis.standard |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.standard | |
---|---|
class |
StandardFilter
Normalizes tokens extracted with StandardTokenizer . |
class |
StandardTokenizer
A grammar-based tokenizer constructed with JFlex |
Methods in org.apache.lucene.analysis.standard that return TokenStream | |
---|---|
TokenStream |
StandardAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
StandardAnalyzer.tokenStream(String fieldName,
Reader reader)
Constructs a StandardTokenizer filtered by a StandardFilter , a LowerCaseFilter and a StopFilter . |
Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream | |
---|---|
StandardFilter(TokenStream in)
Construct filtering in. |
Uses of TokenStream in org.apache.lucene.analysis.th |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.th | |
---|---|
class |
ThaiWordFilter
TokenFilter that use BreakIterator to break each
Token that is Thai into separate Token(s) for each Thai word. |
Methods in org.apache.lucene.analysis.th that return TokenStream | |
---|---|
TokenStream |
ThaiAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
ThaiAnalyzer.tokenStream(String fieldName,
Reader reader)
|
Constructors in org.apache.lucene.analysis.th with parameters of type TokenStream | |
---|---|
ThaiWordFilter(TokenStream input)
|
Uses of TokenStream in org.apache.lucene.collation |
---|
Subclasses of TokenStream in org.apache.lucene.collation | |
---|---|
class |
CollationKeyFilter
Converts each token into its CollationKey , and then
encodes the CollationKey with IndexableBinaryStringTools , to allow
it to be stored as an index term. |
class |
ICUCollationKeyFilter
Converts each token into its CollationKey , and
then encodes the CollationKey with IndexableBinaryStringTools , to
allow it to be stored as an index term. |
Methods in org.apache.lucene.collation that return TokenStream | |
---|---|
TokenStream |
ICUCollationKeyAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
CollationKeyAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
ICUCollationKeyAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
CollationKeyAnalyzer.tokenStream(String fieldName,
Reader reader)
|
Constructors in org.apache.lucene.collation with parameters of type TokenStream | |
---|---|
CollationKeyFilter(TokenStream input,
Collator collator)
|
|
ICUCollationKeyFilter(TokenStream input,
com.ibm.icu.text.Collator collator)
|
Uses of TokenStream in org.apache.lucene.document |
---|
Fields in org.apache.lucene.document declared as TokenStream | |
---|---|
protected TokenStream |
AbstractField.tokenStream
|
Methods in org.apache.lucene.document that return TokenStream | |
---|---|
TokenStream |
NumericField.tokenStreamValue()
Returns a NumericTokenStream for indexing the numeric value. |
TokenStream |
Field.tokenStreamValue()
The TokesStream for this field to be used when indexing, or null. |
TokenStream |
Fieldable.tokenStreamValue()
The TokenStream for this field to be used when indexing, or null. |
Methods in org.apache.lucene.document with parameters of type TokenStream | |
---|---|
void |
Field.setTokenStream(TokenStream tokenStream)
Expert: sets the token stream to be used for indexing and causes isIndexed() and isTokenized() to return true. |
Constructors in org.apache.lucene.document with parameters of type TokenStream | |
---|---|
Field(String name,
TokenStream tokenStream)
Create a tokenized and indexed field that is not stored. |
|
Field(String name,
TokenStream tokenStream,
Field.TermVector termVector)
Create a tokenized and indexed field that is not stored, optionally with storing term vectors. |
Uses of TokenStream in org.apache.lucene.index.memory |
---|
Methods in org.apache.lucene.index.memory that return TokenStream | ||
---|---|---|
|
MemoryIndex.keywordTokenStream(Collection<T> keywords)
Convenience method; Creates and returns a token stream that generates a token for each keyword in the given collection, "as is", without any transforming text analysis. |
Methods in org.apache.lucene.index.memory with parameters of type TokenStream | |
---|---|
void |
MemoryIndex.addField(String fieldName,
TokenStream stream)
Equivalent to addField(fieldName, stream, 1.0f) . |
void |
MemoryIndex.addField(String fieldName,
TokenStream stream,
float boost)
Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, Lucene Field . |
Uses of TokenStream in org.apache.lucene.search.highlight |
---|
Methods in org.apache.lucene.search.highlight that return TokenStream | |
---|---|
static TokenStream |
TokenSources.getAnyTokenStream(IndexReader reader,
int docId,
String field,
Analyzer analyzer)
A convenience method that tries a number of approaches to getting a token stream. |
static TokenStream |
TokenSources.getAnyTokenStream(IndexReader reader,
int docId,
String field,
Document doc,
Analyzer analyzer)
A convenience method that tries to first get a TermPositionVector for the specified docId, then, falls back to using the passed in Document to retrieve the TokenStream. |
TokenStream |
WeightedSpanTermExtractor.getTokenStream()
|
static TokenStream |
TokenSources.getTokenStream(Document doc,
String field,
Analyzer analyzer)
|
static TokenStream |
TokenSources.getTokenStream(IndexReader reader,
int docId,
String field)
|
static TokenStream |
TokenSources.getTokenStream(IndexReader reader,
int docId,
String field,
Analyzer analyzer)
|
static TokenStream |
TokenSources.getTokenStream(String field,
String contents,
Analyzer analyzer)
|
static TokenStream |
TokenSources.getTokenStream(TermPositionVector tpv)
|
static TokenStream |
TokenSources.getTokenStream(TermPositionVector tpv,
boolean tokenPositionsGuaranteedContiguous)
Low level api. |
TokenStream |
Scorer.init(TokenStream tokenStream)
Called to init the Scorer with a TokenStream . |
TokenStream |
QueryScorer.init(TokenStream tokenStream)
|
TokenStream |
QueryTermScorer.init(TokenStream tokenStream)
|
Methods in org.apache.lucene.search.highlight with parameters of type TokenStream | |
---|---|
String |
Highlighter.getBestFragment(TokenStream tokenStream,
String text)
Highlights chosen terms in a text, extracting the most relevant section. |
String[] |
Highlighter.getBestFragments(TokenStream tokenStream,
String text,
int maxNumFragments)
Highlights chosen terms in a text, extracting the most relevant sections. |
String |
Highlighter.getBestFragments(TokenStream tokenStream,
String text,
int maxNumFragments,
String separator)
Highlights terms in the text , extracting the most relevant sections and concatenating the chosen fragments with a separator (typically "..."). |
TextFragment[] |
Highlighter.getBestTextFragments(TokenStream tokenStream,
String text,
boolean mergeContiguousFragments,
int maxNumFragments)
Low level api to get the most relevant (formatted) sections of the document. |
Map<String,WeightedSpanTerm> |
WeightedSpanTermExtractor.getWeightedSpanTerms(Query query,
TokenStream tokenStream)
Creates a Map of WeightedSpanTerms from the given Query and TokenStream . |
Map<String,WeightedSpanTerm> |
WeightedSpanTermExtractor.getWeightedSpanTerms(Query query,
TokenStream tokenStream,
String fieldName)
Creates a Map of WeightedSpanTerms from the given Query and TokenStream . |
Map<String,WeightedSpanTerm> |
WeightedSpanTermExtractor.getWeightedSpanTermsWithScores(Query query,
TokenStream tokenStream,
String fieldName,
IndexReader reader)
Creates a Map of WeightedSpanTerms from the given Query and TokenStream . |
TokenStream |
Scorer.init(TokenStream tokenStream)
Called to init the Scorer with a TokenStream . |
TokenStream |
QueryScorer.init(TokenStream tokenStream)
|
TokenStream |
QueryTermScorer.init(TokenStream tokenStream)
|
void |
Fragmenter.start(String originalText,
TokenStream tokenStream)
Initializes the Fragmenter. |
void |
SimpleSpanFragmenter.start(String originalText,
TokenStream tokenStream)
|
void |
NullFragmenter.start(String s,
TokenStream tokenStream)
|
void |
SimpleFragmenter.start(String originalText,
TokenStream stream)
|
Constructors in org.apache.lucene.search.highlight with parameters of type TokenStream | |
---|---|
TokenGroup(TokenStream tokenStream)
|
Uses of TokenStream in org.apache.lucene.wikipedia.analysis |
---|
Subclasses of TokenStream in org.apache.lucene.wikipedia.analysis | |
---|---|
class |
WikipediaTokenizer
Extension of StandardTokenizer that is aware of Wikipedia syntax. |
Uses of TokenStream in org.apache.lucene.wordnet |
---|
Subclasses of TokenStream in org.apache.lucene.wordnet | |
---|---|
class |
SynonymTokenFilter
Injects additional tokens for synonyms of token terms fetched from the underlying child stream; the child stream must deliver lowercase tokens for synonyms to be found. |
Constructors in org.apache.lucene.wordnet with parameters of type TokenStream | |
---|---|
SynonymTokenFilter(TokenStream input,
SynonymMap synonyms,
int maxSynonyms)
Creates an instance for the given underlying stream and synonym table. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |