Uses of Class

Packages that use TokenStream
org.apache.lucene.analysis API and code to convert text into indexable/searchable tokens. 
org.apache.lucene.analysis.standard A fast grammar-based tokenizer constructed with JFlex. 
org.apache.lucene.document The logical representation of a Document for indexing and searching. 

Uses of TokenStream in org.apache.lucene.analysis

Subclasses of TokenStream in org.apache.lucene.analysis
 class ASCIIFoldingFilter
          This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.
 class CachingTokenFilter
          This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.
 class CharTokenizer
          An abstract base class for simple, character-oriented tokenizers.
 class ISOLatin1AccentFilter
          Deprecated. If you build a new index, use ASCIIFoldingFilter which covers a superset of Latin 1. This class is included for use with existing indexes and will be removed in a future release (possibly Lucene 4.0).
 class KeywordTokenizer
          Emits the entire input as a single token.
 class LengthFilter
          Removes words that are too long or too short from the stream.
 class LetterTokenizer
          A LetterTokenizer is a tokenizer that divides text at non-letters.
 class LowerCaseFilter
          Normalizes token text to lower case.
 class LowerCaseTokenizer
          LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
 class NumericTokenStream
          Expert: This class provides a TokenStream for indexing numeric values that can be used by NumericRangeQuery or NumericRangeFilter.
 class PorterStemFilter
          Transforms the token stream as per the Porter stemming algorithm.
 class StopFilter
          Removes stop words from a token stream.
 class TeeSinkTokenFilter
          This TokenFilter provides the ability to set aside attribute states that have already been analyzed.
static class TeeSinkTokenFilter.SinkTokenStream
 class TokenFilter
          A TokenFilter is a TokenStream whose input is another TokenStream.
 class Tokenizer
          A Tokenizer is a TokenStream whose input is a Reader.
 class WhitespaceTokenizer
          A WhitespaceTokenizer is a tokenizer that divides text at whitespace.

Fields in org.apache.lucene.analysis declared as TokenStream
protected  TokenStream TokenFilter.input
          The source of tokens for this filter.

Methods in org.apache.lucene.analysis that return TokenStream
 TokenStream WhitespaceAnalyzer.reusableTokenStream(String fieldName, Reader reader)
 TokenStream StopAnalyzer.reusableTokenStream(String fieldName, Reader reader)
 TokenStream SimpleAnalyzer.reusableTokenStream(String fieldName, Reader reader)
 TokenStream PerFieldAnalyzerWrapper.reusableTokenStream(String fieldName, Reader reader)
 TokenStream KeywordAnalyzer.reusableTokenStream(String fieldName, Reader reader)
 TokenStream Analyzer.reusableTokenStream(String fieldName, Reader reader)
          Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method.
 TokenStream WhitespaceAnalyzer.tokenStream(String fieldName, Reader reader)
 TokenStream StopAnalyzer.tokenStream(String fieldName, Reader reader)
          Filters LowerCaseTokenizer with StopFilter.
 TokenStream SimpleAnalyzer.tokenStream(String fieldName, Reader reader)
 TokenStream PerFieldAnalyzerWrapper.tokenStream(String fieldName, Reader reader)
 TokenStream KeywordAnalyzer.tokenStream(String fieldName, Reader reader)
abstract  TokenStream Analyzer.tokenStream(String fieldName, Reader reader)
          Creates a TokenStream which tokenizes all the text in the provided Reader.

Constructors in org.apache.lucene.analysis with parameters of type TokenStream
ASCIIFoldingFilter(TokenStream input)
CachingTokenFilter(TokenStream input)
ISOLatin1AccentFilter(TokenStream input)
LengthFilter(TokenStream in, int min, int max)
          Build a filter that removes words that are too long or too short from the text.
LowerCaseFilter(TokenStream in)
PorterStemFilter(TokenStream in)
StopFilter(boolean enablePositionIncrements, TokenStream in, Set<?> stopWords)
          Constructs a filter which removes words from the input TokenStream that are named in the Set.
StopFilter(boolean enablePositionIncrements, TokenStream input, Set<?> stopWords, boolean ignoreCase)
          Construct a token stream filtering the given input.
TeeSinkTokenFilter(TokenStream input)
          Instantiates a new TeeSinkTokenFilter.
TokenFilter(TokenStream input)
          Construct a token stream filtering the given input.

Uses of TokenStream in org.apache.lucene.analysis.standard

Subclasses of TokenStream in org.apache.lucene.analysis.standard
 class StandardFilter
          Normalizes tokens extracted with StandardTokenizer.
 class StandardTokenizer
          A grammar-based tokenizer constructed with JFlex

Methods in org.apache.lucene.analysis.standard that return TokenStream
 TokenStream StandardAnalyzer.reusableTokenStream(String fieldName, Reader reader)
 TokenStream StandardAnalyzer.tokenStream(String fieldName, Reader reader)
          Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.

Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream
StandardFilter(TokenStream in)
          Construct filtering in.

Uses of TokenStream in org.apache.lucene.document

Fields in org.apache.lucene.document declared as TokenStream
protected  TokenStream AbstractField.tokenStream

Methods in org.apache.lucene.document that return TokenStream
 TokenStream NumericField.tokenStreamValue()
          Returns a NumericTokenStream for indexing the numeric value.
 TokenStream Field.tokenStreamValue()
          The TokesStream for this field to be used when indexing, or null.
 TokenStream Fieldable.tokenStreamValue()
          The TokenStream for this field to be used when indexing, or null.

Methods in org.apache.lucene.document with parameters of type TokenStream
 void Field.setTokenStream(TokenStream tokenStream)
          Expert: sets the token stream to be used for indexing and causes isIndexed() and isTokenized() to return true.

Constructors in org.apache.lucene.document with parameters of type TokenStream
Field(String name, TokenStream tokenStream)
          Create a tokenized and indexed field that is not stored.
Field(String name, TokenStream tokenStream, Field.TermVector termVector)
          Create a tokenized and indexed field that is not stored, optionally with storing term vectors.

Copyright © 2000-2010 Apache Software Foundation. All Rights Reserved.