org.apache.lucene.analysis.cn.smart
Class SentenceTokenizer
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.Tokenizer
org.apache.lucene.analysis.cn.smart.SentenceTokenizer
- All Implemented Interfaces:
- Closeable
public final class SentenceTokenizer
- extends org.apache.lucene.analysis.Tokenizer
Tokenizes input text into sentences.
The output tokens can then be broken into words with WordTokenFilter
WARNING: The status of the analyzers/smartcn analysis.cn.smart package is experimental.
The APIs and file formats introduced here might change in the future and will not be
supported anymore in such a case.
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource |
org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State |
Fields inherited from class org.apache.lucene.analysis.Tokenizer |
input |
Methods inherited from class org.apache.lucene.analysis.Tokenizer |
close, correctOffset |
Methods inherited from class org.apache.lucene.util.AttributeSource |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString |
SentenceTokenizer
public SentenceTokenizer(Reader reader)
SentenceTokenizer
public SentenceTokenizer(org.apache.lucene.util.AttributeSource source,
Reader reader)
SentenceTokenizer
public SentenceTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
Reader reader)
incrementToken
public boolean incrementToken()
throws IOException
- Specified by:
incrementToken
in class org.apache.lucene.analysis.TokenStream
- Throws:
IOException
reset
public void reset()
throws IOException
- Overrides:
reset
in class org.apache.lucene.analysis.TokenStream
- Throws:
IOException
reset
public void reset(Reader input)
throws IOException
- Overrides:
reset
in class org.apache.lucene.analysis.Tokenizer
- Throws:
IOException
end
public void end()
throws IOException
- Overrides:
end
in class org.apache.lucene.analysis.TokenStream
- Throws:
IOException
Copyright © 2000-2010 Apache Software Foundation. All Rights Reserved.