public abstract class BaseUIMATokenizer extends Tokenizer
Tokenizer
which is able to analyze the given input with a
UIMA AnalysisEngine
AttributeSource.AttributeFactory, AttributeSource.State
Modifier and Type | Field and Description |
---|---|
protected org.apache.uima.analysis_engine.AnalysisEngine |
ae |
protected org.apache.uima.cas.CAS |
cas |
protected org.apache.uima.cas.FSIterator<org.apache.uima.cas.text.AnnotationFS> |
iterator |
Modifier | Constructor and Description |
---|---|
protected |
BaseUIMATokenizer(Reader reader,
String descriptorPath) |
Modifier and Type | Method and Description |
---|---|
protected void |
analyzeInput()
analyzes the tokenizer input using the given analysis engine
cas will be filled with extracted metadata (UIMA annotations, feature structures) |
void |
end() |
protected abstract void |
initializeIterator()
initialize the FSIterator which is used to build tokens at each incrementToken() method call
|
void |
reset() |
close, correctOffset, setReader
incrementToken
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState
protected org.apache.uima.cas.FSIterator<org.apache.uima.cas.text.AnnotationFS> iterator
protected final org.apache.uima.analysis_engine.AnalysisEngine ae
protected final org.apache.uima.cas.CAS cas
protected void analyzeInput() throws org.apache.uima.analysis_engine.AnalysisEngineProcessException, IOException
cas
will be filled with extracted metadata (UIMA annotations, feature structures)IOException
- If there is a low-level I/O error.org.apache.uima.analysis_engine.AnalysisEngineProcessException
protected abstract void initializeIterator() throws IOException
IOException
- If there is a low-level I/O error.public void reset() throws IOException
reset
in class TokenStream
IOException
public void end() throws IOException
end
in class TokenStream
IOException
Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.