org.apache.lucene.analysis.compound

Class Lucene43HyphenationCompoundWordTokenFilter

java.lang.Object

org.apache.lucene.util.AttributeSource

org.apache.lucene.analysis.TokenStream

org.apache.lucene.analysis.TokenFilter

org.apache.lucene.analysis.compound.Lucene43CompoundWordTokenFilterBase

org.apache.lucene.analysis.compound.Lucene43HyphenationCompoundWordTokenFilter

All Implemented Interfaces:

Closeable, AutoCloseable

Deprecated.

Use HyphenationCompoundWordTokenFilter.

@Deprecated
public class Lucene43HyphenationCompoundWordTokenFilter
extends Lucene43CompoundWordTokenFilterBase

A TokenFilter that decomposes compound words found in many Germanic languages, using pre-4.4 behavior.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.analysis.compound.Lucene43CompoundWordTokenFilterBase

Lucene43CompoundWordTokenFilterBase.CompoundToken

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

AttributeSource.State

Field Summary

Fields inherited from class org.apache.lucene.analysis.compound.Lucene43CompoundWordTokenFilterBase

DEFAULT_MAX_SUBWORD_SIZE, DEFAULT_MIN_SUBWORD_SIZE, DEFAULT_MIN_WORD_SIZE, dictionary, maxSubwordSize, minSubwordSize, minWordSize, offsetAtt, onlyLongestMatch, termAtt, tokens

Fields inherited from class org.apache.lucene.analysis.TokenFilter

input

Fields inherited from class org.apache.lucene.analysis.TokenStream

DEFAULT_TOKEN_ATTRIBUTE_FACTORY

Constructor Summary

Constructors

Constructor and Description
Lucene43HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator) Deprecated. Create a HyphenationCompoundWordTokenFilter with no dictionary.
Lucene43HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary) Deprecated. Creates a new Lucene43HyphenationCompoundWordTokenFilter instance.
Lucene43HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch) Deprecated. Creates a new Lucene43HyphenationCompoundWordTokenFilter instance.
Lucene43HyphenationCompoundWordTokenFilter(TokenStream input, HyphenationTree hyphenator, int minWordSize, int minSubwordSize, int maxSubwordSize) Deprecated. Create a HyphenationCompoundWordTokenFilter with no dictionary.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	decompose() Deprecated. Decomposes the current Lucene43CompoundWordTokenFilterBase.termAtt and places Lucene43CompoundWordTokenFilterBase.CompoundToken instances in the Lucene43CompoundWordTokenFilterBase.tokens list.
static HyphenationTree	getHyphenationTree(InputSource hyphenationSource) Deprecated. Create a hyphenator tree
static HyphenationTree	getHyphenationTree(String hyphenationFilename) Deprecated. Create a hyphenator tree

Methods inherited from class org.apache.lucene.analysis.compound.Lucene43CompoundWordTokenFilterBase

incrementToken, reset

Methods inherited from class org.apache.lucene.analysis.TokenFilter

close, end

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

Lucene43HyphenationCompoundWordTokenFilter

public Lucene43HyphenationCompoundWordTokenFilter(TokenStream input,
                                          HyphenationTree hyphenator,
                                          CharArraySet dictionary)

Deprecated.

Creates a new Lucene43HyphenationCompoundWordTokenFilter instance.

Parameters:

input - the TokenStream to process

hyphenator - the hyphenation pattern tree to use for hyphenation

dictionary - the word dictionary to match against.

Lucene43HyphenationCompoundWordTokenFilter

public Lucene43HyphenationCompoundWordTokenFilter(TokenStream input,
                                          HyphenationTree hyphenator,
                                          CharArraySet dictionary,
                                          int minWordSize,
                                          int minSubwordSize,
                                          int maxSubwordSize,
                                          boolean onlyLongestMatch)

Deprecated.

Creates a new Lucene43HyphenationCompoundWordTokenFilter instance.

Parameters:

input - the TokenStream to process

hyphenator - the hyphenation pattern tree to use for hyphenation

dictionary - the word dictionary to match against.

minWordSize - only words longer than this get processed

minSubwordSize - only subwords longer than this get to the output stream

maxSubwordSize - only subwords shorter than this get to the output stream

onlyLongestMatch - Add only the longest matching subword to the stream

Lucene43HyphenationCompoundWordTokenFilter

public Lucene43HyphenationCompoundWordTokenFilter(TokenStream input,
                                          HyphenationTree hyphenator,
                                          int minWordSize,
                                          int minSubwordSize,
                                          int maxSubwordSize)

Deprecated.

Create a HyphenationCompoundWordTokenFilter with no dictionary.

Calls HyphenationCompoundWordTokenFilter(matchVersion, input, hyphenator, null, minWordSize, minSubwordSize, maxSubwordSize

Lucene43HyphenationCompoundWordTokenFilter

public Lucene43HyphenationCompoundWordTokenFilter(TokenStream input,
                                          HyphenationTree hyphenator)

Deprecated.

Create a HyphenationCompoundWordTokenFilter with no dictionary.

Calls HyphenationCompoundWordTokenFilter(matchVersion, input, hyphenator, DEFAULT_MIN_WORD_SIZE, DEFAULT_MIN_SUBWORD_SIZE, DEFAULT_MAX_SUBWORD_SIZE

Method Detail

getHyphenationTree

public static HyphenationTree getHyphenationTree(String hyphenationFilename)
                                          throws IOException

Deprecated.

Create a hyphenator tree

Parameters:

hyphenationFilename - the filename of the XML grammar to load

Returns:

An object representing the hyphenation patterns

Throws:

IOException - If there is a low-level I/O error.

getHyphenationTree

public static HyphenationTree getHyphenationTree(InputSource hyphenationSource)
                                          throws IOException

Deprecated.

Create a hyphenator tree

Parameters:

hyphenationSource - the InputSource pointing to the XML grammar

Returns:

An object representing the hyphenation patterns

Throws:

IOException - If there is a low-level I/O error.

decompose

protected void decompose()

Deprecated.

Description copied from class: Lucene43CompoundWordTokenFilterBase

Decomposes the current Lucene43CompoundWordTokenFilterBase.termAtt and places Lucene43CompoundWordTokenFilterBase.CompoundToken instances in the Lucene43CompoundWordTokenFilterBase.tokens list. The original token may not be placed in the list, as it is automatically passed through this filter.

Specified by:

decompose in class Lucene43CompoundWordTokenFilterBase