org.apache.lucene.facet.taxonomy.writercache.cl2o

Class CompactLabelToOrdinal

java.lang.Object

org.apache.lucene.facet.taxonomy.writercache.cl2o.LabelToOrdinal

org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal

public class CompactLabelToOrdinal
extends LabelToOrdinal

This is a very efficient LabelToOrdinal implementation that uses a CharBlockArray to store all labels and a configurable number of HashArrays to reference the labels.

Since the HashArrays don't handle collisions, a CollisionMap is used to store the colliding labels.

This data structure grows by adding a new HashArray whenever the number of collisions in the CollisionMap exceeds loadFactor * LabelToOrdinal.getMaxOrdinal(). Growing also includes reinserting all colliding labels into the HashArrays to possibly reduce the number of collisions. For setting the loadFactor see CompactLabelToOrdinal(int, float, int).

This data structure has a much lower memory footprint (~30%) compared to a Java HashMap. It also only uses a small fraction of objects a HashMap would use, thus limiting the GC overhead. Ingestion speed was also ~50% faster compared to a HashMap for 3M unique labels.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary

Fields

Modifier and Type	Field and Description
static float	DefaultLoadFactor

Fields inherited from class org.apache.lucene.facet.taxonomy.writercache.cl2o.LabelToOrdinal

counter, InvalidOrdinal

Constructor Summary

Constructors

Constructor and Description
CompactLabelToOrdinal(int initialCapacity, float loadFactor, int numHashArrays)

Method Summary

Methods

Modifier and Type	Method and Description
void	addLabel(CategoryPath label, int ordinal) Adds a new label if its not yet in the table.
void	addLabel(CategoryPath label, int prefixLen, int ordinal) Adds a new label if its not yet in the table.
int	getOrdinal(CategoryPath label)
int	getOrdinal(CategoryPath label, int prefixLen)
int	sizeOfMap()

Methods inherited from class org.apache.lucene.facet.taxonomy.writercache.cl2o.LabelToOrdinal

getMaxOrdinal, getNextOrdinal

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DefaultLoadFactor

public static final float DefaultLoadFactor

See Also:

Constant Field Values

Constructor Detail

CompactLabelToOrdinal

public CompactLabelToOrdinal(int initialCapacity,
                     float loadFactor,
                     int numHashArrays)

Method Detail

sizeOfMap

public int sizeOfMap()

addLabel

public void addLabel(CategoryPath label,
            int ordinal)

Description copied from class: LabelToOrdinal

Adds a new label if its not yet in the table. Throws an IllegalArgumentException if the same label with a different ordinal was previoulsy added to this table.

Specified by:

addLabel in class LabelToOrdinal

addLabel

public void addLabel(CategoryPath label,
            int prefixLen,
            int ordinal)

Description copied from class: LabelToOrdinal

Adds a new label if its not yet in the table. Throws an IllegalArgumentException if the same label with a different ordinal was previoulsy added to this table.

Specified by:

addLabel in class LabelToOrdinal

getOrdinal

public int getOrdinal(CategoryPath label)

Specified by:

getOrdinal in class LabelToOrdinal

Returns:

the ordinal assigned to the given label, or LabelToOrdinal.InvalidOrdinal if the label cannot be found in this table.

getOrdinal

public int getOrdinal(CategoryPath label,
             int prefixLen)

Specified by:

getOrdinal in class LabelToOrdinal

Returns:

the ordinal assigned to the given label, or LabelToOrdinal.InvalidOrdinal if the label cannot be found in this table.