Package org.apache.lucene.codecs.lucene99

Class Lucene99ScalarQuantizedVectorsFormat

java.lang.Object

org.apache.lucene.codecs.hnsw.FlatVectorsFormat

org.apache.lucene.codecs.lucene99.Lucene99ScalarQuantizedVectorsFormat

public class Lucene99ScalarQuantizedVectorsFormat
extends FlatVectorsFormat

Format supporting vector quantization, storage, and retrieval

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary

Fields

Modifier and Type	Field	Description
static float	DYNAMIC_CONFIDENCE_INTERVAL	Dynamic confidence interval
static String	NAME
static String	QUANTIZED_VECTOR_COMPONENT

Constructor Summary

Constructors

Constructor	Description
Lucene99ScalarQuantizedVectorsFormat()	Constructs a format using default graph construction parameters
Lucene99ScalarQuantizedVectorsFormat(Float confidenceInterval, int bits, boolean compress)	Constructs a format using the given graph construction parameters.

Method Summary

Modifier and Type	Method	Description
static float	calculateDefaultConfidenceInterval(int vectorDimension)
FlatVectorsReader	fieldsReader(SegmentReadState state)	Returns a KnnVectorsReader to read the vectors from the index.
FlatVectorsWriter	fieldsWriter(SegmentWriteState state)	Returns a FlatVectorsWriter to write the vectors to the index.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

QUANTIZED_VECTOR_COMPONENT

public static final String QUANTIZED_VECTOR_COMPONENT

See Also:

Constant Field Values

NAME

public static final String NAME

See Also:

Constant Field Values

DYNAMIC_CONFIDENCE_INTERVAL

public static final float DYNAMIC_CONFIDENCE_INTERVAL

Dynamic confidence interval

See Also:

Constant Field Values

Constructor Detail

Lucene99ScalarQuantizedVectorsFormat

public Lucene99ScalarQuantizedVectorsFormat()

Constructs a format using default graph construction parameters

Lucene99ScalarQuantizedVectorsFormat

public Lucene99ScalarQuantizedVectorsFormat(Float confidenceInterval,
                                            int bits,
                                            boolean compress)

Constructs a format using the given graph construction parameters.

Parameters:

confidenceInterval - the confidenceInterval for scalar quantizing the vectors, when `null` it is calculated based on the vector dimension. When `0`, the quantiles are dynamically determined by sampling many confidence intervals and determining the most accurate pair.

bits - the number of bits to use for scalar quantization (must be between 1 and 8, inclusive)

compress - whether to compress the vectors, if true, the vectors that are quantized with lte 4 bits will be compressed into a single byte. If false, the vectors will be stored as is. This provides a trade-off of memory usage and speed.

Method Detail

calculateDefaultConfidenceInterval

public static float calculateDefaultConfidenceInterval(int vectorDimension)

toString

public String toString()

Overrides:

toString in class Object

fieldsWriter

public FlatVectorsWriter fieldsWriter(SegmentWriteState state)
                               throws IOException

Description copied from class: FlatVectorsFormat

Returns a FlatVectorsWriter to write the vectors to the index.

Specified by:

fieldsWriter in class FlatVectorsFormat

Throws:

IOException

fieldsReader

public FlatVectorsReader fieldsReader(SegmentReadState state)
                               throws IOException

Description copied from class: FlatVectorsFormat

Returns a KnnVectorsReader to read the vectors from the index.

Specified by:

fieldsReader in class FlatVectorsFormat

Throws:

IOException