Package org.apache.lucene.util.hnsw

Class IncrementalHnswGraphMerger

java.lang.Object

org.apache.lucene.util.hnsw.IncrementalHnswGraphMerger

All Implemented Interfaces:

HnswGraphMerger

Direct Known Subclasses:

ConcurrentHnswMerger

public class IncrementalHnswGraphMerger
extends Object
implements HnswGraphMerger

This merges multiple graphs in a single thread in incremental fashion.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static final record	IncrementalHnswGraphMerger.GraphReader	Represents a vector reader that contains graph info.

Field Summary

Fields

Modifier and Type	Field	Description
protected final int	beamWidth
protected final FieldInfo	fieldInfo
protected List<IncrementalHnswGraphMerger.GraphReader>	graphReaders
protected final int	M
protected final RandomVectorScorerSupplier	scorerSupplier

Constructor Summary

Constructors

Constructor	Description
IncrementalHnswGraphMerger(FieldInfo fieldInfo, RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth)

Method Summary

Modifier and Type	Method	Description
IncrementalHnswGraphMerger	addReader(KnnVectorsReader reader, MergeState.DocMap docMap, Bits liveDocs)	Adds a reader to the graph merger if it meets the following criteria: 1.
protected HnswBuilder	createBuilder(KnnVectorValues mergedVectorValues, int maxOrd)	Builds a new HnswGraphBuilder
protected final int[][]	getNewOrdMapping(KnnVectorValues mergedVectorValues, BitSet initializedNodes)
OnHeapHnswGraph	merge(KnnVectorValues mergedVectorValues, InfoStream infoStream, int maxOrd)	Merge and produce the on heap graph

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

fieldInfo

protected final FieldInfo fieldInfo

scorerSupplier

protected final RandomVectorScorerSupplier scorerSupplier

M

protected final int M

beamWidth

protected final int beamWidth

graphReaders

protected List<IncrementalHnswGraphMerger.GraphReader> graphReaders

Constructor Details

IncrementalHnswGraphMerger

public IncrementalHnswGraphMerger(FieldInfo fieldInfo,
 RandomVectorScorerSupplier scorerSupplier,
 int M,
 int beamWidth)

Parameters:

fieldInfo - FieldInfo for the field being merged

Method Details

addReader

public IncrementalHnswGraphMerger addReader(KnnVectorsReader reader,
 MergeState.DocMap docMap,
 Bits liveDocs)
                                     throws IOException

Adds a reader to the graph merger if it meets the following criteria: 1. does not contain any deleted docs 2. is a HnswGraphProvider/PerFieldKnnVectorReader

Specified by:

addReader in interface HnswGraphMerger

Parameters:

reader - KnnVectorsReader to add to the merger

docMap - MergeState.DocMap for the reader

liveDocs - Bits representing live docs, can be null

Returns:

this

Throws:

IOException - If an error occurs while reading from the merge state

createBuilder

protected HnswBuilder createBuilder(KnnVectorValues mergedVectorValues,
 int maxOrd)
                             throws IOException

Builds a new HnswGraphBuilder

Parameters:

mergedVectorValues - vector values in the merged segment

maxOrd - max num of vectors that will be merged into the graph

Returns:

HnswGraphBuilder

Throws:

IOException - If an error occurs while reading from the merge state

getNewOrdMapping

protected final int[][] getNewOrdMapping(KnnVectorValues mergedVectorValues,
 BitSet initializedNodes)
                                  throws IOException

Throws:

IOException

merge

public OnHeapHnswGraph merge(KnnVectorValues mergedVectorValues,
 InfoStream infoStream,
 int maxOrd)
                      throws IOException

Description copied from interface: HnswGraphMerger

Merge and produce the on heap graph

Specified by:

merge in interface HnswGraphMerger

Parameters:

mergedVectorValues - view of the vectors in the merged segment

infoStream - optional info stream to set to builder

maxOrd - max number of vectors that will be added to the graph

Returns:

merged graph

Throws:

IOException - during merge