Package org.apache.lucene.util.hnsw
Class HnswGraphBuilder
java.lang.Object
org.apache.lucene.util.hnsw.HnswGraphBuilder
- All Implemented Interfaces:
HnswBuilder
- Direct Known Subclasses:
InitializedHnswGraphBuilder
,MergingHnswGraphBuilder
Builder for HNSW graph. See
HnswGraph
for a gloss on the algorithm and the meaning of the
hyper-parameters.-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic final class
A restricted, specialized knnCollector that can be used when building a graph. -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final int
Default number of the size of the queue maintained while searching during a graph construction.static final int
Default number of maximum connections per nodeprotected boolean
protected final OnHeapHnswGraph
static final String
A name for the HNSW component for the info-stream *protected final org.apache.lucene.util.hnsw.HnswLock
protected InfoStream
static long
Random seed for level generation; public to expose for testing *protected final RandomVectorScorerSupplier
-
Constructor Summary
ConstructorsModifierConstructorDescriptionprotected
HnswGraphBuilder
(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, int graphSize) Reads all the vectors from vector values, builds a graph connecting them by their dense ordinals, using the given hyperparameter settings, and returns the resulting graph.protected
HnswGraphBuilder
(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw) protected
HnswGraphBuilder
(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw, org.apache.lucene.util.hnsw.HnswLock hnswLock, HnswGraphSearcher graphSearcher) Reads all the vectors from vector values, builds a graph connecting them by their dense ordinals, using the given hyperparameter settings, and returns the resulting graph. -
Method Summary
Modifier and TypeMethodDescriptionvoid
addGraphNode
(int node) Inserts a doc with vector value to the graphvoid
addGraphNode
(int node, UpdateableRandomVectorScorer scorer) void
addGraphNodeWithEps
(int node, IntHashSet eps0) protected void
addVectors
(int minOrd, int maxOrd) add vectors in range [minOrd, maxOrd)build
(int maxOrd) Adds all nodes to the graph up to the providedmaxOrd
.static HnswGraphBuilder
create
(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed) static HnswGraphBuilder
create
(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, int graphSize) Once this method is called no further updates to the graph are accepted (addGraphNode will throw IllegalStateException).getGraph()
void
setInfoStream
(InfoStream infoStream) Set info-stream to output debugging information
-
Field Details
-
DEFAULT_MAX_CONN
public static final int DEFAULT_MAX_CONNDefault number of maximum connections per node- See Also:
-
DEFAULT_BEAM_WIDTH
public static final int DEFAULT_BEAM_WIDTHDefault number of the size of the queue maintained while searching during a graph construction.- See Also:
-
HNSW_COMPONENT
A name for the HNSW component for the info-stream *- See Also:
-
randSeed
public static long randSeedRandom seed for level generation; public to expose for testing * -
scorerSupplier
-
hnsw
-
hnswLock
protected final org.apache.lucene.util.hnsw.HnswLock hnswLock -
infoStream
-
frozen
protected boolean frozen
-
-
Constructor Details
-
HnswGraphBuilder
protected HnswGraphBuilder(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, int graphSize) throws IOException Reads all the vectors from vector values, builds a graph connecting them by their dense ordinals, using the given hyperparameter settings, and returns the resulting graph.- Parameters:
scorerSupplier
- a supplier to create vector scorer from ordinals.M
- – graph fanout parameter used to calculate the maximum number of connections a node can have – M on upper layers, and M * 2 on the lowest level.beamWidth
- the size of the beam search to use when finding nearest neighbors.seed
- the seed for a random number generator used during graph construction. Provide this to ensure repeatable construction.graphSize
- size of graph, if unknown, pass in -1- Throws:
IOException
-
HnswGraphBuilder
protected HnswGraphBuilder(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw) throws IOException - Throws:
IOException
-
HnswGraphBuilder
protected HnswGraphBuilder(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw, org.apache.lucene.util.hnsw.HnswLock hnswLock, HnswGraphSearcher graphSearcher) throws IOException Reads all the vectors from vector values, builds a graph connecting them by their dense ordinals, using the given hyperparameter settings, and returns the resulting graph.- Parameters:
scorerSupplier
- a supplier to create vector scorer from ordinals.M
- – graph fanout parameter used to calculate the maximum number of connections a node can have – M on upper layers, and M * 2 on the lowest level.beamWidth
- the size of the beam search to use when finding nearest neighbors.seed
- the seed for a random number generator used during graph construction. Provide this to ensure repeatable construction.hnsw
- the graph to build, can be previously initialized- Throws:
IOException
-
-
Method Details
-
create
public static HnswGraphBuilder create(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed) throws IOException - Throws:
IOException
-
create
public static HnswGraphBuilder create(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, int graphSize) throws IOException - Throws:
IOException
-
build
Description copied from interface:HnswBuilder
Adds all nodes to the graph up to the providedmaxOrd
.- Specified by:
build
in interfaceHnswBuilder
- Parameters:
maxOrd
- The maximum ordinal (excluded) of the nodes to be added.- Throws:
IOException
-
setInfoStream
Description copied from interface:HnswBuilder
Set info-stream to output debugging information- Specified by:
setInfoStream
in interfaceHnswBuilder
-
getCompletedGraph
Description copied from interface:HnswBuilder
Once this method is called no further updates to the graph are accepted (addGraphNode will throw IllegalStateException). Final modifications to the graph (eg patching up disconnected components, re-ordering node ids for better delta compression) may be triggered, so callers should expect this call to take some time.- Specified by:
getCompletedGraph
in interfaceHnswBuilder
- Throws:
IOException
-
getGraph
- Specified by:
getGraph
in interfaceHnswBuilder
-
addVectors
add vectors in range [minOrd, maxOrd)- Throws:
IOException
-
addGraphNode
- Throws:
IOException
-
addGraphNode
Description copied from interface:HnswBuilder
Inserts a doc with vector value to the graph- Specified by:
addGraphNode
in interfaceHnswBuilder
- Throws:
IOException
-
addGraphNodeWithEps
- Throws:
IOException
-