BKDWriter (Lucene 6.6.1 API)

java.lang.Object
- org.apache.lucene.util.bkd.BKDWriter

All Implemented Interfaces:

Closeable, AutoCloseable
```
public class BKDWriter
extends Object
implements Closeable
```
Recursively builds a block KD-tree to assign all incoming points in N-dim space to smaller and smaller N-dim rectangles (cells) until the number of points in a given rectangle is <= maxPointsInLeafNode. The tree is fully balanced, which means the leaf nodes will have between 50% and 100% of the requested maxPointsInLeafNode. Values that fall exactly on a cell boundary may be in either cell.
The number of dimensions can be 1 to 8, but every byte[] value is fixed length.
See this paper for details.
This consumes heap during writing: it allocates a LongBitSet(numPoints), and then uses up to the specified maxMBSortInHeap heap space for writing.
NOTE: This can write at most Integer.MAX_VALUE * maxPointsInLeafNode total points.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary

Fields
Modifier and Type	Field and Description
`protected int`	`bytesPerDim` How many bytes each value in each dimension takes.
`static String`	`CODEC_NAME`
`static float`	`DEFAULT_MAX_MB_SORT_IN_HEAP` Default maximum heap to use, before spilling to (slower) disk
`static int`	`DEFAULT_MAX_POINTS_IN_LEAF_NODE` Default maximum number of point in each leaf block
`protected FixedBitSet`	`docsSeen`
`protected boolean`	`longOrds` true if we have so many values that we must write ords using long (8 bytes) instead of int (4 bytes)
`static int`	`MAX_DIMS` Maximum number of dimensions
`protected byte[]`	`maxPackedValue` Maximum per-dim values, packed
`protected int`	`maxPointsInLeafNode`
`protected byte[]`	`minPackedValue` Minimum per-dim values, packed
`protected int`	`numDims` How many dimensions we are indexing
`protected OfflineSorter.BufferSize`	`offlineSorterBufferMB` How much heap OfflineSorter is allowed to use
`protected int`	`offlineSorterMaxTempFiles` How much heap OfflineSorter is allowed to use
`protected int`	`packedBytesLength` numDims * bytesPerDim
`protected long`	`pointCount`
`protected boolean`	`singleValuePerDoc` True if every document has at most one value.
`static int`	`VERSION_COMPRESSED_DOC_IDS`
`static int`	`VERSION_COMPRESSED_VALUES`
`static int`	`VERSION_CURRENT`
`static int`	`VERSION_IMPLICIT_SPLIT_DIM_1D`
`static int`	`VERSION_PACKED_INDEX`
`static int`	`VERSION_START`

Constructor Summary

Constructors
Modifier	Constructor and Description
	`BKDWriter(int maxDoc, Directory tempDir, String tempFileNamePrefix, int numDims, int bytesPerDim, int maxPointsInLeafNode, double maxMBSortInHeap, long totalPointCount, boolean singleValuePerDoc)`
`protected`	`BKDWriter(int maxDoc, Directory tempDir, String tempFileNamePrefix, int numDims, int bytesPerDim, int maxPointsInLeafNode, double maxMBSortInHeap, long totalPointCount, boolean singleValuePerDoc, boolean longOrds, long offlineSorterBufferMB, int offlineSorterMaxTempFiles)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`add(byte[] packedValue, int docID)`
`void`	`close()`
`long`	`finish(IndexOutput out)` Writes the BKD tree to the provided `IndexOutput` and returns the file offset where index was written.
`long`	`getPointCount()` How many points have been added so far
`long`	`merge(IndexOutput out, List<MergeState.DocMap> docMaps, List<BKDReader> readers)` More efficient bulk-add for incoming `BKDReader`s.
`protected int`	`split(byte[] minPackedValue, byte[] maxPackedValue, int[] parentSplits)` Pick the next dimension to split.
`static void`	`verifyParams(int numDims, int maxPointsInLeafNode, double maxMBSortInHeap, long totalPointCount)`
`long`	`writeField(IndexOutput out, String fieldName, MutablePointsReader reader)` Write a field from a `MutablePointsReader`.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail
- CODEC_NAME
```
public static final String CODEC_NAME
```
  See Also:
  
  Constant Field Values
- VERSION_START
```
public static final int VERSION_START
```
  See Also:
  
  Constant Field Values
- VERSION_COMPRESSED_DOC_IDS
```
public static final int VERSION_COMPRESSED_DOC_IDS
```
  See Also:
  
  Constant Field Values
- VERSION_COMPRESSED_VALUES
```
public static final int VERSION_COMPRESSED_VALUES
```
  See Also:
  
  Constant Field Values
- VERSION_IMPLICIT_SPLIT_DIM_1D
```
public static final int VERSION_IMPLICIT_SPLIT_DIM_1D
```
  See Also:
  
  Constant Field Values
- VERSION_PACKED_INDEX
```
public static final int VERSION_PACKED_INDEX
```
  See Also:
  
  Constant Field Values
- VERSION_CURRENT
```
public static final int VERSION_CURRENT
```
  See Also:
  
  Constant Field Values
- DEFAULT_MAX_POINTS_IN_LEAF_NODE
```
public static final int DEFAULT_MAX_POINTS_IN_LEAF_NODE
```
  Default maximum number of point in each leaf block
  
  See Also:
  
  Constant Field Values
- DEFAULT_MAX_MB_SORT_IN_HEAP
```
public static final float DEFAULT_MAX_MB_SORT_IN_HEAP
```
  Default maximum heap to use, before spilling to (slower) disk
  
  See Also:
  
  Constant Field Values
- MAX_DIMS
```
public static final int MAX_DIMS
```
  Maximum number of dimensions
  
  See Also:
  
  Constant Field Values
- numDims
```
protected final int numDims
```
  How many dimensions we are indexing
- bytesPerDim
```
protected final int bytesPerDim
```
  How many bytes each value in each dimension takes.
- packedBytesLength
```
protected final int packedBytesLength
```
  numDims * bytesPerDim
- docsSeen
```
protected final FixedBitSet docsSeen
```
- maxPointsInLeafNode
```
protected final int maxPointsInLeafNode
```
- minPackedValue
```
protected final byte[] minPackedValue
```
  Minimum per-dim values, packed
- maxPackedValue
```
protected final byte[] maxPackedValue
```
  Maximum per-dim values, packed
- pointCount
```
protected long pointCount
```
- longOrds
```
protected final boolean longOrds
```
  true if we have so many values that we must write ords using long (8 bytes) instead of int (4 bytes)
- singleValuePerDoc
```
protected final boolean singleValuePerDoc
```
  True if every document has at most one value. We specialize this case by not bothering to store the ord since it's redundant with docID.
- offlineSorterBufferMB
```
protected final OfflineSorter.BufferSize offlineSorterBufferMB
```
  How much heap OfflineSorter is allowed to use
- offlineSorterMaxTempFiles
```
protected final int offlineSorterMaxTempFiles
```
  How much heap OfflineSorter is allowed to use

Constructor Detail

BKDWriter

public BKDWriter(int maxDoc,
                 Directory tempDir,
                 String tempFileNamePrefix,
                 int numDims,
                 int bytesPerDim,
                 int maxPointsInLeafNode,
                 double maxMBSortInHeap,
                 long totalPointCount,
                 boolean singleValuePerDoc)
          throws IOException

Throws:: IOException

BKDWriter

protected BKDWriter(int maxDoc,
                    Directory tempDir,
                    String tempFileNamePrefix,
                    int numDims,
                    int bytesPerDim,
                    int maxPointsInLeafNode,
                    double maxMBSortInHeap,
                    long totalPointCount,
                    boolean singleValuePerDoc,
                    boolean longOrds,
                    long offlineSorterBufferMB,
                    int offlineSorterMaxTempFiles)
             throws IOException

Throws:: IOException

Method Detail
- verifyParams
```
public static void verifyParams(int numDims,
                                int maxPointsInLeafNode,
                                double maxMBSortInHeap,
                                long totalPointCount)
```
- add
```
public void add(byte[] packedValue,
                int docID)
         throws IOException
```
  Throws:
  
  IOException
- getPointCount
```
public long getPointCount()
```
  How many points have been added so far
- writeField
```
public long writeField(IndexOutput out,
                       String fieldName,
                       MutablePointsReader reader)
                throws IOException
```
  Write a field from a MutablePointsReader. This way of writing points is faster than regular writes with add(byte[], int) since there is opportunity for reordering points before writing them to disk. This method does not use transient disk in order to reorder points.
  
  Throws:
  
  IOException
- merge
```
public long merge(IndexOutput out,
                  List<MergeState.DocMap> docMaps,
                  List<BKDReader> readers)
           throws IOException
```
  More efficient bulk-add for incoming BKDReaders. This does a merge sort of the already sorted values and currently only works when numDims==1. This returns -1 if all documents containing dimensional values were deleted.
  
  Throws:
  
  IOException
- finish
```
public long finish(IndexOutput out)
            throws IOException
```
  Writes the BKD tree to the provided IndexOutput and returns the file offset where index was written.
  
  Throws:
  
  IOException
- close
```
public void close()
           throws IOException
```
  Specified by:
  
  close in interface Closeable
  
  Specified by:
  
  close in interface AutoCloseable
  
  Throws:
  
  IOException
- split
```
protected int split(byte[] minPackedValue,
                    byte[] maxPackedValue,
                    int[] parentSplits)
```
  Pick the next dimension to split.
  
  Parameters:
  
  minPackedValue - the min values for all dimensions
  
  maxPackedValue - the max values for all dimensions
  
  parentSplits - how many times each dim has been split on the parent levels
  
  Returns:
  
  the dimension to split

Class BKDWriter

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

CODEC_NAME

VERSION_START

VERSION_COMPRESSED_DOC_IDS

VERSION_COMPRESSED_VALUES

VERSION_IMPLICIT_SPLIT_DIM_1D

VERSION_PACKED_INDEX

VERSION_CURRENT

DEFAULT_MAX_POINTS_IN_LEAF_NODE

DEFAULT_MAX_MB_SORT_IN_HEAP

MAX_DIMS

numDims

bytesPerDim

packedBytesLength

docsSeen

maxPointsInLeafNode

minPackedValue

maxPackedValue

pointCount

longOrds

singleValuePerDoc

offlineSorterBufferMB

offlineSorterMaxTempFiles

Constructor Detail

BKDWriter

BKDWriter

Method Detail

verifyParams

add

getPointCount

writeField

merge

finish

close

split