This writer generates a file which can be loaded into memory using memory-efficient data structures to quickly locate the block that contains any document.
In order to have a compact in-memory representation, for every block of 1024 chunks, this index computes the average number of bytes per chunk and for every chunk, only stores the difference between
Data is written as follows:
VInt, this marks the end of blocks since blocks are not allowed to start with 0
VIntwhich is the number of chunks encoded in the block
packedarray of BlockChunks elements of BitsPerDocBaseDelta bits each, representing the deltas from the average doc base using ZigZag encoding.
packedarray of BlockChunks elements of BitsPerStartPointerDelta bits each, representing the deltas from the average start pointer using ZigZag encoding
DocBase + AvgChunkDocs * n + DocBaseDeltas[n].
StartPointerBase + AvgChunkSize * n + StartPointerDeltas[n].
Copyright © 2000-2020 Apache Software Foundation. All Rights Reserved.