public class PackedInts extends Object
>= 0
and <=
a specified maximum value. The
values are stored as packed ints, with each value
consuming a fixed number of bits.Modifier and Type | Class and Description |
---|---|
static interface |
PackedInts.Decoder
A decoder for packed integers.
|
static interface |
PackedInts.Encoder
An encoder for packed integers.
|
static class |
PackedInts.Format
A format to write packed ints.
|
static class |
PackedInts.FormatAndBits
Simple class that holds a format and a number of bits per value.
|
static class |
PackedInts.Mutable
A packed integer array that can be modified.
|
static class |
PackedInts.NullReader
A
PackedInts.Reader which has all its values equal to 0 (bitsPerValue = 0). |
static class |
PackedInts.Reader
A read-only random access array of positive integers.
|
static interface |
PackedInts.ReaderIterator
Run-once iterator interface, to decode previously saved PackedInts.
|
static class |
PackedInts.Writer
A write-once Writer.
|
Modifier and Type | Field and Description |
---|---|
static String |
CODEC_NAME |
static float |
COMPACT
No memory overhead at all, but the returned implementation may be slow.
|
static float |
DEFAULT
At most 25% memory overhead.
|
static int |
DEFAULT_BUFFER_SIZE
Default amount of memory to use for bulk operations.
|
static float |
FAST
At most 50% memory overhead, always select a reasonably fast implementation.
|
static float |
FASTEST
At most 700% memory overhead, always select a direct implementation.
|
static int |
VERSION_CURRENT |
static int |
VERSION_MONOTONIC_WITHOUT_ZIGZAG |
static int |
VERSION_START |
Constructor and Description |
---|
PackedInts() |
Modifier and Type | Method and Description |
---|---|
static int |
bitsRequired(long maxValue)
Returns how many bits are required to hold values up
to and including maxValue
NOTE: This method returns at least 1.
|
static void |
checkVersion(int version)
Check the validity of a version number.
|
static void |
copy(PackedInts.Reader src,
int srcPos,
PackedInts.Mutable dest,
int destPos,
int len,
int mem)
Copy
src[srcPos:srcPos+len] into
dest[destPos:destPos+len] using at most mem
bytes. |
static PackedInts.FormatAndBits |
fastestFormatAndBits(int valueCount,
int bitsPerValue,
float acceptableOverheadRatio)
Try to find the
PackedInts.Format and number of bits per value that would
restore from disk the fastest reader whose overhead is less than
acceptableOverheadRatio . |
static PackedInts.Decoder |
getDecoder(PackedInts.Format format,
int version,
int bitsPerValue)
Get a
PackedInts.Decoder . |
static PackedInts.Reader |
getDirectReader(IndexInput in)
Construct a direct
PackedInts.Reader from an IndexInput . |
static PackedInts.Reader |
getDirectReaderNoHeader(IndexInput in,
PackedInts.Format format,
int version,
int valueCount,
int bitsPerValue)
Expert: Construct a direct
PackedInts.Reader from a stream without reading
metadata at the beginning of the stream. |
static PackedInts.Encoder |
getEncoder(PackedInts.Format format,
int version,
int bitsPerValue)
Get an
PackedInts.Encoder . |
static PackedInts.Mutable |
getMutable(int valueCount,
int bitsPerValue,
float acceptableOverheadRatio)
Create a packed integer array with the given amount of values initialized
to 0.
|
static PackedInts.Mutable |
getMutable(int valueCount,
int bitsPerValue,
PackedInts.Format format)
Same as
getMutable(int, int, float) with a pre-computed number
of bits per value and format. |
static PackedInts.Reader |
getReader(DataInput in)
Restore a
PackedInts.Reader from a stream. |
static PackedInts.ReaderIterator |
getReaderIterator(DataInput in,
int mem)
Retrieve PackedInts as a
PackedInts.ReaderIterator |
static PackedInts.ReaderIterator |
getReaderIteratorNoHeader(DataInput in,
PackedInts.Format format,
int version,
int valueCount,
int bitsPerValue,
int mem)
Expert: Restore a
PackedInts.ReaderIterator from a stream without reading
metadata at the beginning of the stream. |
static PackedInts.Reader |
getReaderNoHeader(DataInput in,
PackedInts.Format format,
int version,
int valueCount,
int bitsPerValue)
Expert: Restore a
PackedInts.Reader from a stream without reading metadata at
the beginning of the stream. |
static PackedInts.Writer |
getWriter(DataOutput out,
int valueCount,
int bitsPerValue,
float acceptableOverheadRatio)
Create a packed integer array writer for the given output, format, value
count, and number of bits per value.
|
static PackedInts.Writer |
getWriterNoHeader(DataOutput out,
PackedInts.Format format,
int valueCount,
int bitsPerValue,
int mem)
Expert: Create a packed integer array writer for the given output, format,
value count, and number of bits per value.
|
static long |
maxValue(int bitsPerValue)
Calculates the maximum unsigned long that can be expressed with the given
number of bits.
|
static int |
unsignedBitsRequired(long bits)
Returns how many bits are required to store
bits ,
interpreted as an unsigned value. |
public static final float FASTEST
public static final float FAST
public static final float DEFAULT
public static final float COMPACT
public static final int DEFAULT_BUFFER_SIZE
public static final String CODEC_NAME
public static final int VERSION_MONOTONIC_WITHOUT_ZIGZAG
public static final int VERSION_START
public static final int VERSION_CURRENT
public static void checkVersion(int version)
public static PackedInts.FormatAndBits fastestFormatAndBits(int valueCount, int bitsPerValue, float acceptableOverheadRatio)
PackedInts.Format
and number of bits per value that would
restore from disk the fastest reader whose overhead is less than
acceptableOverheadRatio
.
The acceptableOverheadRatio
parameter makes sense for
random-access PackedInts.Reader
s. In case you only plan to perform
sequential access on this stream later on, you should probably use
COMPACT
.
If you don't know how many values you are going to write, use
valueCount = -1
.
public static PackedInts.Decoder getDecoder(PackedInts.Format format, int version, int bitsPerValue)
PackedInts.Decoder
.format
- the format used to store packed intsversion
- the compatibility versionbitsPerValue
- the number of bits per valuepublic static PackedInts.Encoder getEncoder(PackedInts.Format format, int version, int bitsPerValue)
PackedInts.Encoder
.format
- the format used to store packed intsversion
- the compatibility versionbitsPerValue
- the number of bits per valuepublic static PackedInts.Reader getReaderNoHeader(DataInput in, PackedInts.Format format, int version, int valueCount, int bitsPerValue) throws IOException
PackedInts.Reader
from a stream without reading metadata at
the beginning of the stream. This method is useful to restore data from
streams which have been created using
getWriterNoHeader(DataOutput, Format, int, int, int)
.in
- the stream to read data from, positioned at the beginning of the packed valuesformat
- the format used to serializeversion
- the version used to serialize the datavalueCount
- how many values the stream holdsbitsPerValue
- the number of bits per valueIOException
- If there is a low-level I/O errorgetWriterNoHeader(DataOutput, Format, int, int, int)
public static PackedInts.Reader getReader(DataInput in) throws IOException
PackedInts.Reader
from a stream.in
- the stream to read data fromIOException
- If there is a low-level I/O errorpublic static PackedInts.ReaderIterator getReaderIteratorNoHeader(DataInput in, PackedInts.Format format, int version, int valueCount, int bitsPerValue, int mem)
PackedInts.ReaderIterator
from a stream without reading
metadata at the beginning of the stream. This method is useful to restore
data from streams which have been created using
getWriterNoHeader(DataOutput, Format, int, int, int)
.in
- the stream to read data from, positioned at the beginning of the packed valuesformat
- the format used to serializeversion
- the version used to serialize the datavalueCount
- how many values the stream holdsbitsPerValue
- the number of bits per valuemem
- how much memory the iterator is allowed to use to read-ahead (likely to speed up iteration)getWriterNoHeader(DataOutput, Format, int, int, int)
public static PackedInts.ReaderIterator getReaderIterator(DataInput in, int mem) throws IOException
PackedInts.ReaderIterator
in
- positioned at the beginning of a stored packed int structure.mem
- how much memory the iterator is allowed to use to read-ahead (likely to speed up iteration)IOException
- if the structure could not be retrieved.public static PackedInts.Reader getDirectReaderNoHeader(IndexInput in, PackedInts.Format format, int version, int valueCount, int bitsPerValue)
PackedInts.Reader
from a stream without reading
metadata at the beginning of the stream. This method is useful to restore
data from streams which have been created using
getWriterNoHeader(DataOutput, Format, int, int, int)
.
The returned reader will have very little memory overhead, but every call
to NumericDocValues.get(int)
is likely to perform a disk seek.
in
- the stream to read data fromformat
- the format used to serializeversion
- the version used to serialize the datavalueCount
- how many values the stream holdsbitsPerValue
- the number of bits per valuepublic static PackedInts.Reader getDirectReader(IndexInput in) throws IOException
PackedInts.Reader
from an IndexInput
. This method
is useful to restore data from streams which have been created using
getWriter(DataOutput, int, int, float)
.
The returned reader will have very little memory overhead, but every call
to NumericDocValues.get(int)
is likely to perform a disk seek.
in
- the stream to read data fromIOException
- If there is a low-level I/O errorpublic static PackedInts.Mutable getMutable(int valueCount, int bitsPerValue, float acceptableOverheadRatio)
Positive values of acceptableOverheadRatio
will trade space
for speed by selecting a faster but potentially less memory-efficient
implementation. An acceptableOverheadRatio
of
COMPACT
will make sure that the most memory-efficient
implementation is selected whereas FASTEST
will make sure
that the fastest implementation is selected.
valueCount
- the number of elementsbitsPerValue
- the number of bits available for any given valueacceptableOverheadRatio
- an acceptable overhead
ratio per valuepublic static PackedInts.Mutable getMutable(int valueCount, int bitsPerValue, PackedInts.Format format)
getMutable(int, int, float)
with a pre-computed number
of bits per value and format.public static PackedInts.Writer getWriterNoHeader(DataOutput out, PackedInts.Format format, int valueCount, int bitsPerValue, int mem)
The resulting stream will be long-aligned. This means that depending on
the format which is used, up to 63 bits will be wasted. An easy way to
make sure that no space is lost is to always use a valueCount
that is a multiple of 64.
This method does not write any metadata to the stream, meaning that it is your responsibility to store it somewhere else in order to be able to recover data from the stream later on:
format
(using PackedInts.Format.getId()
),valueCount
,bitsPerValue
,VERSION_CURRENT
.
It is possible to start writing values without knowing how many of them you
are actually going to write. To do this, just pass -1
as
valueCount
. On the other hand, for any positive value of
valueCount
, the returned writer will make sure that you don't
write more values than expected and pad the end of stream with zeros in
case you have written less than valueCount
when calling
PackedInts.Writer.finish()
.
The mem
parameter lets you control how much memory can be used
to buffer changes in memory before flushing to disk. High values of
mem
are likely to improve throughput. On the other hand, if
speed is not that important to you, a value of 0
will use as
little memory as possible and should already offer reasonable throughput.
out
- the data outputformat
- the format to use to serialize the valuesvalueCount
- the number of valuesbitsPerValue
- the number of bits per valuemem
- how much memory (in bytes) can be used to speed up serializationgetReaderIteratorNoHeader(DataInput, Format, int, int, int, int)
,
getReaderNoHeader(DataInput, Format, int, int, int)
public static PackedInts.Writer getWriter(DataOutput out, int valueCount, int bitsPerValue, float acceptableOverheadRatio) throws IOException
The resulting stream will be long-aligned. This means that depending on
the format which is used under the hoods, up to 63 bits will be wasted.
An easy way to make sure that no space is lost is to always use a
valueCount
that is a multiple of 64.
This method writes metadata to the stream, so that the resulting stream is
sufficient to restore a PackedInts.Reader
from it. You don't need to track
valueCount
or bitsPerValue
by yourself. In case
this is a problem, you should probably look at
getWriterNoHeader(DataOutput, Format, int, int, int)
.
The acceptableOverheadRatio
parameter controls how
readers that will be restored from this stream trade space
for speed by selecting a faster but potentially less memory-efficient
implementation. An acceptableOverheadRatio
of
COMPACT
will make sure that the most memory-efficient
implementation is selected whereas FASTEST
will make sure
that the fastest implementation is selected. In case you are only interested
in reading this stream sequentially later on, you should probably use
COMPACT
.
out
- the data outputvalueCount
- the number of valuesbitsPerValue
- the number of bits per valueacceptableOverheadRatio
- an acceptable overhead ratio per valueIOException
- If there is a low-level I/O errorpublic static int bitsRequired(long maxValue)
maxValue
- the maximum value that should be representable.public static int unsignedBitsRequired(long bits)
bits
,
interpreted as an unsigned value.
NOTE: This method returns at least 1.public static long maxValue(int bitsPerValue)
bitsPerValue
- the number of bits available for any given value.public static void copy(PackedInts.Reader src, int srcPos, PackedInts.Mutable dest, int destPos, int len, int mem)
src[srcPos:srcPos+len]
into
dest[destPos:destPos+len]
using at most mem
bytes.Copyright © 2000-2016 Apache Software Foundation. All Rights Reserved.