Package org.apache.lucene.store

Class ByteBuffersDataOutput

java.lang.Object

org.apache.lucene.store.DataOutput

org.apache.lucene.store.ByteBuffersDataOutput

All Implemented Interfaces:

Accountable

public final class ByteBuffersDataOutput
extends DataOutput
implements Accountable

A DataOutput storing data in a list of ByteBuffers.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ByteBuffersDataOutput.ByteBufferRecycler	An implementation of a ByteBuffer allocation and recycling policy.

Field Summary

Fields

Modifier and Type	Field	Description
static IntFunction<ByteBuffer>	ALLOCATE_BB_ON_HEAP
static int	DEFAULT_MAX_BITS_PER_BLOCK	Default maxBitsPerBlock
static int	DEFAULT_MIN_BITS_PER_BLOCK	Default minBitsPerBlock
static int	LIMIT_MAX_BITS_PER_BLOCK	Largest maxBitsPerBlock allowed
static int	LIMIT_MIN_BITS_PER_BLOCK	Smallest minBitsPerBlock allowed
static Consumer<ByteBuffer>	NO_REUSE	A singleton instance of "no-reuse" buffer strategy.

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Constructor	Description
ByteBuffersDataOutput()	Creates a new output with all defaults.
ByteBuffersDataOutput(int minBitsPerBlock, int maxBitsPerBlock, IntFunction<ByteBuffer> blockAllocate, Consumer<ByteBuffer> blockReuse)	Expert: Creates a new output with custom parameters.
ByteBuffersDataOutput(long expectedSize)	Create a new output, suitable for writing a file of around expectedSize bytes.

Method Summary

Modifier and Type	Method	Description
void	copyBytes(DataInput input, long numBytes)	Copy numBytes bytes from input to ourself.
void	copyTo(DataOutput output)	Copy the current content of this object into another DataOutput.
static ByteBuffersDataOutput	newResettableInstance()
long	ramBytesUsed()	Return the memory usage of this object in bytes.
void	reset()	This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor.
long	size()
byte[]	toArrayCopy()	Return a contiguous array with the current content written to the output.
ArrayList<ByteBuffer>	toBufferList()	Return a list of read-only view of ByteBuffer blocks over the current content written to the output.
ByteBuffersDataInput	toDataInput()	Return a ByteBuffersDataInput for the set of current buffers (toBufferList()).
String	toString()
ArrayList<ByteBuffer>	toWriteableBufferList()	Returns a list of writeable blocks over the (source) content buffers.
void	writeByte(byte b)	Writes a single byte.
void	writeBytes(byte[] b)
void	writeBytes(byte[] b, int length)	Writes an array of bytes.
void	writeBytes(byte[] src, int offset, int length)	Writes an array of bytes.
void	writeBytes(ByteBuffer buffer)
void	writeInt(int v)	Writes an int as four bytes (LE byte order).
void	writeLong(long v)	Writes a long as eight bytes (LE byte order).
void	writeMapOfStrings(Map<String,String> map)	Writes a String map.
void	writeSetOfStrings(Set<String> set)	Writes a String set.
void	writeShort(short v)	Writes a short as two bytes (LE byte order).
void	writeString(String v)	Writes a string.

Methods inherited from class org.apache.lucene.store.DataOutput

writeGroupVInts, writeVInt, writeVLong, writeZInt, writeZLong

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Field Detail

ALLOCATE_BB_ON_HEAP

public static final IntFunction<ByteBuffer> ALLOCATE_BB_ON_HEAP

NO_REUSE

public static final Consumer<ByteBuffer> NO_REUSE

A singleton instance of "no-reuse" buffer strategy.

DEFAULT_MIN_BITS_PER_BLOCK

public static final int DEFAULT_MIN_BITS_PER_BLOCK

Default minBitsPerBlock

See Also:

Constant Field Values

DEFAULT_MAX_BITS_PER_BLOCK

public static final int DEFAULT_MAX_BITS_PER_BLOCK

Default maxBitsPerBlock

See Also:

Constant Field Values

LIMIT_MIN_BITS_PER_BLOCK

public static final int LIMIT_MIN_BITS_PER_BLOCK

Smallest minBitsPerBlock allowed

See Also:

Constant Field Values

LIMIT_MAX_BITS_PER_BLOCK

public static final int LIMIT_MAX_BITS_PER_BLOCK

Largest maxBitsPerBlock allowed

See Also:

Constant Field Values

Constructor Detail

ByteBuffersDataOutput

public ByteBuffersDataOutput(long expectedSize)

Create a new output, suitable for writing a file of around expectedSize bytes.

Memory allocation will be optimized based on the expectedSize hint, so that there is less overhead for larger files.

Parameters:

expectedSize - estimated size of the output file

ByteBuffersDataOutput

public ByteBuffersDataOutput()

Creates a new output with all defaults.

ByteBuffersDataOutput

public ByteBuffersDataOutput(int minBitsPerBlock,
                             int maxBitsPerBlock,
                             IntFunction<ByteBuffer> blockAllocate,
                             Consumer<ByteBuffer> blockReuse)

Expert: Creates a new output with custom parameters.

Parameters:

minBitsPerBlock - minimum bits per block

maxBitsPerBlock - maximum bits per block

blockAllocate - block allocator

blockReuse - block recycler

Method Detail

writeByte

public void writeByte(byte b)

Description copied from class: DataOutput

Writes a single byte.

The most primitive data type is an eight-bit byte. Files are accessed as sequences of bytes. All other data types are defined as sequences of bytes, so file formats are byte-order independent.

Specified by:

writeByte in class DataOutput

See Also:

DataInput.readByte()

writeBytes

public void writeBytes(byte[] src,
                       int offset,
                       int length)

Description copied from class: DataOutput

Writes an array of bytes.

Specified by:

writeBytes in class DataOutput

Parameters:

src - the bytes to write

offset - the offset in the byte array

length - the number of bytes to write

See Also:

DataInput.readBytes(byte[],int,int)

writeBytes

public void writeBytes(byte[] b,
                       int length)

Description copied from class: DataOutput

Writes an array of bytes.

Overrides:

writeBytes in class DataOutput

Parameters:

b - the bytes to write

length - the number of bytes to write

See Also:

DataInput.readBytes(byte[],int,int)

writeBytes

public void writeBytes(byte[] b)

writeBytes

public void writeBytes(ByteBuffer buffer)

toBufferList

public ArrayList<ByteBuffer> toBufferList()

Return a list of read-only view of ByteBuffer blocks over the current content written to the output.

toWriteableBufferList

public ArrayList<ByteBuffer> toWriteableBufferList()

Returns a list of writeable blocks over the (source) content buffers.

This method returns the raw content of source buffers that may change over the lifetime of this object (blocks can be recycled or discarded, for example). Most applications should favor calling toBufferList() which returns a read-only view over the content of the source buffers.

The difference between toBufferList() and toWriteableBufferList() is that read-only view of source buffers will always return false from ByteBuffer.hasArray() (which sometimes may be required to avoid double copying).

toDataInput

public ByteBuffersDataInput toDataInput()

Return a ByteBuffersDataInput for the set of current buffers (toBufferList()).

toArrayCopy

public byte[] toArrayCopy()

Return a contiguous array with the current content written to the output. The returned array is always a copy (can be mutated).

If the size() of the underlying buffers exceeds maximum size of Java array, an RuntimeException will be thrown.

copyBytes

public void copyBytes(DataInput input,
                      long numBytes)
               throws IOException

Description copied from class: DataOutput

Copy numBytes bytes from input to ourself.

Overrides:

copyBytes in class DataOutput

Throws:

IOException

copyTo

public void copyTo(DataOutput output)
            throws IOException

Copy the current content of this object into another DataOutput.

Throws:

IOException

size

public long size()

Returns:

The number of bytes written to this output so far.

toString

public String toString()

Overrides:

toString in class Object

writeShort

public void writeShort(short v)

Description copied from class: DataOutput

Writes a short as two bytes (LE byte order).

Overrides:

writeShort in class DataOutput

See Also:

DataInput.readShort(), BitUtil.VH_LE_SHORT

writeInt

public void writeInt(int v)

Description copied from class: DataOutput

Writes an int as four bytes (LE byte order).

Overrides:

writeInt in class DataOutput

See Also:

DataInput.readInt(), BitUtil.VH_LE_INT

writeLong

public void writeLong(long v)

Description copied from class: DataOutput

Writes a long as eight bytes (LE byte order).

Overrides:

writeLong in class DataOutput

See Also:

DataInput.readLong(), BitUtil.VH_LE_LONG

writeString

public void writeString(String v)

Description copied from class: DataOutput

Writes a string.

Writes strings as UTF-8 encoded bytes. First the length, in bytes, is written as a VInt, followed by the bytes.

Overrides:

writeString in class DataOutput

See Also:

DataInput.readString()

writeMapOfStrings

public void writeMapOfStrings(Map<String,String> map)

Description copied from class: DataOutput

Writes a String map.

First the size is written as an vInt, followed by each key-value pair written as two consecutive Strings.

Overrides:

writeMapOfStrings in class DataOutput

Parameters:

map - Input map.

writeSetOfStrings

public void writeSetOfStrings(Set<String> set)

Description copied from class: DataOutput

Writes a String set.

First the size is written as an vInt, followed by each value written as a String.

Overrides:

writeSetOfStrings in class DataOutput

Parameters:

set - Input set.

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

reset

public void reset()

This method resets this object to a clean (zero-size) state and publishes any currently allocated buffers for reuse to the reuse strategy provided in the constructor.

Sharing byte buffers for reads and writes is dangerous and will very likely lead to hard-to-debug issues, use with great care.

newResettableInstance

public static ByteBuffersDataOutput newResettableInstance()

Returns:

Returns a new ByteBuffersDataOutput with the reset() capability.