org.apache.lucene.util

Class OfflineSorter

java.lang.Object

org.apache.lucene.util.OfflineSorter

public final class OfflineSorter
extends Object

On-disk sorting of byte arrays. Each byte array (entry) is a composed of the following fields:

• (two bytes) length of the following byte array,
• exactly the above count of bytes for the sequence to be sorted.

See Also:

sort(Path, Path)

WARNING: This API is experimental and might change in incompatible ways in the next release.

NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	OfflineSorter.BufferSize A bit more descriptive unit for constructors.
static class	OfflineSorter.ByteSequencesReader Utility class to read length-prefixed byte[] entries from an input.
static class	OfflineSorter.ByteSequencesWriter Utility class to emit length-prefixed byte[] entries to an output stream for sorting.
class	OfflineSorter.SortInfo Sort info (debugging mostly).

Field Summary

Fields

Modifier and Type	Field and Description
static long	ABSOLUTE_MIN_SORT_BUFFER_SIZE Absolute minimum required buffer size for sorting.
static Comparator<BytesRef>	DEFAULT_COMPARATOR Default comparator: sorts in binary (codepoint) order
static long	GB Convenience constant for gigabytes
static int	MAX_TEMPFILES Maximum number of temporary files before doing an intermediate merge.
static long	MB Convenience constant for megabytes
static long	MIN_BUFFER_SIZE_MB Minimum recommended buffer size for sorting.

Constructor Summary

Constructors

Constructor and Description
OfflineSorter() Defaults constructor.
OfflineSorter(Comparator<BytesRef> comparator) Defaults constructor with a custom comparator.
OfflineSorter(Comparator<BytesRef> comparator, OfflineSorter.BufferSize ramBufferSize, Path tempDirectory, int maxTempfiles) All-details constructor.

Method Summary

Methods

Modifier and Type	Method and Description
Comparator<BytesRef>	getComparator() Returns the comparator in use to sort entries
static Path	getDefaultTempDir() Returns the default temporary directory.
OfflineSorter.SortInfo	sort(Path input, Path output) Sort input to output, explicit hint for the buffer size.
protected Path	sortPartition(int len) Sort a single partition in-memory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MB

public static final long MB

Convenience constant for megabytes

See Also:

Constant Field Values

GB

public static final long GB

Convenience constant for gigabytes

See Also:

Constant Field Values

MIN_BUFFER_SIZE_MB

public static final long MIN_BUFFER_SIZE_MB

Minimum recommended buffer size for sorting.

See Also:

Constant Field Values

ABSOLUTE_MIN_SORT_BUFFER_SIZE

public static final long ABSOLUTE_MIN_SORT_BUFFER_SIZE

Absolute minimum required buffer size for sorting.

See Also:

Constant Field Values

MAX_TEMPFILES

public static final int MAX_TEMPFILES

Maximum number of temporary files before doing an intermediate merge.

See Also:

Constant Field Values

DEFAULT_COMPARATOR

public static final Comparator<BytesRef> DEFAULT_COMPARATOR

Default comparator: sorts in binary (codepoint) order

Constructor Detail

OfflineSorter

public OfflineSorter()
              throws IOException

Defaults constructor.

Throws:

IOException

See Also:

getDefaultTempDir(), OfflineSorter.BufferSize.automatic()

OfflineSorter

public OfflineSorter(Comparator<BytesRef> comparator)
              throws IOException

Defaults constructor with a custom comparator.

Throws:

IOException

See Also:

getDefaultTempDir(), OfflineSorter.BufferSize.automatic()

OfflineSorter

public OfflineSorter(Comparator<BytesRef> comparator,
             OfflineSorter.BufferSize ramBufferSize,
             Path tempDirectory,
             int maxTempfiles)

All-details constructor.

Method Detail

sort

public OfflineSorter.SortInfo sort(Path input,
                          Path output)
                            throws IOException

Sort input to output, explicit hint for the buffer size. The amount of allocated memory may deviate from the hint (may be smaller or larger).

Throws:

IOException

getDefaultTempDir

public static Path getDefaultTempDir()
                              throws IOException

Returns the default temporary directory. By default, java.io.tmpdir. If not accessible or not available, an IOException is thrown

Throws:

IOException

sortPartition

protected Path sortPartition(int len)
                      throws IOException

Sort a single partition in-memory.

Throws:

IOException

getComparator

public Comparator<BytesRef> getComparator()

Returns the comparator in use to sort entries