public final class CodecUtil extends Object
Writing codec headers is useful to ensure that a file is in the format you think it is.
Modifier and Type | Field and Description |
---|---|
static int |
CODEC_MAGIC
Constant to identify the start of a codec header.
|
static int |
FOOTER_MAGIC
Constant to identify the start of a codec footer.
|
Modifier and Type | Method and Description |
---|---|
static long |
checkFooter(ChecksumIndexInput in)
Validates the codec footer previously written by
writeFooter(org.apache.lucene.store.IndexOutput) . |
static void |
checkFooter(ChecksumIndexInput in,
Throwable priorException)
Validates the codec footer previously written by
writeFooter(org.apache.lucene.store.IndexOutput) , optionally
passing an unexpected exception that has already occurred. |
static int |
checkHeader(DataInput in,
String codec,
int minVersion,
int maxVersion)
Reads and validates a header previously written with
writeHeader(DataOutput, String, int) . |
static int |
checkHeaderNoMagic(DataInput in,
String codec,
int minVersion,
int maxVersion)
Like
checkHeader(DataInput,String,int,int) except this
version assumes the first int has already been read
and validated from the input. |
static int |
checkIndexHeader(DataInput in,
String codec,
int minVersion,
int maxVersion,
byte[] expectedID,
String expectedSuffix)
Reads and validates a header previously written with
writeIndexHeader(DataOutput, String, int, byte[], String) . |
static byte[] |
checkIndexHeaderID(DataInput in,
byte[] expectedID)
Expert: just reads and verifies the object ID of an index header
|
static String |
checkIndexHeaderSuffix(DataInput in,
String expectedSuffix)
Expert: just reads and verifies the suffix of an index header
|
static long |
checksumEntireFile(IndexInput input)
Clones the provided input, reads all bytes from the file, and calls
checkFooter(org.apache.lucene.store.ChecksumIndexInput) |
static int |
footerLength()
Computes the length of a codec footer.
|
static int |
headerLength(String codec)
Computes the length of a codec header.
|
static int |
indexHeaderLength(String codec,
String suffix)
Computes the length of an index header.
|
static byte[] |
readFooter(IndexInput in)
Retrieves the full footer from the provided
IndexInput . |
static byte[] |
readIndexHeader(IndexInput in)
Retrieves the full index header from the provided
IndexInput . |
static long |
retrieveChecksum(IndexInput in)
Returns (but does not validate) the checksum previously written by
checkFooter(org.apache.lucene.store.ChecksumIndexInput) . |
static void |
verifyAndCopyIndexHeader(IndexInput in,
DataOutput out,
byte[] expectedID)
Expert: verifies the incoming
IndexInput has an index header
and that its segment ID matches the expected one, and then copies
that index header into the provided DataOutput . |
static void |
writeFooter(IndexOutput out)
Writes a codec footer, which records both a checksum
algorithm ID and a checksum.
|
static void |
writeHeader(DataOutput out,
String codec,
int version)
Writes a codec header, which records both a string to
identify the file and a version number.
|
static void |
writeIndexHeader(DataOutput out,
String codec,
int version,
byte[] id,
String suffix)
Writes a codec header for an index file, which records both a string to
identify the format of the file, a version number, and data to identify
the file instance (ID and auxiliary suffix such as generation).
|
public static final int CODEC_MAGIC
public static final int FOOTER_MAGIC
public static void writeHeader(DataOutput out, String codec, int version) throws IOException
checkHeader()
.
CodecHeader --> Magic,CodecName,Version
Uint32
. This
identifies the start of the header. It is always 1071082519.
String
. This
is a string to identify this file.
Uint32
. Records
the version of the file.
Note that the length of a codec header depends only upon the
name of the codec, so this length can be computed at any time
with headerLength(String)
.
out
- Output streamcodec
- String to identify this file. It should be simple ASCII,
less than 128 characters in length.version
- Version numberIOException
- If there is an I/O error writing to the underlying medium.IllegalArgumentException
- If the codec name is not simple ASCII, or is more than 127 characters in lengthpublic static void writeIndexHeader(DataOutput out, String codec, int version, byte[] id, String suffix) throws IOException
This header can be parsed and validated with
checkIndexHeader()
.
IndexHeader --> CodecHeader,ObjectID,ObjectSuffix
writeHeader(org.apache.lucene.store.DataOutput, java.lang.String, int)
byte
16
byte
byte
SuffixLength
Note that the length of an index header depends only upon the
name of the codec and suffix, so this length can be computed at any time
with indexHeaderLength(String,String)
.
out
- Output streamcodec
- String to identify the format of this file. It should be simple ASCII,
less than 128 characters in length.id
- Unique identifier for this particular file instance.suffix
- auxiliary suffix information for the file. It should be simple ASCII,
less than 256 characters in length.version
- Version numberIOException
- If there is an I/O error writing to the underlying medium.IllegalArgumentException
- If the codec name is not simple ASCII, or
is more than 127 characters in length, or if id is invalid,
or if the suffix is not simple ASCII, or more than 255 characters
in length.public static int headerLength(String codec)
codec
- Codec name.writeHeader(DataOutput, String, int)
public static int indexHeaderLength(String codec, String suffix)
codec
- Codec name.writeIndexHeader(DataOutput, String, int, byte[], String)
public static int checkHeader(DataInput in, String codec, int minVersion, int maxVersion) throws IOException
writeHeader(DataOutput, String, int)
.
When reading a file, supply the expected codec
and
an expected version range (minVersion to maxVersion
).
in
- Input stream, positioned at the point where the
header was previously written. Typically this is located
at the beginning of the file.codec
- The expected codec name.minVersion
- The minimum supported expected version number.maxVersion
- The maximum supported expected version number.codec
, with an actual version
where minVersion <= actual <= maxVersion
.
Otherwise an exception is thrown.CorruptIndexException
- If the first four bytes are not
CODEC_MAGIC
, or if the actual codec found is
not codec
.IndexFormatTooOldException
- If the actual version is less
than minVersion
.IndexFormatTooNewException
- If the actual version is greater
than maxVersion
.IOException
- If there is an I/O error reading from the underlying medium.writeHeader(DataOutput, String, int)
public static int checkHeaderNoMagic(DataInput in, String codec, int minVersion, int maxVersion) throws IOException
checkHeader(DataInput,String,int,int)
except this
version assumes the first int has already been read
and validated from the input.IOException
public static int checkIndexHeader(DataInput in, String codec, int minVersion, int maxVersion, byte[] expectedID, String expectedSuffix) throws IOException
writeIndexHeader(DataOutput, String, int, byte[], String)
.
When reading a file, supply the expected codec
,
expected version range (minVersion to maxVersion
),
and object ID and suffix.
in
- Input stream, positioned at the point where the
header was previously written. Typically this is located
at the beginning of the file.codec
- The expected codec name.minVersion
- The minimum supported expected version number.maxVersion
- The maximum supported expected version number.expectedID
- The expected object identifier for this file.expectedSuffix
- The expected auxiliary suffix for this file.codec
, with an actual version
where minVersion <= actual <= maxVersion
,
and matching expectedID
and expectedSuffix
Otherwise an exception is thrown.CorruptIndexException
- If the first four bytes are not
CODEC_MAGIC
, or if the actual codec found is
not codec
, or if the expectedID
or expectedSuffix
do not match.IndexFormatTooOldException
- If the actual version is less
than minVersion
.IndexFormatTooNewException
- If the actual version is greater
than maxVersion
.IOException
- If there is an I/O error reading from the underlying medium.writeIndexHeader(DataOutput, String, int, byte[],String)
public static void verifyAndCopyIndexHeader(IndexInput in, DataOutput out, byte[] expectedID) throws IOException
IndexInput
has an index header
and that its segment ID matches the expected one, and then copies
that index header into the provided DataOutput
. This is
useful when building compound files.in
- Input stream, positioned at the point where the
index header was previously written. Typically this is located
at the beginning of the file.out
- Output stream, where the header will be copied to.expectedID
- Expected segment IDCorruptIndexException
- If the first four bytes are not
CODEC_MAGIC
, or if the expectedID
does not match.IOException
- If there is an I/O error reading from the underlying medium.public static byte[] readIndexHeader(IndexInput in) throws IOException
IndexInput
.
This throws CorruptIndexException
if this file does
not appear to be an index file.IOException
public static byte[] readFooter(IndexInput in) throws IOException
IndexInput
. This throws
CorruptIndexException
if this file does not have a valid footer.IOException
public static byte[] checkIndexHeaderID(DataInput in, byte[] expectedID) throws IOException
IOException
public static String checkIndexHeaderSuffix(DataInput in, String expectedSuffix) throws IOException
IOException
public static void writeFooter(IndexOutput out) throws IOException
checkFooter()
.
CodecFooter --> Magic,AlgorithmID,Checksum
Uint32
. This
identifies the start of the footer. It is always -1071082520.
Uint32
. This
indicates the checksum algorithm used. Currently this is always 0,
for zlib-crc32.
Uint64
. The
actual checksum value for all previous bytes in the stream, including
the bytes from Magic and AlgorithmID.
out
- Output streamIOException
- If there is an I/O error writing to the underlying medium.public static int footerLength()
writeFooter(IndexOutput)
public static long checkFooter(ChecksumIndexInput in) throws IOException
writeFooter(org.apache.lucene.store.IndexOutput)
.IOException
- if the footer is invalid, if the checksum does not match,
or if in
is not properly positioned before the footer
at the end of the stream.public static void checkFooter(ChecksumIndexInput in, Throwable priorException) throws IOException
writeFooter(org.apache.lucene.store.IndexOutput)
, optionally
passing an unexpected exception that has already occurred.
When a priorException
is provided, this method will add a suppressed exception
indicating whether the checksum for the stream passes, fails, or cannot be computed, and
rethrow it. Otherwise it behaves the same as checkFooter(ChecksumIndexInput)
.
Example usage:
try (ChecksumIndexInput input = ...) { Throwable priorE = null; try { // ... read a bunch of stuff ... } catch (Throwable exception) { priorE = exception; } finally { CodecUtil.checkFooter(input, priorE); } }
IOException
public static long retrieveChecksum(IndexInput in) throws IOException
checkFooter(org.apache.lucene.store.ChecksumIndexInput)
.IOException
- if the footer is invalidpublic static long checksumEntireFile(IndexInput input) throws IOException
checkFooter(org.apache.lucene.store.ChecksumIndexInput)
Note that this method may be slow, as it must process the entire file.
If you just need to extract the checksum value, call retrieveChecksum(org.apache.lucene.store.IndexInput)
.
IOException
Copyright © 2000-2019 Apache Software Foundation. All Rights Reserved.