Class CodecUtil
Writing codec headers is useful to ensure that a file is in the format you think it is.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Field Summary
Modifier and TypeFieldDescriptionstatic final int
Constant to identify the start of a codec header.static final int
Constant to identify the start of a codec footer. -
Method Summary
Modifier and TypeMethodDescriptionstatic long
Validates the codec footer previously written bywriteFooter(org.apache.lucene.store.IndexOutput)
.static void
checkFooter
(ChecksumIndexInput in, Throwable priorException) Validates the codec footer previously written bywriteFooter(org.apache.lucene.store.IndexOutput)
, optionally passing an unexpected exception that has already occurred.static int
checkHeader
(DataInput in, String codec, int minVersion, int maxVersion) Reads and validates a header previously written withwriteHeader(DataOutput, String, int)
.static int
checkHeaderNoMagic
(DataInput in, String codec, int minVersion, int maxVersion) LikecheckHeader(DataInput,String,int,int)
except this version assumes the first int has already been read and validated from the input.static int
checkIndexHeader
(DataInput in, String codec, int minVersion, int maxVersion, byte[] expectedID, String expectedSuffix) Reads and validates a header previously written withwriteIndexHeader(DataOutput, String, int, byte[], String)
.static byte[]
checkIndexHeaderID
(DataInput in, byte[] expectedID) Expert: just reads and verifies the object ID of an index headerstatic String
checkIndexHeaderSuffix
(DataInput in, String expectedSuffix) Expert: just reads and verifies the suffix of an index headerstatic long
checksumEntireFile
(IndexInput input) Clones the provided input, reads all bytes from the file, and callscheckFooter(org.apache.lucene.store.ChecksumIndexInput)
static int
Computes the length of a codec footer.static int
headerLength
(String codec) Computes the length of a codec header.static int
indexHeaderLength
(String codec, String suffix) Computes the length of an index header.static int
read int value from header / footer with big endian orderstatic long
readBELong
(DataInput in) read long value from header / footer with big endian orderstatic byte[]
readFooter
(IndexInput in) Retrieves the full footer from the providedIndexInput
.static byte[]
Retrieves the full index header from the providedIndexInput
.static long
Returns (but does not validate) the checksum previously written bycheckFooter(org.apache.lucene.store.ChecksumIndexInput)
.static long
retrieveChecksum
(IndexInput in, long expectedLength) Returns (but does not validate) the checksum previously written bycheckFooter(org.apache.lucene.store.ChecksumIndexInput)
.static void
verifyAndCopyIndexHeader
(IndexInput in, DataOutput out, byte[] expectedID) Expert: verifies the incomingIndexInput
has an index header and that its segment ID matches the expected one, and then copies that index header into the providedDataOutput
.static void
writeBEInt
(DataOutput out, int i) write int value on header / footer with big endian orderstatic void
writeBELong
(DataOutput out, long l) write long value on header / footer with big endian orderstatic void
writeFooter
(IndexOutput out) Writes a codec footer, which records both a checksum algorithm ID and a checksum.static void
writeHeader
(DataOutput out, String codec, int version) Writes a codec header, which records both a string to identify the file and a version number.static void
writeIndexHeader
(DataOutput out, String codec, int version, byte[] id, String suffix) Writes a codec header for an index file, which records both a string to identify the format of the file, a version number, and data to identify the file instance (ID and auxiliary suffix such as generation).
-
Field Details
-
CODEC_MAGIC
public static final int CODEC_MAGICConstant to identify the start of a codec header.- See Also:
-
FOOTER_MAGIC
public static final int FOOTER_MAGICConstant to identify the start of a codec footer.- See Also:
-
-
Method Details
-
writeHeader
Writes a codec header, which records both a string to identify the file and a version number. This header can be parsed and validated withcheckHeader()
.CodecHeader --> Magic,CodecName,Version
- Magic -->
Uint32
. This identifies the start of the header. It is always 1071082519. - CodecName -->
String
. This is a string to identify this file. - Version -->
Uint32
. Records the version of the file.
Note that the length of a codec header depends only upon the name of the codec, so this length can be computed at any time with
headerLength(String)
.- Parameters:
out
- Output streamcodec
- String to identify this file. It should be simple ASCII, less than 128 characters in length.version
- Version number- Throws:
IOException
- If there is an I/O error writing to the underlying medium.IllegalArgumentException
- If the codec name is not simple ASCII, or is more than 127 characters in length
- Magic -->
-
writeIndexHeader
public static void writeIndexHeader(DataOutput out, String codec, int version, byte[] id, String suffix) throws IOException Writes a codec header for an index file, which records both a string to identify the format of the file, a version number, and data to identify the file instance (ID and auxiliary suffix such as generation).This header can be parsed and validated with
checkIndexHeader()
.IndexHeader --> CodecHeader,ObjectID,ObjectSuffix
- CodecHeader -->
writeHeader(org.apache.lucene.store.DataOutput, java.lang.String, int)
- ObjectID -->
byte
16 - ObjectSuffix --> SuffixLength,SuffixBytes
- SuffixLength -->
byte
- SuffixBytes -->
byte
SuffixLength
Note that the length of an index header depends only upon the name of the codec and suffix, so this length can be computed at any time with
indexHeaderLength(String,String)
.- Parameters:
out
- Output streamcodec
- String to identify the format of this file. It should be simple ASCII, less than 128 characters in length.id
- Unique identifier for this particular file instance.suffix
- auxiliary suffix information for the file. It should be simple ASCII, less than 256 characters in length.version
- Version number- Throws:
IOException
- If there is an I/O error writing to the underlying medium.IllegalArgumentException
- If the codec name is not simple ASCII, or is more than 127 characters in length, or if id is invalid, or if the suffix is not simple ASCII, or more than 255 characters in length.
- CodecHeader -->
-
headerLength
Computes the length of a codec header.- Parameters:
codec
- Codec name.- Returns:
- length of the entire codec header.
- See Also:
-
indexHeaderLength
Computes the length of an index header.- Parameters:
codec
- Codec name.- Returns:
- length of the entire index header.
- See Also:
-
checkHeader
public static int checkHeader(DataInput in, String codec, int minVersion, int maxVersion) throws IOException Reads and validates a header previously written withwriteHeader(DataOutput, String, int)
.When reading a file, supply the expected
codec
and an expected version range (minVersion to maxVersion
).- Parameters:
in
- Input stream, positioned at the point where the header was previously written. Typically this is located at the beginning of the file.codec
- The expected codec name.minVersion
- The minimum supported expected version number.maxVersion
- The maximum supported expected version number.- Returns:
- The actual version found, when a valid header is found that matches
codec
, with an actual version whereminVersion <= actual <= maxVersion
. Otherwise an exception is thrown. - Throws:
CorruptIndexException
- If the first four bytes are notCODEC_MAGIC
, or if the actual codec found is notcodec
.IndexFormatTooOldException
- If the actual version is less thanminVersion
.IndexFormatTooNewException
- If the actual version is greater thanmaxVersion
.IOException
- If there is an I/O error reading from the underlying medium.- See Also:
-
checkHeaderNoMagic
public static int checkHeaderNoMagic(DataInput in, String codec, int minVersion, int maxVersion) throws IOException LikecheckHeader(DataInput,String,int,int)
except this version assumes the first int has already been read and validated from the input.- Throws:
IOException
-
checkIndexHeader
public static int checkIndexHeader(DataInput in, String codec, int minVersion, int maxVersion, byte[] expectedID, String expectedSuffix) throws IOException Reads and validates a header previously written withwriteIndexHeader(DataOutput, String, int, byte[], String)
.When reading a file, supply the expected
codec
, expected version range (minVersion to maxVersion
), and object ID and suffix.- Parameters:
in
- Input stream, positioned at the point where the header was previously written. Typically this is located at the beginning of the file.codec
- The expected codec name.minVersion
- The minimum supported expected version number.maxVersion
- The maximum supported expected version number.expectedID
- The expected object identifier for this file.expectedSuffix
- The expected auxiliary suffix for this file.- Returns:
- The actual version found, when a valid header is found that matches
codec
, with an actual version whereminVersion <= actual <= maxVersion
, and matchingexpectedID
andexpectedSuffix
Otherwise an exception is thrown. - Throws:
CorruptIndexException
- If the first four bytes are notCODEC_MAGIC
, or if the actual codec found is notcodec
, or if theexpectedID
orexpectedSuffix
do not match.IndexFormatTooOldException
- If the actual version is less thanminVersion
.IndexFormatTooNewException
- If the actual version is greater thanmaxVersion
.IOException
- If there is an I/O error reading from the underlying medium.- See Also:
-
verifyAndCopyIndexHeader
public static void verifyAndCopyIndexHeader(IndexInput in, DataOutput out, byte[] expectedID) throws IOException Expert: verifies the incomingIndexInput
has an index header and that its segment ID matches the expected one, and then copies that index header into the providedDataOutput
. This is useful when building compound files.- Parameters:
in
- Input stream, positioned at the point where the index header was previously written. Typically this is located at the beginning of the file.out
- Output stream, where the header will be copied to.expectedID
- Expected segment ID- Throws:
CorruptIndexException
- If the first four bytes are notCODEC_MAGIC
, or if theexpectedID
does not match.IOException
- If there is an I/O error reading from the underlying medium.- NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.
-
readIndexHeader
Retrieves the full index header from the providedIndexInput
. This throwsCorruptIndexException
if this file does not appear to be an index file.- Throws:
IOException
-
checkIndexHeaderID
Expert: just reads and verifies the object ID of an index header- Throws:
IOException
-
checkIndexHeaderSuffix
Expert: just reads and verifies the suffix of an index header- Throws:
IOException
-
retrieveChecksum
Returns (but does not validate) the checksum previously written bycheckFooter(org.apache.lucene.store.ChecksumIndexInput)
.- Returns:
- actual checksum value
- Throws:
IOException
- if the footer is invalid
-
retrieveChecksum
Returns (but does not validate) the checksum previously written bycheckFooter(org.apache.lucene.store.ChecksumIndexInput)
.- Returns:
- actual checksum value
- Throws:
IOException
- if the footer is invalid
-
checksumEntireFile
Clones the provided input, reads all bytes from the file, and callscheckFooter(org.apache.lucene.store.ChecksumIndexInput)
Note that this method may be slow, as it must process the entire file. If you just need to extract the checksum value, call
retrieveChecksum(org.apache.lucene.store.IndexInput)
.- Throws:
IOException
-
writeBEInt
write int value on header / footer with big endian order- Throws:
IOException
-
writeBELong
write long value on header / footer with big endian order- Throws:
IOException
-
readBEInt
read int value from header / footer with big endian order- Throws:
IOException
-
readBELong
read long value from header / footer with big endian order- Throws:
IOException
-