public final class Lucene41StoredFieldsFormat extends CompressingStoredFieldsFormat
StoredFieldsFormat compresses blocks of 16KB of documents in
order to improve the compression ratio compared to document-level
compression. It uses the LZ4
compression algorithm, which is fast to compress and very fast to decompress
data. Although the compression method that is used focuses more on speed
than on compression ratio, it should provide interesting compression ratios
for redundant inputs (such as log files, HTML or plain text).
Stored fields are represented by two files:
A fields data file (extension .fdt). This file stores a compact representation of documents in compressed blocks of 16KB or more. When writing a segment, documents are appended to an in-memory byte buffer. When its size reaches 16KB or more, some metadata about the documents is flushed to disk, immediately followed by a compressed representation of the buffer using the LZ4 compression format.
Here is a more detailed description of the field data file format:
VInt(let's call it bitsRequired)
VLong, whose 3 last bits are Type and other bits are FieldNum
String| BinaryValue | Int | Float | Long | Double depending on Type
StoredFieldVisitors which are only interested in the first fields of a document to not have to decompress 10MB of data if the document is 10MB, but only 16KB.
A fields index file (extension .fdx).
StoredFieldsFormat does not support individual documents
larger than (231 - 214) bytes. In case this
is a problem, you should use another format, such as
|Constructor and Description|
fieldsReader, fieldsWriter, toString
Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.