org.apache.lucene.codecs.lucene40
Class Lucene40PostingsFormat

java.lang.Object
  extended by org.apache.lucene.codecs.PostingsFormat
      extended by org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat
All Implemented Interfaces:
NamedSPILoader.NamedSPI

Deprecated. Only for reading old 4.0 segments

@Deprecated
public class Lucene40PostingsFormat
extends PostingsFormat

Lucene 4.0 Postings format.

Files:

Term Dictionary

The .tim file contains the list of terms in each field along with per-term statistics (such as docfreq) and pointers to the frequencies, positions and skip data in the .frq and .prx files. See BlockTreeTermsWriter for more details on the format.

NOTE: The term dictionary can plug into different postings implementations: the postings writer/reader are actually responsible for encoding and decoding the Postings Metadata and Term Metadata sections described here:

Notes:

Term Index

The .tip file contains an index into the term dictionary, so that it can be accessed randomly. See BlockTreeTermsWriter for more details on the format.

Frequencies

The .frq file contains the lists of documents which contain each term, along with the frequency of the term in that document (except when frequencies are omitted: FieldInfo.IndexOptions.DOCS_ONLY).

TermFreqs are ordered by term (the term is implicit, from the term dictionary).

TermFreq entries are ordered by increasing document number.

DocDelta: if frequencies are indexed, this determines both the document number and the frequency. In particular, DocDelta/2 is the difference between this document number and the previous document number (or zero when this is the first document in a TermFreqs). When DocDelta is odd, the frequency is one. When DocDelta is even, the frequency is read as another VInt. If frequencies are omitted, DocDelta contains the gap (not multiplied by 2) between document numbers and no frequency information is stored.

For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven, with frequencies indexed, would be the following sequence of VInts:

15, 8, 3

If frequencies were omitted (FieldInfo.IndexOptions.DOCS_ONLY) it would be this sequence of VInts instead:

7,4

DocSkip records the document number before every SkipInterval th document in TermFreqs. If payloads and offsets are disabled for the term's field, then DocSkip represents the difference from the previous value in the sequence. If payloads and/or offsets are enabled for the term's field, then DocSkip/2 represents the difference from the previous value in the sequence. In this case when DocSkip is odd, then PayloadLength and/or OffsetLength are stored indicating the length of the last payload/offset before the SkipIntervalth document in TermPositions.

PayloadLength indicates the length of the last payload.

OffsetLength indicates the length of the last offset (endOffset-startOffset).

FreqSkip and ProxSkip record the position of every SkipInterval th entry in FreqFile and ProxFile, respectively. File positions are relative to the start of TermFreqs and Positions, to the previous SkipDatum in the sequence.

For example, if DocFreq=35 and SkipInterval=16, then there are two SkipData entries, containing the 15 th and 31 st document numbers in TermFreqs. The first FreqSkip names the number of bytes after the beginning of TermFreqs that the 16 th SkipDatum starts, and the second the number of bytes after that that the 32 nd starts. The first ProxSkip names the number of bytes after the beginning of Positions that the 16 th SkipDatum starts, and the second the number of bytes after that that the 32 nd starts.

Each term can have multiple skip levels. The amount of skip levels for a term is NumSkipLevels = Min(MaxSkipLevels, floor(log(DocFreq/log(SkipInterval)))). The number of SkipData entries for a skip level is DocFreq/(SkipInterval^(Level + 1)), whereas the lowest skip level is Level=0.
Example: SkipInterval = 4, MaxSkipLevels = 2, DocFreq = 35. Then skip level 0 has 8 SkipData entries, containing the 3rd, 7th, 11th, 15th, 19th, 23rd, 27th, and 31st document numbers in TermFreqs. Skip level 1 has 2 SkipData entries, containing the 15th and 31st document numbers in TermFreqs.
The SkipData entries on all upper levels > 0 contain a SkipChildLevelPointer referencing the corresponding SkipData entry in level-1. In the example has entry 15 on level 1 a pointer to entry 15 on level 0 and entry 31 on level 1 a pointer to entry 31 on level 0.

Positions

The .prx file contains the lists of positions that each term occurs at within documents. Note that fields omitting positional data do not store anything into this file, and if all fields in the index omit positional data then the .prx file will not exist.

TermPositions are ordered by term (the term is implicit, from the term dictionary).

Positions entries are ordered by increasing document number (the document number is implicit from the .frq file).

PositionDelta is, if payloads are disabled for the term's field, the difference between the position of the current occurrence in the document and the previous occurrence (or zero, if this is the first occurrence in this document). If payloads are enabled for the term's field, then PositionDelta/2 is the difference between the current and the previous position. If payloads are enabled and PositionDelta is odd, then PayloadLength is stored, indicating the length of the payload at the current term position.

For example, the TermPositions for a term which occurs as the fourth term in one document, and as the fifth and ninth term in a subsequent document, would be the following sequence of VInts (payloads disabled):

4, 5, 4

PayloadData is metadata associated with the current term position. If PayloadLength is stored at the current position, then it indicates the length of this payload. If PayloadLength is not stored, then this payload has the same length as the payload at the previous position.

OffsetDelta/2 is the difference between this position's startOffset from the previous occurrence (or zero, if this is the first occurrence in this document). If OffsetDelta is odd, then the length (endOffset-startOffset) differs from the previous occurrence and an OffsetLength follows. Offset data is only written for FieldInfo.IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS.


Field Summary
protected  int maxBlockSize
          Deprecated. maximum items (terms or sub-blocks) per block for BlockTree
protected  int minBlockSize
          Deprecated. minimum items (terms or sub-blocks) per block for BlockTree
 
Fields inherited from class org.apache.lucene.codecs.PostingsFormat
EMPTY
 
Constructor Summary
Lucene40PostingsFormat()
          Deprecated. Creates Lucene40PostingsFormat with default settings.
 
Method Summary
 FieldsConsumer fieldsConsumer(SegmentWriteState state)
          Deprecated. Writes a new segment
 FieldsProducer fieldsProducer(SegmentReadState state)
          Deprecated. Reads a segment.
 String toString()
          Deprecated.  
 
Methods inherited from class org.apache.lucene.codecs.PostingsFormat
availablePostingsFormats, forName, getName, reloadPostingsFormats
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

minBlockSize

protected final int minBlockSize
Deprecated. 
minimum items (terms or sub-blocks) per block for BlockTree


maxBlockSize

protected final int maxBlockSize
Deprecated. 
maximum items (terms or sub-blocks) per block for BlockTree

Constructor Detail

Lucene40PostingsFormat

public Lucene40PostingsFormat()
Deprecated. 
Creates Lucene40PostingsFormat with default settings.

Method Detail

fieldsConsumer

public FieldsConsumer fieldsConsumer(SegmentWriteState state)
                              throws IOException
Deprecated. 
Description copied from class: PostingsFormat
Writes a new segment

Specified by:
fieldsConsumer in class PostingsFormat
Throws:
IOException

fieldsProducer

public FieldsProducer fieldsProducer(SegmentReadState state)
                              throws IOException
Deprecated. 
Description copied from class: PostingsFormat
Reads a segment. NOTE: by the time this call returns, it must hold open any files it will need to use; else, those files may be deleted. Additionally, required files may be deleted during the execution of this call before there is a chance to open them. Under these circumstances an IOException should be thrown by the implementation. IOExceptions are expected and will automatically cause a retry of the segment opening logic with the newly revised segments.

Specified by:
fieldsProducer in class PostingsFormat
Throws:
IOException

toString

public String toString()
Deprecated. 
Overrides:
toString in class PostingsFormat


Copyright © 2000-2014 Apache Software Foundation. All Rights Reserved.