Package org.apache.lucene.index

Class SegmentInfos

  • All Implemented Interfaces:
    Cloneable, Iterable<SegmentCommitInfo>
    public final class SegmentInfos
extends Object
implements Cloneable, Iterable<SegmentCommitInfo>
    A collection of segmentInfo objects with methods for operating on those segments in relation to the file system.

    The active segments in the index are stored in the segment info file, segments_N. There may be one or more segments_N files in the index; however, the one with the largest generation is the active one (when older segments_N files are present it's because they temporarily cannot be deleted, or a custom IndexDeletionPolicy is in use). This file lists each segment by name and has details about the codec and generation of deletes.

    Files:

    • segments_N: Header, LuceneVersion, Version, NameCounter, SegCount, MinSegmentLuceneVersion, <SegName, SegID, SegCodec, DelGen, DeletionCount, FieldInfosGen, DocValuesGen, UpdatesFiles>SegCount, CommitUserData, Footer
    Data types:
    • Header --> IndexHeader
    • LuceneVersion --> Which Lucene code Version was used for this commit, written as three vInt: major, minor, bugfix
    • MinSegmentLuceneVersion --> Lucene code Version of the oldest segment, written as three vInt: major, minor, bugfix; this is only written only if there's at least one segment
    • NameCounter, SegCount, DeletionCount --> Int32
    • Generation, Version, DelGen, Checksum, FieldInfosGen, DocValuesGen --> Int64
    • SegID --> Int8ID_LENGTH
    • SegName, SegCodec --> String
    • CommitUserData --> Map<String,String>
    • UpdatesFiles --> Map<Int32, Set<String>>
    • Footer --> CodecFooter
    Field Descriptions:
    • Version counts how often the index has been changed by adding or deleting documents.
    • NameCounter is used to generate names for new segment files.
    • SegName is the name of the segment, and is used as the file name prefix for all of the files that compose the segment's index.
    • DelGen is the generation count of the deletes file. If this is -1, there are no deletes. Anything above zero means there are deletes stored by LiveDocsFormat.
    • DeletionCount records the number of deleted documents in this segment.
    • SegCodec is the name of the Codec that encoded this segment.
    • SegID is the identifier of the Codec that encoded this segment.
    • CommitUserData stores an optional user-supplied opaque Map<String,String> that was passed to IndexWriter.setLiveCommitData(Iterable).
    • FieldInfosGen is the generation count of the fieldInfos file. If this is -1, there are no updates to the fieldInfos in that segment. Anything above zero means there are updates to fieldInfos stored by FieldInfosFormat .
    • DocValuesGen is the generation count of the updatable DocValues. If this is -1, there are no updates to DocValues in that segment. Anything above zero means there are updates to DocValues stored by DocValuesFormat.
    • UpdatesFiles stores the set of files that were updated in that segment per field.
    WARNING: This API is experimental and might change in incompatible ways in the next release.

    • Field Detail

      • VERSION_70

        public static final int VERSION_70
        The version that added information about the Lucene version at the time when the index has been created.
        See Also:
        Constant Field Values

      • VERSION_72

        public static final int VERSION_72
        The version that updated segment name counter to be long instead of int.
        See Also:
        Constant Field Values

      • VERSION_74

        public static final int VERSION_74
        The version that recorded softDelCount
        See Also:
        Constant Field Values

      • VERSION_86

        public static final int VERSION_86
        The version that recorded SegmentCommitInfo IDs
        See Also:
        Constant Field Values

      • counter

        public long counter
        Used to name new segments.

      • version

        public long version
        Counts how often the index has been changed.

      • userData

        public Map<String,​String> userData
        Opaque Map<String, String> that user can specify during IndexWriter.commit

    • Constructor Detail

      • SegmentInfos

        public SegmentInfos​(int indexCreatedVersionMajor)
        Sole constructor.
        Parameters:
        indexCreatedVersionMajor - the Lucene version major at index creation time, or 6 if the index was created before 7.0

    • Method Detail

      • getLastCommitGeneration

        public static long getLastCommitGeneration​(String[] files)
        Get the generation of the most recent commit to the list of index files (N in the segments_N file).
        Parameters:
        files - -- array of file names to check

      • getLastCommitGeneration

        public static long getLastCommitGeneration​(Directory directory)
                                    throws IOException
        Get the generation of the most recent commit to the index in this directory (N in the segments_N file).
        Parameters:
        directory - -- directory to search for the latest segments_N file
        Throws:
        IOException

      • getLastCommitSegmentsFileName

        public static String getLastCommitSegmentsFileName​(String[] files)
        Get the filename of the segments_N file for the most recent commit in the list of index files.
        Parameters:
        files - -- array of file names to check

      • getLastCommitSegmentsFileName

        public static String getLastCommitSegmentsFileName​(Directory directory)
                                            throws IOException
        Get the filename of the segments_N file for the most recent commit to the index in this Directory.
        Parameters:
        directory - -- directory to search for the latest segments_N file
        Throws:
        IOException

      • getSegmentsFileName

        public String getSegmentsFileName()
        Get the segments_N filename in use by this segment infos.

      • generationFromSegmentsFileName

        public static long generationFromSegmentsFileName​(String fileName)
        Parse the generation off the segments file name and return it.

      • getId

        public byte[] getId()
        Since Lucene 5.0, every commit (segments_N) writes a unique id. This will return that id

      • readCommit

        public static final SegmentInfos readCommit​(Directory directory,
                                            String segmentFileName)
                                     throws IOException
        Read a particular segmentFileName. Note that this may throw an IOException if a commit is in process.
        Parameters:
        directory - -- directory containing the segments file
        segmentFileName - -- segment file to load
        Throws:
        CorruptIndexException - if the index is corrupt
        IOException - if there is a low-level IO error

      • clone

        public SegmentInfos clone()
        Returns a copy of this instance, also copying each SegmentInfo.
        Overrides:
        clone in class Object

      • getVersion

        public long getVersion()
        version number when this SegmentInfos was generated.

      • getGeneration

        public long getGeneration()
        Returns current generation.

      • getLastGeneration

        public long getLastGeneration()
        Returns last succesfully read or written generation.

      • setInfoStream

        public static void setInfoStream​(PrintStream infoStream)
        If non-null, information about retries when loading the segments file will be printed to this.

      • updateGeneration

        public void updateGeneration​(SegmentInfos other)
        Carry over generation numbers from another SegmentInfos
        NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.

      • setNextWriteGeneration

        public void setNextWriteGeneration​(long generation)
        Set the generation to be used for the next commit

      • files

        public Collection<String> files​(boolean includeSegmentsFile)
                         throws IOException
        Returns all file names referenced by SegmentInfo. The returned collection is recomputed on each invocation.
        Throws:
        IOException

      • commit

        public final void commit​(Directory dir)
                  throws IOException
        Writes and syncs to the Directory dir, taking care to remove the segments file on exception

        Note: changed() should be called prior to this method if changes have been made to this SegmentInfos instance

        Throws:
        IOException

      • toString

        public String toString()
        Returns readable description of this segment.
        Overrides:
        toString in class Object

      • setUserData

        public void setUserData​(Map<String,​String> data,
                        boolean doIncrementVersion)
        Sets the commit data.

      • totalMaxDoc

        public int totalMaxDoc()
        Returns sum of all segment's maxDocs. Note that this does not include deletions

      • changed

        public void changed()
        Call this before committing if changes have been made to the segments.

      • getCommitLuceneVersion

        public Version getCommitLuceneVersion()
        Returns which Lucene Version wrote this commit, or null if the version this index was written with did not directly record the version.

      • getMinSegmentLuceneVersion

        public Version getMinSegmentLuceneVersion()
        Returns the version of the oldest segment, or null if there are no segments.

      • getIndexCreatedVersionMajor

        public int getIndexCreatedVersionMajor()
        Return the version major that was used to initially create the index. This version is set when the index is first created and then never changes. This information was added as of version 7.0 so older indices report 6 as a creation version.