public abstract class LogMergePolicy extends MergePolicy
This class implements a MergePolicy
that tries
to merge segments into levels of exponentially
increasing size, where each level has fewer segments than
the value of the merge factor. Whenever extra segments
(beyond the merge factor upper bound) are encountered,
all segments within the level are merged. You can get or
set the merge factor using getMergeFactor()
and
setMergeFactor(int)
respectively.
This class is abstract and requires a subclass to
define the size(org.apache.lucene.index.SegmentInfo)
method which specifies how a
segment's size is determined. LogDocMergePolicy
is one subclass that measures size by document count in
the segment. LogByteSizeMergePolicy
is another
subclass that measures size as the total byte size of the
file(s) for the segment.
MergePolicy.MergeAbortedException, MergePolicy.MergeException, MergePolicy.MergeSpecification, MergePolicy.OneMerge
Modifier and Type | Field and Description |
---|---|
protected boolean |
calibrateSizeByDeletes |
static int |
DEFAULT_MAX_MERGE_DOCS
Default maximum segment size.
|
static int |
DEFAULT_MERGE_FACTOR
Default merge factor, which is how many segments are
merged at a time
|
static double |
DEFAULT_NO_CFS_RATIO
Default noCFSRatio.
|
static double |
LEVEL_LOG_SPAN
Defines the allowed range of log(size) for each
level.
|
protected int |
maxMergeDocs |
protected long |
maxMergeSize |
protected long |
maxMergeSizeForForcedMerge |
protected int |
mergeFactor |
protected long |
minMergeSize |
protected double |
noCFSRatio |
protected boolean |
useCompoundFile |
writer
Constructor and Description |
---|
LogMergePolicy() |
Modifier and Type | Method and Description |
---|---|
void |
close()
Release all resources for the policy.
|
MergePolicy.MergeSpecification |
findForcedDeletesMerges(SegmentInfos segmentInfos)
Finds merges necessary to force-merge all deletes from the
index.
|
MergePolicy.MergeSpecification |
findForcedMerges(SegmentInfos infos,
int maxNumSegments,
Map<SegmentInfo,Boolean> segmentsToMerge)
Returns the merges necessary to merge the index down
to a specified number of segments.
|
MergePolicy.MergeSpecification |
findMerges(SegmentInfos infos)
Checks if any merges are now necessary and returns a
MergePolicy.MergeSpecification if so. |
boolean |
getCalibrateSizeByDeletes()
Returns true if the segment size should be calibrated
by the number of deletes when choosing segments for merge.
|
int |
getMaxMergeDocs()
Returns the largest segment (measured by document
count) that may be merged with other segments.
|
int |
getMergeFactor()
Returns the number of segments that are merged at
once and also controls the total number of segments
allowed to accumulate in the index.
|
double |
getNoCFSRatio() |
boolean |
getUseCompoundFile()
Returns true if newly flushed and newly merge segments
are written in compound file format.
|
protected boolean |
isMerged(SegmentInfo info)
Returns true if this single info is already fully merged (has no
pending norms or deletes, is in the same dir as the
writer, and matches the current compound file setting
|
protected boolean |
isMerged(SegmentInfos infos,
int maxNumSegments,
Map<SegmentInfo,Boolean> segmentsToMerge) |
protected void |
message(String message) |
void |
setCalibrateSizeByDeletes(boolean calibrateSizeByDeletes)
Sets whether the segment size should be calibrated by
the number of deletes when choosing segments for merge.
|
void |
setMaxMergeDocs(int maxMergeDocs)
Determines the largest segment (measured by
document count) that may be merged with other segments.
|
void |
setMergeFactor(int mergeFactor)
Determines how often segment indices are merged by
addDocument().
|
void |
setNoCFSRatio(double noCFSRatio)
If a merged segment will be more than this percentage
of the total size of the index, leave the segment as
non-compound file even if compound file is enabled.
|
void |
setUseCompoundFile(boolean useCompoundFile)
Sets whether compound file format should be used for
newly flushed and newly merged segments.
|
protected abstract long |
size(SegmentInfo info) |
protected long |
sizeBytes(SegmentInfo info) |
protected long |
sizeDocs(SegmentInfo info) |
String |
toString() |
boolean |
useCompoundFile(SegmentInfos infos,
SegmentInfo mergedInfo)
Returns true if a new segment (regardless of its origin) should use the compound file format.
|
protected boolean |
verbose() |
setIndexWriter
public static final double LEVEL_LOG_SPAN
public static final int DEFAULT_MERGE_FACTOR
public static final int DEFAULT_MAX_MERGE_DOCS
public static final double DEFAULT_NO_CFS_RATIO
setNoCFSRatio(double)
,
Constant Field Valuesprotected int mergeFactor
protected long minMergeSize
protected long maxMergeSize
protected long maxMergeSizeForForcedMerge
protected int maxMergeDocs
protected double noCFSRatio
protected boolean calibrateSizeByDeletes
protected boolean useCompoundFile
protected boolean verbose()
public double getNoCFSRatio()
setNoCFSRatio(double)
public void setNoCFSRatio(double noCFSRatio)
protected void message(String message)
public int getMergeFactor()
Returns the number of segments that are merged at once and also controls the total number of segments allowed to accumulate in the index.
public void setMergeFactor(int mergeFactor)
public boolean useCompoundFile(SegmentInfos infos, SegmentInfo mergedInfo) throws IOException
MergePolicy
useCompoundFile
in class MergePolicy
IOException
public void setUseCompoundFile(boolean useCompoundFile)
public boolean getUseCompoundFile()
public void setCalibrateSizeByDeletes(boolean calibrateSizeByDeletes)
public boolean getCalibrateSizeByDeletes()
public void close()
MergePolicy
close
in interface Closeable
close
in class MergePolicy
protected abstract long size(SegmentInfo info) throws IOException
IOException
protected long sizeDocs(SegmentInfo info) throws IOException
IOException
protected long sizeBytes(SegmentInfo info) throws IOException
IOException
protected boolean isMerged(SegmentInfos infos, int maxNumSegments, Map<SegmentInfo,Boolean> segmentsToMerge) throws IOException
IOException
protected boolean isMerged(SegmentInfo info) throws IOException
IOException
public MergePolicy.MergeSpecification findForcedMerges(SegmentInfos infos, int maxNumSegments, Map<SegmentInfo,Boolean> segmentsToMerge) throws IOException
maxMergeSizeForForcedMerge
setting.
By default, and assuming maxNumSegments=1
, only
one segment will be left in the index, where that segment
has no deletions pending nor separate norms, and it is in
compound file format if the current useCompoundFile
setting is true. This method returns multiple merges
(mergeFactor at a time) so the MergeScheduler
in use may make use of concurrency.findForcedMerges
in class MergePolicy
infos
- the total set of segments in the indexmaxNumSegments
- requested maximum number of segments in the index (currently this
is always 1)segmentsToMerge
- contains the specific SegmentInfo instances that must be merged
away. This may be a subset of all
SegmentInfos. If the value is True for a
given SegmentInfo, that means this segment was
an original segment present in the
to-be-merged index; else, it was a segment
produced by a cascaded merge.IOException
public MergePolicy.MergeSpecification findForcedDeletesMerges(SegmentInfos segmentInfos) throws CorruptIndexException, IOException
findForcedDeletesMerges
in class MergePolicy
segmentInfos
- the total set of segments in the indexCorruptIndexException
IOException
public MergePolicy.MergeSpecification findMerges(SegmentInfos infos) throws IOException
MergePolicy.MergeSpecification
if so. A merge
is necessary when there are more than setMergeFactor(int)
segments at a given level. When
multiple levels have too many segments, this method
will return multiple merges, allowing the MergeScheduler
to use concurrency.findMerges
in class MergePolicy
infos
- the total set of segments in the indexIOException
public void setMaxMergeDocs(int maxMergeDocs)
Determines the largest segment (measured by document count) that may be merged with other segments. Small values (e.g., less than 10,000) are best for interactive indexing, as this limits the length of pauses while indexing to a few seconds. Larger values are best for batched indexing and speedier searches.
The default value is Integer.MAX_VALUE
.
The default merge policy (LogByteSizeMergePolicy
) also allows you to set this
limit by net size (in MB) of the segment, using LogByteSizeMergePolicy.setMaxMergeMB(double)
.
public int getMaxMergeDocs()
setMaxMergeDocs(int)