Package org.apache.lucene.index

Class FilterMergePolicy

java.lang.Object

org.apache.lucene.index.MergePolicy

org.apache.lucene.index.FilterMergePolicy

All Implemented Interfaces:

Unwrappable<MergePolicy>

Direct Known Subclasses:

OneMergeWrappingMergePolicy, UpgradeIndexMergePolicy

public class FilterMergePolicy
extends MergePolicy
implements Unwrappable<MergePolicy>

A wrapper for MergePolicy instances.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.index.MergePolicy

MergePolicy.MergeAbortedException, MergePolicy.MergeContext, MergePolicy.MergeException, MergePolicy.MergeSpecification, MergePolicy.OneMerge, MergePolicy.OneMergeProgress

Field Summary

Fields

Modifier and Type	Field	Description
protected final MergePolicy	in	The wrapped MergePolicy.

Fields inherited from class org.apache.lucene.index.MergePolicy

DEFAULT_MAX_CFS_SEGMENT_SIZE, DEFAULT_NO_CFS_RATIO, maxCFSSegmentSize, noCFSRatio

Constructor Summary

Constructors

Constructor	Description
FilterMergePolicy(MergePolicy in)	Creates a new filter merge policy instance wrapping another.

Method Summary

Modifier and Type	Method	Description
MergePolicy.MergeSpecification	findForcedDeletesMerges(SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext)	Determine what set of merge operations is necessary in order to expunge all deletes from the index.
MergePolicy.MergeSpecification	findForcedMerges(SegmentInfos segmentInfos, int maxSegmentCount, Map<SegmentCommitInfo,Boolean> segmentsToMerge, MergePolicy.MergeContext mergeContext)	Determine what set of merge operations is necessary in order to merge to <= the specified segment count.
MergePolicy.MergeSpecification	findFullFlushMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext)	Identifies merges that we want to execute (synchronously) on commit.
MergePolicy.MergeSpecification	findMerges(CodecReader... readers)	Define the set of merge operations to perform on provided codec readers in IndexWriter.addIndexes(CodecReader...).
MergePolicy.MergeSpecification	findMerges(MergeTrigger mergeTrigger, SegmentInfos segmentInfos, MergePolicy.MergeContext mergeContext)	Determine what set of merge operations are now necessary on the index.
final double	getMaxCFSSegmentSizeMB()	Returns the largest size allowed for a compound file segment
double	getNoCFSRatio()	Returns current noCFSRatio.
boolean	keepFullyDeletedSegment(IOSupplier<CodecReader> readerIOSupplier)	Returns true if the segment represented by the given CodecReader should be keep even if it's fully deleted.
protected long	maxFullFlushMergeSize()	Return the maximum size of segments to be included in full-flush merges by the default implementation of MergePolicy.findFullFlushMerges(org.apache.lucene.index.MergeTrigger, org.apache.lucene.index.SegmentInfos, org.apache.lucene.index.MergePolicy.MergeContext).
int	numDeletesToMerge(SegmentCommitInfo info, int delCount, IOSupplier<CodecReader> readerSupplier)	Returns the number of deletes that a merge would claim on the given segment.
final void	setMaxCFSSegmentSizeMB(double v)	If a merged segment will be more than this value, leave the segment as non-compound file even if compound file is enabled.
final void	setNoCFSRatio(double noCFSRatio)	If a merged segment will be more than this percentage of the total size of the index, leave the segment as non-compound file even if compound file is enabled.
protected long	size(SegmentCommitInfo info, MergePolicy.MergeContext context)	Return the byte size of the provided SegmentCommitInfo, pro-rated by percentage of non-deleted documents is set.
String	toString()
MergePolicy	unwrap()	Unwraps this instance
boolean	useCompoundFile(SegmentInfos infos, SegmentCommitInfo mergedInfo, MergePolicy.MergeContext mergeContext)	Returns true if a new segment (regardless of its origin) should use the compound file format.

Methods inherited from class org.apache.lucene.index.MergePolicy

assertDelCount, isMerged, message, segString, verbose

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

in

protected final MergePolicy in

The wrapped MergePolicy.

Constructor Details

FilterMergePolicy

public FilterMergePolicy(MergePolicy in)

Creates a new filter merge policy instance wrapping another.

Parameters:

in - the wrapped MergePolicy

Method Details

findMerges

public MergePolicy.MergeSpecification findMerges(MergeTrigger mergeTrigger,
 SegmentInfos segmentInfos,
 MergePolicy.MergeContext mergeContext)
                                          throws IOException

Description copied from class: MergePolicy

Determine what set of merge operations are now necessary on the index. IndexWriter calls this whenever there is a change to the segments. This call is always synchronized on the IndexWriter instance so only one thread at a time will call this method.

Specified by:

findMerges in class MergePolicy

Parameters:

mergeTrigger - the event that triggered the merge

segmentInfos - the total set of segments in the index

mergeContext - the IndexWriter to find the merges on

Throws:

IOException

findMerges

public MergePolicy.MergeSpecification findMerges(CodecReader... readers)
                                          throws IOException

Description copied from class: MergePolicy

Define the set of merge operations to perform on provided codec readers in IndexWriter.addIndexes(CodecReader...).

The merge operation is required to convert provided readers into segments that can be added to the writer. This API can be overridden in custom merge policies to control the concurrency for addIndexes. Default implementation creates a single merge operation for all provided readers (lowest concurrency). Creating a merge for each reader, would provide the highest level of concurrency possible with the configured merge scheduler.

Overrides:

findMerges in class MergePolicy

Parameters:

readers - CodecReader(s) to merge into the main index

Throws:

IOException

findForcedMerges

public MergePolicy.MergeSpecification findForcedMerges(SegmentInfos segmentInfos,
 int maxSegmentCount,
 Map<SegmentCommitInfo,Boolean> segmentsToMerge,
 MergePolicy.MergeContext mergeContext)
                                                throws IOException

Description copied from class: MergePolicy

Determine what set of merge operations is necessary in order to merge to <= the specified segment count. IndexWriter calls this when its IndexWriter.forceMerge(int) method is called. This call is always synchronized on the IndexWriter instance so only one thread at a time will call this method.

Specified by:

findForcedMerges in class MergePolicy

Parameters:

segmentInfos - the total set of segments in the index

maxSegmentCount - requested maximum number of segments in the index

segmentsToMerge - contains the specific SegmentInfo instances that must be merged away. This may be a subset of all SegmentInfos. If the value is True for a given SegmentInfo, that means this segment was an original segment present in the to-be-merged index; else, it was a segment produced by a cascaded merge.

mergeContext - the MergeContext to find the merges on

Throws:

IOException

findForcedDeletesMerges

public MergePolicy.MergeSpecification findForcedDeletesMerges(SegmentInfos segmentInfos,
 MergePolicy.MergeContext mergeContext)
                                                       throws IOException

Description copied from class: MergePolicy

Determine what set of merge operations is necessary in order to expunge all deletes from the index.

Specified by:

findForcedDeletesMerges in class MergePolicy

Parameters:

segmentInfos - the total set of segments in the index

mergeContext - the MergeContext to find the merges on

Throws:

IOException

findFullFlushMerges

public MergePolicy.MergeSpecification findFullFlushMerges(MergeTrigger mergeTrigger,
 SegmentInfos segmentInfos,
 MergePolicy.MergeContext mergeContext)
                                                   throws IOException

Description copied from class: MergePolicy

Identifies merges that we want to execute (synchronously) on commit. By default, this will return natural merges whose segments are all less than the max segment size for full flushes.

Any merges returned here will make IndexWriter.commit(), IndexWriter.prepareCommit() or IndexWriter.getReader(boolean, boolean) block until the merges complete or until LiveIndexWriterConfig.getMaxFullFlushMergeWaitMillis() has elapsed. This may be used to merge small segments that have just been flushed, reducing the number of segments in the point in time snapshot. If a merge does not complete in the allotted time, it will continue to execute, and eventually finish and apply to future point in time snapshot, but will not be reflected in the current one.

If a MergePolicy.OneMerge in the returned MergePolicy.MergeSpecification includes a segment already included in a registered merge, then IndexWriter.commit() or IndexWriter.prepareCommit() will throw a IllegalStateException. Use MergePolicy.MergeContext.getMergingSegments() to determine which segments are currently registered to merge.

Overrides:

findFullFlushMerges in class MergePolicy

Parameters:

mergeTrigger - the event that triggered the merge (COMMIT or GET_READER).

segmentInfos - the total set of segments in the index (while preparing the commit)

mergeContext - the MergeContext to find the merges on, which should be used to determine which segments are already in a registered merge (see MergePolicy.MergeContext.getMergingSegments()).

Throws:

IOException

useCompoundFile

public boolean useCompoundFile(SegmentInfos infos,
 SegmentCommitInfo mergedInfo,
 MergePolicy.MergeContext mergeContext)
                        throws IOException

Description copied from class: MergePolicy

Returns true if a new segment (regardless of its origin) should use the compound file format. The default implementation returns true iff the size of the given mergedInfo is less or equal to MergePolicy.getMaxCFSSegmentSizeMB() and the size is less or equal to the TotalIndexSize * MergePolicy.getNoCFSRatio() otherwise false.

Overrides:

useCompoundFile in class MergePolicy

Throws:

IOException

size

protected long size(SegmentCommitInfo info,
 MergePolicy.MergeContext context)
             throws IOException

Description copied from class: MergePolicy

Return the byte size of the provided SegmentCommitInfo, pro-rated by percentage of non-deleted documents is set.

Overrides:

size in class MergePolicy

Throws:

IOException

getNoCFSRatio

public double getNoCFSRatio()

Description copied from class: MergePolicy

Returns current noCFSRatio.

Overrides:

getNoCFSRatio in class MergePolicy

See Also:

• MergePolicy.setNoCFSRatio(double)

setNoCFSRatio

public final void setNoCFSRatio(double noCFSRatio)

Description copied from class: MergePolicy

If a merged segment will be more than this percentage of the total size of the index, leave the segment as non-compound file even if compound file is enabled. Set to 1.0 to always use CFS regardless of merge size.

Overrides:

setNoCFSRatio in class MergePolicy

setMaxCFSSegmentSizeMB

public final void setMaxCFSSegmentSizeMB(double v)

Description copied from class: MergePolicy

If a merged segment will be more than this value, leave the segment as non-compound file even if compound file is enabled. Set this to Double.POSITIVE_INFINITY (default) and noCFSRatio to 1.0 to always use CFS regardless of merge size.

Overrides:

setMaxCFSSegmentSizeMB in class MergePolicy

getMaxCFSSegmentSizeMB

public final double getMaxCFSSegmentSizeMB()

Description copied from class: MergePolicy

Returns the largest size allowed for a compound file segment

Overrides:

getMaxCFSSegmentSizeMB in class MergePolicy

toString

public String toString()

Overrides:

toString in class Object

keepFullyDeletedSegment

public boolean keepFullyDeletedSegment(IOSupplier<CodecReader> readerIOSupplier)
                                throws IOException

Description copied from class: MergePolicy

Returns true if the segment represented by the given CodecReader should be keep even if it's fully deleted. This is useful for testing of for instance if the merge policy implements retention policies for soft deletes.

Overrides:

keepFullyDeletedSegment in class MergePolicy

Throws:

IOException

numDeletesToMerge

public int numDeletesToMerge(SegmentCommitInfo info,
 int delCount,
 IOSupplier<CodecReader> readerSupplier)
                      throws IOException

Description copied from class: MergePolicy

Returns the number of deletes that a merge would claim on the given segment. This method will by default return the sum of the del count on disk and the pending delete count. Yet, subclasses that wrap merge readers might modify this to reflect deletes that are carried over to the target segment in the case of soft deletes.

Soft deletes all deletes to survive across merges in order to control when the soft-deleted data is claimed.

Overrides:

numDeletesToMerge in class MergePolicy

Parameters:

info - the segment info that identifies the segment

delCount - the number deleted documents for this segment

readerSupplier - a supplier that allows to obtain a CodecReader for this segment

Throws:

IOException

See Also:

• IndexWriter.softUpdateDocument(Term, Iterable, Field...)
• IndexWriterConfig.setSoftDeletesField(String)

unwrap

public MergePolicy unwrap()

Description copied from interface: Unwrappable

Unwraps this instance

Specified by:

unwrap in interface Unwrappable<MergePolicy>

maxFullFlushMergeSize

protected long maxFullFlushMergeSize()

Description copied from class: MergePolicy

Return the maximum size of segments to be included in full-flush merges by the default implementation of MergePolicy.findFullFlushMerges(org.apache.lucene.index.MergeTrigger, org.apache.lucene.index.SegmentInfos, org.apache.lucene.index.MergePolicy.MergeContext).

Overrides:

maxFullFlushMergeSize in class MergePolicy