org.apache.lucene.index

Class ConcurrentMergeScheduler

java.lang.Object

org.apache.lucene.index.MergeScheduler

org.apache.lucene.index.ConcurrentMergeScheduler

All Implemented Interfaces:

Closeable, Cloneable

public class ConcurrentMergeScheduler
extends MergeScheduler

A MergeScheduler that runs each merge using a separate thread.

Specify the max number of threads that may run at once, and the maximum number of simultaneous merges with setMaxMergesAndThreads(int, int).

If the number of merges exceeds the max number of threads then the largest merges are paused until one of the smaller merges completes.

If more than getMaxMergeCount() merges are requested then this class will forcefully throttle the incoming threads by pausing until one more more merges complete.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	ConcurrentMergeScheduler.MergeThread Runs a merge thread, which may run one or more merges in sequence.

Field Summary

Fields

Modifier and Type	Field and Description
protected static Comparator<ConcurrentMergeScheduler.MergeThread>	compareByMergeDocCount Sorts ConcurrentMergeScheduler.MergeThreads; larger merges come first.
static int	DEFAULT_MAX_MERGE_COUNT Default maxMergeCount.
static int	DEFAULT_MAX_THREAD_COUNT Default maxThreadCount.
protected Directory	dir Directory that holds the index.
protected int	mergeThreadCount How many ConcurrentMergeScheduler.MergeThreads have kicked off (this is use to name them).
protected List<ConcurrentMergeScheduler.MergeThread>	mergeThreads List of currently active ConcurrentMergeScheduler.MergeThreads.
protected IndexWriter	writer IndexWriter that owns this instance.

Constructor Summary

Constructors

Constructor and Description
ConcurrentMergeScheduler() Sole constructor, with all settings set to default values.

Method Summary

Methods

Modifier and Type	Method and Description
MergeScheduler	clone()
void	close() Close this MergeScheduler.
protected void	doMerge(MergePolicy.OneMerge merge) Does the actual merge, by calling IndexWriter.merge(org.apache.lucene.index.MergePolicy.OneMerge)
int	getMaxMergeCount() See setMaxMergesAndThreads(int, int).
int	getMaxThreadCount() Returns maxThreadCount.
protected ConcurrentMergeScheduler.MergeThread	getMergeThread(IndexWriter writer, MergePolicy.OneMerge merge) Create and return a new MergeThread
int	getMergeThreadPriority() Return the priority that merge threads run at.
protected void	handleMergeException(Throwable exc) Called when an exception is hit in a background merge thread
void	merge(IndexWriter writer) Run the merges provided by IndexWriter.getNextMerge().
protected int	mergeThreadCount() Returns the number of merge threads that are alive.
protected void	message(String message) Outputs the given message - this method assumes verbose() was called and returned true.
void	setMaxMergesAndThreads(int maxMergeCount, int maxThreadCount) Sets the maximum number of merge threads and simultaneous merges allowed.
void	setMergeThreadPriority(int pri) Set the base priority that merge threads run at.
void	sync() Wait for any running merge threads to finish.
String	toString()
protected void	updateMergeThreads() Called whenever the running merges have changed, to pause & unpause threads.
protected boolean	verbose() Returns true if verbosing is enabled.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

mergeThreads

protected List<ConcurrentMergeScheduler.MergeThread> mergeThreads

List of currently active ConcurrentMergeScheduler.MergeThreads.

DEFAULT_MAX_THREAD_COUNT

public static final int DEFAULT_MAX_THREAD_COUNT

Default maxThreadCount. We default to 1: tests on spinning-magnet drives showed slower indexing performance if more than one merge thread runs at once (though on an SSD it was faster)

See Also:

Constant Field Values

DEFAULT_MAX_MERGE_COUNT

public static final int DEFAULT_MAX_MERGE_COUNT

Default maxMergeCount.

See Also:

Constant Field Values

dir

protected Directory dir

Directory that holds the index.

writer

protected IndexWriter writer

IndexWriter that owns this instance.

mergeThreadCount

protected int mergeThreadCount

How many ConcurrentMergeScheduler.MergeThreads have kicked off (this is use to name them).

compareByMergeDocCount

protected static final Comparator<ConcurrentMergeScheduler.MergeThread> compareByMergeDocCount

Sorts ConcurrentMergeScheduler.MergeThreads; larger merges come first.

Constructor Detail

ConcurrentMergeScheduler

public ConcurrentMergeScheduler()

Sole constructor, with all settings set to default values.

Method Detail

setMaxMergesAndThreads

public void setMaxMergesAndThreads(int maxMergeCount,
                          int maxThreadCount)

Sets the maximum number of merge threads and simultaneous merges allowed.

Parameters:

maxMergeCount - the max # simultaneous merges that are allowed. If a merge is necessary yet we already have this many threads running, the incoming thread (that is calling add/updateDocument) will block until a merge thread has completed. Note that we will only run the smallest maxThreadCount merges at a time.

maxThreadCount - the max # simultaneous merge threads that should be running at once. This must be <= maxMergeCount

getMaxThreadCount

public int getMaxThreadCount()

Returns maxThreadCount.

See Also:

setMaxMergesAndThreads(int, int)

getMaxMergeCount

public int getMaxMergeCount()

See setMaxMergesAndThreads(int, int).

getMergeThreadPriority

public int getMergeThreadPriority()

Return the priority that merge threads run at. By default the priority is 1 plus the priority of (ie, slightly higher priority than) the first thread that calls merge.

setMergeThreadPriority

public void setMergeThreadPriority(int pri)

Set the base priority that merge threads run at. Note that CMS may increase priority of some merge threads beyond this base priority. It's best not to set this any higher than Thread.MAX_PRIORITY-maxThreadCount, so that CMS has room to set relative priority among threads.

updateMergeThreads

protected void updateMergeThreads()

Called whenever the running merges have changed, to pause & unpause threads. This method sorts the merge threads by their merge size in descending order and then pauses/unpauses threads from first to last -- that way, smaller merges are guaranteed to run before larger ones.

verbose

protected boolean verbose()

Returns true if verbosing is enabled. This method is usually used in conjunction with message(String), like that:

 if (verbose()) {
   message("your message");
 }
 

message

protected void message(String message)

Outputs the given message - this method assumes verbose() was called and returned true.

close

public void close()

Description copied from class: MergeScheduler

Close this MergeScheduler.

Specified by:

close in interface Closeable

Specified by:

close in class MergeScheduler

sync

public void sync()

Wait for any running merge threads to finish. This call is not interruptible as used by close().

mergeThreadCount

protected int mergeThreadCount()

Returns the number of merge threads that are alive. Note that this number is ≤ mergeThreads size.

merge

public void merge(IndexWriter writer)
           throws IOException

Description copied from class: MergeScheduler

Run the merges provided by IndexWriter.getNextMerge().

Specified by:

merge in class MergeScheduler

Throws:

IOException

doMerge

protected void doMerge(MergePolicy.OneMerge merge)
                throws IOException

Does the actual merge, by calling IndexWriter.merge(org.apache.lucene.index.MergePolicy.OneMerge)

Throws:

IOException

getMergeThread

protected ConcurrentMergeScheduler.MergeThread getMergeThread(IndexWriter writer,
                                                  MergePolicy.OneMerge merge)
                                                       throws IOException

Create and return a new MergeThread

Throws:

IOException

handleMergeException

protected void handleMergeException(Throwable exc)

Called when an exception is hit in a background merge thread

toString

public String toString()

Overrides:

toString in class Object

clone

public MergeScheduler clone()

Overrides:

clone in class MergeScheduler