org.apache.lucene.replicator

Class IndexReplicationHandler

java.lang.Object

org.apache.lucene.replicator.IndexReplicationHandler

All Implemented Interfaces:

ReplicationClient.ReplicationHandler

public class IndexReplicationHandler
extends Object
implements ReplicationClient.ReplicationHandler

A ReplicationClient.ReplicationHandler for replication of an index. Implements revisionReady(java.lang.String, java.util.Map<java.lang.String, java.util.List<org.apache.lucene.replicator.RevisionFile>>, java.util.Map<java.lang.String, java.util.List<java.lang.String>>, java.util.Map<java.lang.String, org.apache.lucene.store.Directory>) by copying the files pointed by the client resolver to the index Directory and then touches the index with IndexWriter to make sure any unused files are deleted.

NOTE: this handler assumes that IndexWriter is not opened by another process on the index directory. In fact, opening an IndexWriter on the same directory to which files are copied can lead to undefined behavior, where some or all the files will be deleted, override other files or simply create a mess. When you replicate an index, it is best if the index is never modified by IndexWriter, except the one that is open on the source index, from which you replicate.

This handler notifies the application via a provided Callable when an updated index commit was made available for it.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary

Fields

Modifier and Type	Field and Description
static String	INFO_STREAM_COMPONENT The component used to log messages to the default InfoStream.

Constructor Summary

Constructors

Constructor and Description
IndexReplicationHandler(Directory indexDir, Callable<Boolean> callback) Constructor with the given index directory and callback to notify when the indexes were updated.

Method Summary

Methods

Modifier and Type	Method and Description
static void	cleanupFilesOnFailure(Directory dir, List<String> files) Cleanup the index directory by deleting all given files.
static void	cleanupOldIndexFiles(Directory dir, String segmentsFile, InfoStream infoStream) Cleans up the index directory from old index files.
static void	copyFiles(Directory source, Directory target, List<String> files) Copies the files from the source directory to the target one, if they are not the same.
Map<String,List<RevisionFile>>	currentRevisionFiles() Returns the current revision files held by the handler.
String	currentVersion() Returns the current revision version held by the handler.
static IndexCommit	getLastCommit(Directory dir) Returns the last IndexCommit found in the Directory, or null if there are no commits.
static String	getSegmentsFile(List<String> files, boolean allowEmpty) Verifies that the last file is segments_N and fails otherwise.
void	revisionReady(String version, Map<String,List<RevisionFile>> revisionFiles, Map<String,List<String>> copiedFiles, Map<String,Directory> sourceDirectory) Called when a new revision was obtained and is available (i.e.
void	setInfoStream(InfoStream infoStream) Sets the InfoStream to use for logging messages.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INFO_STREAM_COMPONENT

public static final String INFO_STREAM_COMPONENT

The component used to log messages to the default InfoStream.

See Also:

Constant Field Values

Constructor Detail

IndexReplicationHandler

public IndexReplicationHandler(Directory indexDir,
                       Callable<Boolean> callback)
                        throws IOException

Constructor with the given index directory and callback to notify when the indexes were updated.

Throws:

IOException

Method Detail

getLastCommit

public static IndexCommit getLastCommit(Directory dir)
                                 throws IOException

Returns the last IndexCommit found in the Directory, or null if there are no commits.

Throws:

IOException

getSegmentsFile

public static String getSegmentsFile(List<String> files,
                     boolean allowEmpty)

Verifies that the last file is segments_N and fails otherwise. It also removes and returns the file from the list, because it needs to be handled last, after all files. This is important in order to guarantee that if a reader sees the new segments_N, all other segment files are already on stable storage.

The reason why the code fails instead of putting segments_N file last is that this indicates an error in the Revision implementation.

cleanupFilesOnFailure

public static void cleanupFilesOnFailure(Directory dir,
                         List<String> files)

Cleanup the index directory by deleting all given files. Called when file copy or sync failed.

cleanupOldIndexFiles

public static void cleanupOldIndexFiles(Directory dir,
                        String segmentsFile,
                        InfoStream infoStream)

Cleans up the index directory from old index files. This method uses the last commit found by getLastCommit(Directory). If it matches the expected segmentsFile, then all files not referenced by this commit point are deleted.

NOTE: this method does a best effort attempt to clean the index directory. It suppresses any exceptions that occur, as this can be retried the next time.

copyFiles

public static void copyFiles(Directory source,
             Directory target,
             List<String> files)
                      throws IOException

Copies the files from the source directory to the target one, if they are not the same.

Throws:

IOException

currentVersion

public String currentVersion()

Description copied from interface: ReplicationClient.ReplicationHandler

Returns the current revision version held by the handler.

Specified by:

currentVersion in interface ReplicationClient.ReplicationHandler

currentRevisionFiles

public Map<String,List<RevisionFile>> currentRevisionFiles()

Description copied from interface: ReplicationClient.ReplicationHandler

Returns the current revision files held by the handler.

Specified by:

currentRevisionFiles in interface ReplicationClient.ReplicationHandler

revisionReady

public void revisionReady(String version,
                 Map<String,List<RevisionFile>> revisionFiles,
                 Map<String,List<String>> copiedFiles,
                 Map<String,Directory> sourceDirectory)
                   throws IOException

Description copied from interface: ReplicationClient.ReplicationHandler

Called when a new revision was obtained and is available (i.e. all needed files were successfully copied).

Specified by:

revisionReady in interface ReplicationClient.ReplicationHandler

Parameters:

version - the version of the Revision that was copied

revisionFiles - the files contained by this Revision

copiedFiles - the files that were actually copied

sourceDirectory - a mapping from a source of files to the Directory they were copied into

Throws:

IOException

setInfoStream

public void setInfoStream(InfoStream infoStream)

Sets the InfoStream to use for logging messages.