Package org.apache.lucene.replicator.nrt

Class ReplicaNode

java.lang.Object

org.apache.lucene.replicator.nrt.Node

org.apache.lucene.replicator.nrt.ReplicaNode

All Implemented Interfaces:

Closeable, AutoCloseable

public abstract class ReplicaNode
extends Node

Replica node, that pulls index changes from the primary node by copying newly flushed or merged index files.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary

Fields

Modifier and Type	Field	Description
protected CopyJob	curNRTCopy	Non-null when we are currently copying files from a new NRT point:
protected final Collection<String>	lastNRTFiles	IncRef'd files in the current NRT point:
protected long	lastPrimaryGen	Primary gen last time we successfully replicated:
protected final Set<CopyJob>	mergeCopyJobs	Currently running merge pre-copy jobs

Fields inherited from class org.apache.lucene.replicator.nrt.Node

dir, globalStartNS, id, lastFileMetaData, localStartNS, mgr, PRIMARY_GEN_KEY, printStream, searcherFactory, VERBOSE_CONNECTIONS, VERBOSE_FILES, VERSION_KEY

Constructor Summary

Constructors

Constructor	Description
ReplicaNode(int id, Directory dir, SearcherFactory searcherFactory, PrintStream printStream)

Method Summary

Modifier and Type	Method	Description
void	close()
void	commit()
IndexOutput	createTempOutput(String prefix, String suffix, IOContext ioContext)
void	finishCopyFile(String name)
protected void	finishNRTCopy(CopyJob job, long startNS)
List<Map.Entry<String,FileMetaData>>	getFilesToCopy(Map<String,FileMetaData> files)	Compares incoming per-file identity (id, checksum, header, footer) versus what we have locally and returns the subset of the incoming files that need copying
boolean	isClosed()
boolean	isCopying()
protected abstract void	launch(CopyJob job)	Runs this job async'd
protected CopyJob	launchPreCopyMerge(AtomicBoolean finished, long newPrimaryGen, Map<String,FileMetaData> files)
protected void	maybeNewPrimary(long newPrimaryGen)	Called when the primary changed
protected abstract CopyJob	newCopyJob(String reason, Map<String,FileMetaData> files, Map<String,FileMetaData> prevFiles, boolean highPriority, CopyJob.OnceDone onceDone)	Start a background copying job, to copy the specified files from the current primary node.
CopyJob	newNRTPoint(long newPrimaryGen, long version)	Call this to notify this replica node that a new NRT infos is available on the primary.
protected abstract void	sendNewReplica()	Tell primary we (replica) just started, so primary can tell us to warm any already warming merges.
protected void	start(long curPrimaryGen)	Start up this replica, which possibly requires heavy copying of files from the primary node, if we were down for a long time
void	startCopyFile(String name)

Methods inherited from class org.apache.lucene.replicator.nrt.Node

bytesToString, getCurrentSearchingVersion, getDirectory, getSearcherManager, message, name, nodeMessage, nodeMessage, readLocalFileMetaData, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

lastNRTFiles

protected final Collection<String> lastNRTFiles

IncRef'd files in the current NRT point:

mergeCopyJobs

protected final Set<CopyJob> mergeCopyJobs

Currently running merge pre-copy jobs

curNRTCopy

protected CopyJob curNRTCopy

Non-null when we are currently copying files from a new NRT point:

lastPrimaryGen

protected long lastPrimaryGen

Primary gen last time we successfully replicated:

Constructor Details

ReplicaNode

public ReplicaNode(int id,
 Directory dir,
 SearcherFactory searcherFactory,
 PrintStream printStream)
            throws IOException

Throws:

IOException

Method Details

start

protected void start(long curPrimaryGen)
              throws IOException

Start up this replica, which possibly requires heavy copying of files from the primary node, if we were down for a long time

Throws:

IOException

commit

public void commit()
            throws IOException

Specified by:

commit in class Node

Throws:

IOException

finishNRTCopy

protected void finishNRTCopy(CopyJob job,
 long startNS)
                      throws IOException

Throws:

IOException

newCopyJob

protected abstract CopyJob newCopyJob(String reason,
 Map<String,FileMetaData> files,
 Map<String,FileMetaData> prevFiles,
 boolean highPriority,
 CopyJob.OnceDone onceDone)
                               throws IOException

Start a background copying job, to copy the specified files from the current primary node. If files is null then the latest copy state should be copied. If prevJob is not null, then the new copy job is replacing it and should 1) cancel the previous one, and 2) optionally salvage e.g. partially copied and, shared with the new copy job, files.

Throws:

IOException

launch

protected abstract void launch(CopyJob job)

Runs this job async'd

sendNewReplica

protected abstract void sendNewReplica()
                                throws IOException

Tell primary we (replica) just started, so primary can tell us to warm any already warming merges. This lets us keep low nrt refresh time for the first nrt sync after we started.

Throws:

IOException

newNRTPoint

public CopyJob newNRTPoint(long newPrimaryGen,
 long version)
                    throws IOException

Call this to notify this replica node that a new NRT infos is available on the primary. We kick off a job (runs in the background) to copy files across, and open a new reader once that's done.

Throws:

IOException

isCopying

public boolean isCopying()

isClosed

public boolean isClosed()

Specified by:

isClosed in class Node

close

public void close()
           throws IOException

Throws:

IOException

maybeNewPrimary

protected void maybeNewPrimary(long newPrimaryGen)
                        throws IOException

Called when the primary changed

Throws:

IOException

launchPreCopyMerge

protected CopyJob launchPreCopyMerge(AtomicBoolean finished,
 long newPrimaryGen,
 Map<String,FileMetaData> files)
                              throws IOException

Throws:

IOException

createTempOutput

public IndexOutput createTempOutput(String prefix,
 String suffix,
 IOContext ioContext)
                             throws IOException

Throws:

IOException

getFilesToCopy

public List<Map.Entry<String,FileMetaData>> getFilesToCopy(Map<String,FileMetaData> files)
                                                    throws IOException

Compares incoming per-file identity (id, checksum, header, footer) versus what we have locally and returns the subset of the incoming files that need copying

Throws:

IOException

startCopyFile

public void startCopyFile(String name)

finishCopyFile

public void finishCopyFile(String name)