public class NRTCachingDirectory extends FilterDirectory implements Accountable
RAMDirectory
around any provided delegate directory, to
be used during NRT search.
This class is likely only useful in a near-real-time context, where indexing rate is lowish but reopen rate is highish, resulting in many tiny files being written. This directory keeps such segments (as well as the segments produced by merging them, as long as they are small enough), in RAM.
This is safe to use: when your app calls {IndexWriter#commit}, all cached files will be flushed from the cached and sync'd.
Here's a simple example usage:
Directory fsDir = FSDirectory.open(new File("/path/to/index"));
NRTCachingDirectory cachedFSDir = new NRTCachingDirectory(fsDir, 5.0, 60.0);
IndexWriterConfig conf = new IndexWriterConfig(Version.LUCENE_4_10_0, analyzer);
IndexWriter writer = new IndexWriter(cachedFSDir, conf);
This will cache all newly flushed segments, all merges whose expected segment size is <= 5 MB, unless the net cached bytes exceeds 60 MB at which point all writes will not be cached (until the net bytes falls below 60 MB).
in| Constructor and Description |
|---|
NRTCachingDirectory(Directory delegate,
double maxMergeSizeMB,
double maxCachedMB)
We will cache a newly created output if 1) it's a
flush or a merge and the estimated size of the merged segment is <=
maxMergeSizeMB, and 2) the total cached bytes is <=
maxCachedMB
|
| Modifier and Type | Method and Description |
|---|---|
void |
close()
Close this directory, which flushes any cached files
to the delegate and then closes the delegate.
|
IndexOutput |
createOutput(String name,
IOContext context)
Creates a new, empty file in the directory with the given name.
|
void |
deleteFile(String name)
Removes an existing file in the directory.
|
protected boolean |
doCacheWrite(String name,
IOContext context)
Subclass can override this to customize logic; return
true if this file should be written to the RAMDirectory.
|
boolean |
fileExists(String name)
Returns true iff a file with the given name exists.
|
long |
fileLength(String name)
Returns the length of a file in the directory.
|
String[] |
listAll()
Returns an array of strings, one for each file in the directory.
|
String[] |
listCachedFiles() |
IndexInput |
openInput(String name,
IOContext context)
Returns a stream reading an existing file, with the
specified read buffer size.
|
long |
ramBytesUsed()
Return the memory usage of this object in bytes.
|
void |
sync(Collection<String> fileNames)
Ensure that any writes to these files are moved to
stable storage.
|
String |
toString() |
clearLock, getDelegate, getLockFactory, getLockID, makeLock, setLockFactorycopy, ensureOpen, openChecksumInputpublic NRTCachingDirectory(Directory delegate, double maxMergeSizeMB, double maxCachedMB)
public String toString()
toString in class FilterDirectorypublic String[] listAll() throws IOException
DirectorylistAll in class FilterDirectoryNoSuchDirectoryException - if the directory is not prepared for any
write operations (such as Directory.createOutput(String, IOContext)).IOException - in case of other IO errorspublic boolean fileExists(String name) throws IOException
DirectoryfileExists in class FilterDirectoryIOExceptionpublic void deleteFile(String name) throws IOException
DirectorydeleteFile in class FilterDirectoryIOExceptionpublic long fileLength(String name) throws IOException
DirectoryFileNotFoundException or NoSuchFileException
if the file does not exist.
fileLength in class FilterDirectoryname - the name of the file for which to return the length.IOException - if there was an IO error while retrieving the file's
length.public String[] listCachedFiles()
public IndexOutput createOutput(String name, IOContext context) throws IOException
DirectorycreateOutput in class FilterDirectoryIOExceptionpublic void sync(Collection<String> fileNames) throws IOException
Directorysync in class FilterDirectoryIOExceptionpublic IndexInput openInput(String name, IOContext context) throws IOException
DirectoryFSDirectory and CompoundFileDirectory.
Throws FileNotFoundException or NoSuchFileException
if the file does not exist.
openInput in class FilterDirectoryIOExceptionpublic void close()
throws IOException
close in interface Closeableclose in interface AutoCloseableclose in class FilterDirectoryIOExceptionprotected boolean doCacheWrite(String name, IOContext context)
public long ramBytesUsed()
AccountableramBytesUsed in interface AccountableCopyright © 2000-2014 Apache Software Foundation. All Rights Reserved.