NRTCachingDirectory (Lucene 3.2.0 API)

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

org.apache.lucene.store
Class NRTCachingDirectory

java.lang.Object
  org.apache.lucene.store.Directory
      org.apache.lucene.store.NRTCachingDirectory

All Implemented Interfaces:: Closeable

public class NRTCachingDirectory
extends org.apache.lucene.store.Directory
extends org.apache.lucene.store.Directory

Wraps a RAMDirectory around any provided delegate directory, to be used during NRT search. Make sure you pull the merge scheduler using getMergeScheduler() and pass that to your IndexWriter; this class uses that to keep track of which merges are being done by which threads, to decide when to cache each written file.

This class is likely only useful in a near-real-time context, where indexing rate is lowish but reopen rate is highish, resulting in many tiny files being written. This directory keeps such segments (as well as the segments produced by merging them, as long as they are small enough), in RAM.

This is safe to use: when your app calls {IndexWriter#commit}, all cached files will be flushed from the cached and sync'd.

NOTE: this class is somewhat sneaky in its approach for spying on merges to determine the size of a merge: it records which threads are running which merges by watching ConcurrentMergeScheduler's doMerge method. While this works correctly, likely future versions of this class will take a more general approach.

Here's a simple example usage:

   Directory fsDir = FSDirectory.open(new File("/path/to/index"));
   NRTCachingDirectory cachedFSDir = new NRTCachingDirectory(fsDir, 5.0, 60.0);
   IndexWriterConfig conf = new IndexWriterConfig(Version.LUCENE_32, analyzer);
   conf.setMergeScheduler(cachedFSDir.getMergeScheduler());
   IndexWriter writer = new IndexWriter(cachedFSDir, conf);

This will cache all newly flushed segments, all merges whose expected segment size is <= 5 MB, unless the net cached bytes exceeds 60 MB at which point all writes will not be cached (until the net bytes falls below 60 MB).

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary

Fields inherited from class org.apache.lucene.store.Directory
`isOpen, lockFactory`

Constructor Summary
`NRTCachingDirectory(org.apache.lucene.store.Directory delegate, double maxMergeSizeMB, double maxCachedMB)` We will cache a newly created output if 1) it's a flush or a merge and the estimated size of the merged segmnt is <= maxMergeSizeMB, and 2) the total cached bytes is <= maxCachedMB

Method Summary
`void`	`clearLock(String name)`
`void`	`close()` Close thius directory, which flushes any cached files to the delegate and then closes the delegate.
`org.apache.lucene.store.IndexOutput`	`createOutput(String name)`
`void`	`deleteFile(String name)`
`protected boolean`	`doCacheWrite(String name)` Subclass can override this to customize logic; return true if this file should be written to the RAMDirectory.
`boolean`	`fileExists(String name)`
`long`	`fileLength(String name)`
`long`	`fileModified(String name)`
`org.apache.lucene.index.MergeScheduler`	`getMergeScheduler()`
`String[]`	`listAll()`
`String[]`	`listCachedFiles()`
`org.apache.lucene.store.Lock`	`makeLock(String name)`
`org.apache.lucene.store.IndexInput`	`openInput(String name)`
`org.apache.lucene.store.IndexInput`	`openInput(String name, int bufferSize)`
`long`	`sizeInBytes()` Returns how many bytes are being used by the RAMDirectory cache
`void`	`sync(Collection<String> fileNames)`
`void`	`touchFile(String name)` Deprecated.

Methods inherited from class org.apache.lucene.store.Directory
`copy, copy, ensureOpen, getLockFactory, getLockID, setLockFactory, sync, toString`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Constructor Detail