public abstract class FSDirectory extends BaseDirectory
SimpleFSDirectory
is a straightforward
implementation using Files.newByteChannel.
However, it has poor concurrent performance
(multiple threads will bottleneck) as it
synchronizes when multiple threads read from the
same file.
NIOFSDirectory
uses java.nio's
FileChannel's positional io when reading to avoid
synchronization when reading from the same file.
Unfortunately, due to a Windows-only Sun
JRE bug this is a poor choice for Windows, but
on all other platforms this is the preferred
choice. Applications using Thread.interrupt()
or
Future.cancel(boolean)
should use
RAFDirectory
instead. See NIOFSDirectory
java doc
for details.
MMapDirectory
uses memory-mapped IO when
reading. This is a good choice if you have plenty
of virtual memory relative to your index size, eg
if you are running on a 64 bit JRE, or you are
running on a 32 bit JRE but your index sizes are
small enough to fit into the virtual memory space.
Java has currently the limitation of not being able to
unmap files from user code. The files are unmapped, when GC
releases the byte buffers. Due to
this bug in Sun's JRE, MMapDirectory's IndexInput.close()
is unable to close the underlying OS file handle. Only when
GC finally collects the underlying objects, which could be
quite some time later, will the file handle be closed.
This will consume additional transient disk usage: on Windows,
attempts to delete or overwrite the files will result in an
exception; on other platforms, which typically have a "delete on
last close" semantics, while such operations will succeed, the bytes
are still consuming space on disk. For many applications this
limitation is not a problem (e.g. if you have plenty of disk space,
and you don't rely on overwriting files on Windows) but it's still
an important limitation to be aware of. This class supplies a
(possibly dangerous) workaround mentioned in the bug report,
which may fail on non-Sun JVMs.
Unfortunately, because of system peculiarities, there is
no single overall best implementation. Therefore, we've
added the open(java.nio.file.Path)
method, to allow Lucene to choose
the best FSDirectory implementation given your
environment, and the known limitations of each
implementation. For users who have no reason to prefer a
specific implementation, it's best to simply use open(java.nio.file.Path)
. For all others, you should instantiate the
desired implementation directly.
NOTE: Accessing one of the above subclasses either directly or
indirectly from a thread while it's interrupted can close the
underlying channel immediately if at the same time the thread is
blocked on IO. The channel will remain closed and subsequent access
to the index will throw a ClosedChannelException
.
Applications using Thread.interrupt()
or
Future.cancel(boolean)
should use the slower legacy
RAFDirectory
from the misc
Lucene module instead.
The locking implementation is by default NativeFSLockFactory
, but can be changed by
passing in a custom LockFactory
instance.
Directory
Modifier and Type | Field and Description |
---|---|
protected Path |
directory |
isOpen, lockFactory
Modifier | Constructor and Description |
---|---|
protected |
FSDirectory(Path path,
LockFactory lockFactory)
Create a new FSDirectory for the named location (ctor for subclasses).
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Closes the directory.
|
IndexOutput |
createOutput(String name,
IOContext context)
Creates a new, empty file in the directory and returns an
IndexOutput
instance for appending data to this file. |
IndexOutput |
createTempOutput(String prefix,
String suffix,
IOContext context)
Creates a new, empty, temporary file in the directory and returns an
IndexOutput
instance for appending data to this file. |
void |
deleteFile(String name)
Removes an existing file in the directory.
|
void |
deletePendingFiles()
Try to delete any pending files that we had previously tried to delete but failed
because we are on Windows and the files were still held open.
|
protected void |
ensureCanRead(String name) |
long |
fileLength(String name)
Returns the byte length of a file in the directory.
|
protected void |
fsync(String name) |
Path |
getDirectory() |
Set<String> |
getPendingDeletions()
Returns a set of files currently pending deletion in this directory.
|
String[] |
listAll()
Returns names of all files stored in this directory.
|
static String[] |
listAll(Path dir)
Lists all files (including subdirectories) in the directory.
|
static FSDirectory |
open(Path path)
Creates an FSDirectory instance, trying to pick the
best implementation given the current environment.
|
static FSDirectory |
open(Path path,
LockFactory lockFactory)
Just like
open(Path) , but allows you to
also specify a custom LockFactory . |
void |
rename(String source,
String dest)
Renames
source file to dest file where
dest must not already exist in the directory. |
void |
sync(Collection<String> names)
Ensures that any writes to these files are moved to
stable storage (made durable).
|
void |
syncMetaData()
Ensures that directory metadata, such as recent file renames, are moved to stable
storage.
|
String |
toString() |
ensureOpen, obtainLock
copyFrom, openChecksumInput, openInput
protected final Path directory
protected FSDirectory(Path path, LockFactory lockFactory) throws IOException
FSDirectory
resolves the given Path to a canonical /
real path to ensure it can correctly lock the index directory and no other process
can interfere with changing possible symlinks to the index directory inbetween.
If you want to use symlinks and change them dynamically, close all
IndexWriters
and create a new FSDirecory
instance.
path
- the path of the directorylockFactory
- the lock factory to use, or null for the default
(NativeFSLockFactory
);IOException
- if there is a low-level I/O errorpublic static FSDirectory open(Path path) throws IOException
NativeFSLockFactory
.
The directory is created at the named location if it does not yet exist.
FSDirectory
resolves the given Path when calling this method to a canonical /
real path to ensure it can correctly lock the index directory and no other process
can interfere with changing possible symlinks to the index directory inbetween.
If you want to use symlinks and change them dynamically, close all
IndexWriters
and create a new FSDirecory
instance.
Currently this returns MMapDirectory
for Linux, MacOSX, Solaris,
and Windows 64-bit JREs, NIOFSDirectory
for other
non-Windows JREs, and SimpleFSDirectory
for other
JREs on Windows. It is highly recommended that you consult the
implementation's documentation for your platform before
using this method.
NOTE: this method may suddenly change which
implementation is returned from release to release, in
the event that higher performance defaults become
possible; if the precise implementation is important to
your application, please instantiate it directly,
instead. For optimal performance you should consider using
MMapDirectory
on 64 bit JVMs.
See above
IOException
public static FSDirectory open(Path path, LockFactory lockFactory) throws IOException
open(Path)
, but allows you to
also specify a custom LockFactory
.IOException
public static String[] listAll(Path dir) throws IOException
IOException
- if there was an I/O error during listingpublic String[] listAll() throws IOException
Directory
String.compareTo(java.lang.String)
) order.listAll
in class Directory
IOException
- in case of I/O errorpublic long fileLength(String name) throws IOException
Directory
NoSuchFileException
or FileNotFoundException
if name
points to a non-existing file.fileLength
in class Directory
name
- the name of an existing file.IOException
- in case of I/O errorpublic IndexOutput createOutput(String name, IOContext context) throws IOException
Directory
IndexOutput
instance for appending data to this file.
This method must throw FileAlreadyExistsException
if the file
already exists.createOutput
in class Directory
name
- the name of the file to create.IOException
- in case of I/O errorpublic IndexOutput createTempOutput(String prefix, String suffix, IOContext context) throws IOException
Directory
IndexOutput
instance for appending data to this file.
The temporary file name (accessible via IndexOutput.getName()
) will start with
prefix
, end with suffix
and have a reserved file extension .tmp
.createTempOutput
in class Directory
IOException
protected void ensureCanRead(String name) throws IOException
IOException
public void sync(Collection<String> names) throws IOException
Directory
sync
in class Directory
IOException
Directory.syncMetaData()
public void rename(String source, String dest) throws IOException
Directory
source
file to dest
file where
dest
must not already exist in the directory.
It is permitted for this operation to not be truly atomic, for example
both source
and dest
can be visible temporarily in Directory.listAll()
.
However, the implementation of this method must ensure the content of
dest
appears as the entire source
atomically. So once
dest
is visible for readers, the entire content of previous source
is visible.
This method is used by IndexWriter to publish commits.rename
in class Directory
IOException
public void syncMetaData() throws IOException
Directory
syncMetaData
in class Directory
IOException
Directory.sync(Collection)
public void close() throws IOException
Directory
close
in interface Closeable
close
in interface AutoCloseable
close
in class Directory
IOException
public Path getDirectory()
public String toString()
toString
in class BaseDirectory
protected void fsync(String name) throws IOException
IOException
public void deleteFile(String name) throws IOException
Directory
NoSuchFileException
or FileNotFoundException
if name
points to a non-existing file.deleteFile
in class Directory
name
- the name of an existing file.IOException
- in case of I/O errorpublic void deletePendingFiles() throws IOException
IOException
public Set<String> getPendingDeletions() throws IOException
Directory
getPendingDeletions
in class Directory
IOException
Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.