Class ParallelLeafReader
- java.lang.Object
-
- org.apache.lucene.index.IndexReader
-
- org.apache.lucene.index.LeafReader
-
- org.apache.lucene.index.ParallelLeafReader
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public class ParallelLeafReader extends LeafReader
AnLeafReader
which reads multiple, parallel indexes. Each index added must have the same number of documents, but typically each contains different fields. Deletions are taken from the first reader. Each document contains the union of the fields of all documents with the same document number. When searching, matches for a query term are from the first index added that has the field.This is useful, e.g., with collections that have large fields which change rarely and small fields that change more frequently. The smaller fields may be re-indexed in a new index and both indexes may be searched together.
Warning: It is up to you to make sure all indexes are created and modified the same way. For example, if you add documents to one index, you need to add the same documents in the same order to the other indexes. Failure to do so will result in undefined behavior.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.index.IndexReader
IndexReader.CacheHelper, IndexReader.CacheKey, IndexReader.ClosedListener
-
-
Constructor Summary
Constructors Constructor Description ParallelLeafReader(boolean closeSubReaders, LeafReader... readers)
Create a ParallelLeafReader based on the provided readers.ParallelLeafReader(boolean closeSubReaders, LeafReader[] readers, LeafReader[] storedFieldsReaders)
Expert: create a ParallelLeafReader based on the provided readers and storedFieldReaders; when a document is loaded, only storedFieldsReaders will be used.ParallelLeafReader(LeafReader... readers)
Create a ParallelLeafReader based on the provided readers; auto-closes the given readers onIndexReader.close()
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
checkIntegrity()
Checks consistency of this reader.protected void
doClose()
Implements close.void
document(int docID, StoredFieldVisitor visitor)
Expert: visits the fields of a stored document, for custom processing/loading of each field.BinaryDocValues
getBinaryDocValues(String field)
ReturnsBinaryDocValues
for this field, or null if no binary doc values were indexed for this field.ByteVectorValues
getByteVectorValues(String fieldName)
ReturnsByteVectorValues
for this field, or null if noByteVectorValues
were indexed.IndexReader.CacheHelper
getCoreCacheHelper()
Optional method: Return aIndexReader.CacheHelper
that can be used to cache based on the content of this leaf regardless of deletions.FieldInfos
getFieldInfos()
Get theFieldInfos
describing all fields in this reader.FloatVectorValues
getFloatVectorValues(String fieldName)
ReturnsFloatVectorValues
for this field, or null if noFloatVectorValues
were indexed.Bits
getLiveDocs()
Returns theBits
representing live (not deleted) docs.LeafMetaData
getMetaData()
Return metadata about this leaf.NumericDocValues
getNormValues(String field)
ReturnsNumericDocValues
representing norms for this field, or null if noNumericDocValues
were indexed.NumericDocValues
getNumericDocValues(String field)
ReturnsNumericDocValues
for this field, or null if no numeric doc values were indexed for this field.LeafReader[]
getParallelReaders()
Returns theLeafReader
s that were passed on init.PointValues
getPointValues(String fieldName)
Returns thePointValues
used for numeric or spatial searches for the given field, or null if there are no point fields.IndexReader.CacheHelper
getReaderCacheHelper()
Optional method: Return aIndexReader.CacheHelper
that can be used to cache based on the content of this reader.SortedDocValues
getSortedDocValues(String field)
ReturnsSortedDocValues
for this field, or null if noSortedDocValues
were indexed for this field.SortedNumericDocValues
getSortedNumericDocValues(String field)
ReturnsSortedNumericDocValues
for this field, or null if noSortedNumericDocValues
were indexed for this field.SortedSetDocValues
getSortedSetDocValues(String field)
ReturnsSortedSetDocValues
for this field, or null if noSortedSetDocValues
were indexed for this field.Fields
getTermVectors(int docID)
Retrieve term vectors for this document, or null if term vectors were not indexed.int
maxDoc()
Returns one greater than the largest possible document number.int
numDocs()
Returns the number of documents in this index.void
searchNearestVectors(String fieldName, byte[] target, KnnCollector knnCollector, Bits acceptDocs)
Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.void
searchNearestVectors(String fieldName, float[] target, KnnCollector knnCollector, Bits acceptDocs)
Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.StoredFields
storedFields()
Returns aStoredFields
reader for the stored fields of this index.Terms
terms(String field)
Returns theTerms
index for this field, or null if it has none.TermVectors
termVectors()
Returns aTermVectors
reader for the term vectors of this index.String
toString()
-
Methods inherited from class org.apache.lucene.index.LeafReader
docFreq, getContext, getDocCount, getSumDocFreq, getSumTotalTermFreq, postings, postings, searchNearestVectors, searchNearestVectors, totalTermFreq
-
Methods inherited from class org.apache.lucene.index.IndexReader
close, decRef, document, document, ensureOpen, equals, getRefCount, getTermVector, hasDeletions, hashCode, incRef, leaves, notifyReaderClosedListeners, numDeletedDocs, registerParentReader, tryIncRef
-
-
-
-
Constructor Detail
-
ParallelLeafReader
public ParallelLeafReader(LeafReader... readers) throws IOException
Create a ParallelLeafReader based on the provided readers; auto-closes the given readers onIndexReader.close()
.- Throws:
IOException
-
ParallelLeafReader
public ParallelLeafReader(boolean closeSubReaders, LeafReader... readers) throws IOException
Create a ParallelLeafReader based on the provided readers.- Throws:
IOException
-
ParallelLeafReader
public ParallelLeafReader(boolean closeSubReaders, LeafReader[] readers, LeafReader[] storedFieldsReaders) throws IOException
Expert: create a ParallelLeafReader based on the provided readers and storedFieldReaders; when a document is loaded, only storedFieldsReaders will be used.- Throws:
IOException
-
-
Method Detail
-
getFieldInfos
public FieldInfos getFieldInfos()
Get theFieldInfos
describing all fields in this reader.Note: Implementations should cache the FieldInfos instance returned by this method such that subsequent calls to this method return the same instance.
NOTE: the returned field numbers will likely not correspond to the actual field numbers in the underlying readers, and codec metadata (
FieldInfo.getAttribute(String)
will be unavailable.- Specified by:
getFieldInfos
in classLeafReader
-
getLiveDocs
public Bits getLiveDocs()
Description copied from class:LeafReader
Returns theBits
representing live (not deleted) docs. A set bit indicates the doc ID has not been deleted. If this method returns null it means there are no deleted documents (all documents are live).The returned instance has been safely published for use by multiple threads without additional synchronization.
- Specified by:
getLiveDocs
in classLeafReader
-
terms
public Terms terms(String field) throws IOException
Description copied from class:LeafReader
Returns theTerms
index for this field, or null if it has none.- Specified by:
terms
in classLeafReader
- Throws:
IOException
-
numDocs
public int numDocs()
Description copied from class:IndexReader
Returns the number of documents in this index.NOTE: This operation may run in O(maxDoc). Implementations that can't return this number in constant-time should cache it.
- Specified by:
numDocs
in classIndexReader
-
maxDoc
public int maxDoc()
Description copied from class:IndexReader
Returns one greater than the largest possible document number. This may be used to, e.g., determine how big to allocate an array which will have an element for every document number in an index.- Specified by:
maxDoc
in classIndexReader
-
document
public void document(int docID, StoredFieldVisitor visitor) throws IOException
Description copied from class:IndexReader
Expert: visits the fields of a stored document, for custom processing/loading of each field. If you simply want to load all fields, useIndexReader.document(int)
. If you want to load a subset, useDocumentStoredFieldVisitor
.- Specified by:
document
in classIndexReader
- Throws:
IOException
-
storedFields
public StoredFields storedFields() throws IOException
Description copied from class:IndexReader
Returns aStoredFields
reader for the stored fields of this index.This call never returns
null
, even if no stored fields were indexed. The returned instance should only be used by a single thread.Example:
TopDocs hits = searcher.search(query, 10); StoredFields storedFields = reader.storedFields(); for (ScoreDoc hit : hits.scoreDocs) { Document doc = storedFields.document(hit.doc); }
- Specified by:
storedFields
in classIndexReader
- Throws:
IOException
- If there is a low-level IO error
-
getCoreCacheHelper
public IndexReader.CacheHelper getCoreCacheHelper()
Description copied from class:LeafReader
Optional method: Return aIndexReader.CacheHelper
that can be used to cache based on the content of this leaf regardless of deletions. Two readers that have the same data but different sets of deleted documents or doc values updates may be considered equal. Consider usingIndexReader.getReaderCacheHelper()
if you need deletions or dv updates to be taken into account.A return value of
null
indicates that this reader is not suited for caching, which is typically the case for short-lived wrappers that alter the content of the wrapped leaf reader.- Specified by:
getCoreCacheHelper
in classLeafReader
-
getReaderCacheHelper
public IndexReader.CacheHelper getReaderCacheHelper()
Description copied from class:IndexReader
Optional method: Return aIndexReader.CacheHelper
that can be used to cache based on the content of this reader. Two readers that have different data or different sets of deleted documents will be considered different.A return value of
null
indicates that this reader is not suited for caching, which is typically the case for short-lived wrappers that alter the content of the wrapped reader.- Specified by:
getReaderCacheHelper
in classIndexReader
-
getTermVectors
public Fields getTermVectors(int docID) throws IOException
Description copied from class:IndexReader
Retrieve term vectors for this document, or null if term vectors were not indexed. The returned Fields instance acts like a single-document inverted index (the docID will be 0).- Specified by:
getTermVectors
in classIndexReader
- Throws:
IOException
-
termVectors
public TermVectors termVectors() throws IOException
Description copied from class:IndexReader
Returns aTermVectors
reader for the term vectors of this index.This call never returns
null
, even if no term vectors were indexed. The returned instance should only be used by a single thread.Example:
TopDocs hits = searcher.search(query, 10); TermVectors termVectors = reader.termVectors(); for (ScoreDoc hit : hits.scoreDocs) { Fields vector = termVectors.get(hit.doc); }
- Specified by:
termVectors
in classIndexReader
- Throws:
IOException
- If there is a low-level IO error
-
doClose
protected void doClose() throws IOException
Description copied from class:IndexReader
Implements close.- Specified by:
doClose
in classIndexReader
- Throws:
IOException
-
getNumericDocValues
public NumericDocValues getNumericDocValues(String field) throws IOException
Description copied from class:LeafReader
ReturnsNumericDocValues
for this field, or null if no numeric doc values were indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getNumericDocValues
in classLeafReader
- Throws:
IOException
-
getBinaryDocValues
public BinaryDocValues getBinaryDocValues(String field) throws IOException
Description copied from class:LeafReader
ReturnsBinaryDocValues
for this field, or null if no binary doc values were indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getBinaryDocValues
in classLeafReader
- Throws:
IOException
-
getSortedDocValues
public SortedDocValues getSortedDocValues(String field) throws IOException
Description copied from class:LeafReader
ReturnsSortedDocValues
for this field, or null if noSortedDocValues
were indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getSortedDocValues
in classLeafReader
- Throws:
IOException
-
getSortedNumericDocValues
public SortedNumericDocValues getSortedNumericDocValues(String field) throws IOException
Description copied from class:LeafReader
ReturnsSortedNumericDocValues
for this field, or null if noSortedNumericDocValues
were indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getSortedNumericDocValues
in classLeafReader
- Throws:
IOException
-
getSortedSetDocValues
public SortedSetDocValues getSortedSetDocValues(String field) throws IOException
Description copied from class:LeafReader
ReturnsSortedSetDocValues
for this field, or null if noSortedSetDocValues
were indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getSortedSetDocValues
in classLeafReader
- Throws:
IOException
-
getNormValues
public NumericDocValues getNormValues(String field) throws IOException
Description copied from class:LeafReader
ReturnsNumericDocValues
representing norms for this field, or null if noNumericDocValues
were indexed. The returned instance should only be used by a single thread.- Specified by:
getNormValues
in classLeafReader
- Throws:
IOException
-
getPointValues
public PointValues getPointValues(String fieldName) throws IOException
Description copied from class:LeafReader
Returns thePointValues
used for numeric or spatial searches for the given field, or null if there are no point fields.- Specified by:
getPointValues
in classLeafReader
- Throws:
IOException
-
getFloatVectorValues
public FloatVectorValues getFloatVectorValues(String fieldName) throws IOException
Description copied from class:LeafReader
ReturnsFloatVectorValues
for this field, or null if noFloatVectorValues
were indexed. The returned instance should only be used by a single thread.- Specified by:
getFloatVectorValues
in classLeafReader
- Throws:
IOException
-
getByteVectorValues
public ByteVectorValues getByteVectorValues(String fieldName) throws IOException
Description copied from class:LeafReader
ReturnsByteVectorValues
for this field, or null if noByteVectorValues
were indexed. The returned instance should only be used by a single thread.- Specified by:
getByteVectorValues
in classLeafReader
- Throws:
IOException
-
searchNearestVectors
public void searchNearestVectors(String fieldName, float[] target, KnnCollector knnCollector, Bits acceptDocs) throws IOException
Description copied from class:LeafReader
Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.
The returned
TopDocs
will contain aScoreDoc
for each nearest neighbor, in order of their similarity to the query vector (decreasing scores). TheTotalHits
contains the number of documents visited during the search. If the search stopped early because it hitvisitedLimit
, it is indicated through the relationTotalHits.Relation.GREATER_THAN_OR_EQUAL_TO
.The behavior is undefined if the given field doesn't have KNN vectors enabled on its
FieldInfo
. The return value is nevernull
.- Specified by:
searchNearestVectors
in classLeafReader
- Parameters:
fieldName
- the vector field to searchtarget
- the vector-valued queryknnCollector
- collector with settings for gathering the vector results.acceptDocs
-Bits
that represents the allowed documents to match, ornull
if they are all allowed to match.- Throws:
IOException
-
searchNearestVectors
public void searchNearestVectors(String fieldName, byte[] target, KnnCollector knnCollector, Bits acceptDocs) throws IOException
Description copied from class:LeafReader
Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.
The returned
TopDocs
will contain aScoreDoc
for each nearest neighbor, in order of their similarity to the query vector (decreasing scores). TheTotalHits
contains the number of documents visited during the search. If the search stopped early because it hitvisitedLimit
, it is indicated through the relationTotalHits.Relation.GREATER_THAN_OR_EQUAL_TO
.The behavior is undefined if the given field doesn't have KNN vectors enabled on its
FieldInfo
. The return value is nevernull
.- Specified by:
searchNearestVectors
in classLeafReader
- Parameters:
fieldName
- the vector field to searchtarget
- the vector-valued queryknnCollector
- collector with settings for gathering the vector results.acceptDocs
-Bits
that represents the allowed documents to match, ornull
if they are all allowed to match.- Throws:
IOException
-
checkIntegrity
public void checkIntegrity() throws IOException
Description copied from class:LeafReader
Checks consistency of this reader.Note that this may be costly in terms of I/O, e.g. may involve computing a checksum value against large data files.
- Specified by:
checkIntegrity
in classLeafReader
- Throws:
IOException
-
getParallelReaders
public LeafReader[] getParallelReaders()
Returns theLeafReader
s that were passed on init.
-
getMetaData
public LeafMetaData getMetaData()
Description copied from class:LeafReader
Return metadata about this leaf.- Specified by:
getMetaData
in classLeafReader
-
-