Package org.apache.lucene.index
Class BaseDocValuesFormatTestCase
- java.lang.Object
-
- org.junit.Assert
-
- org.apache.lucene.util.LuceneTestCase
-
- org.apache.lucene.index.BaseDocValuesFormatTestCase
-
- Direct Known Subclasses:
BaseCompressingDocValuesFormatTestCase
public abstract class BaseDocValuesFormatTestCase extends LuceneTestCase
Abstract class to do basic tests for a docvalues format. NOTE: This test focuses on the docvalues impl, nothing else. The [stretch] goal is for this test to be so thorough in testing a new DocValuesFormat that if this test passes, then all Lucene/Solr tests should also pass. Ie, if there is some bug in a given DocValuesFormat that this test fails to catch then this test needs to be improved!
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.util.LuceneTestCase
LuceneTestCase.AwaitsFix, LuceneTestCase.BadApple, LuceneTestCase.Monster, LuceneTestCase.Nightly, LuceneTestCase.Slow, LuceneTestCase.SuppressCodecs, LuceneTestCase.SuppressFileSystems, LuceneTestCase.SuppressFsync, LuceneTestCase.SuppressReproduceLine, LuceneTestCase.SuppressSysoutChecks, LuceneTestCase.SuppressTempFileChecks, LuceneTestCase.ThrowingRunnable, LuceneTestCase.Weekly
-
-
Field Summary
-
Fields inherited from class org.apache.lucene.util.LuceneTestCase
assertsAreEnabled, classRules, DEFAULT_LINE_DOCS_FILE, INFOSTREAM, JENKINS_LARGE_LINE_DOCS_FILE, LEAVE_TEMPORARY, MAYBE_CACHE_POLICY, RANDOM_MULTIPLIER, ruleChain, suiteFailureMarker, SYSPROP_AWAITSFIX, SYSPROP_BADAPPLES, SYSPROP_FAILFAST, SYSPROP_MAXFAILURES, SYSPROP_MONSTER, SYSPROP_NIGHTLY, SYSPROP_SLOW, SYSPROP_WEEKLY, TEST_ASSERTS_ENABLED, TEST_AWAITSFIX, TEST_BADAPPLES, TEST_CODEC, TEST_DIRECTORY, TEST_DOCVALUESFORMAT, TEST_LINE_DOCS_FILE, TEST_MONSTER, TEST_NIGHTLY, TEST_POSTINGSFORMAT, TEST_SLOW, TEST_THROTTLING, TEST_WEEKLY, VERBOSE
-
-
Constructor Summary
Constructors Constructor Description BaseDocValuesFormatTestCase()
-
Method Summary
-
Methods inherited from class org.apache.lucene.util.LuceneTestCase
addVirusChecker, assertDeletedDocsEquals, assertDocsAndPositionsEnumEquals, assertDocsEnumEquals, assertDocsSkippingEquals, assertDocValuesEquals, assertDocValuesEquals, assertFieldInfosEquals, assertNormsEquals, assertPointsEquals, assertPositionsSkippingEquals, assertReaderEquals, assertReaderStatisticsEquals, assertStoredFieldEquals, assertStoredFieldsEquals, assertTermsEnumEquals, assertTermsEquals, assertTermsEquals, assertTermsStatisticsEquals, assertTermStatsEquals, assertTermVectorsEquals, asSet, assumeFalse, assumeNoException, assumeTrue, assumeWorkingMMapOnWindows, atLeast, atLeast, callStackContains, callStackContains, callStackContainsAnyOf, closeAfterSuite, closeAfterTest, collate, createTempDir, createTempDir, createTempFile, createTempFile, dumpArray, dumpIterator, ensureSaneIWCOnNightly, expectThrows, expectThrows, expectThrows, expectThrowsAnyOf, expectThrowsAnyOf, getDataInputStream, getDataPath, getOnlyLeafReader, getTestClass, getTestName, hasWorkingMMapOnWindows, isTestThread, localeForLanguageTag, maybeChangeLiveIndexWriterConfig, maybeWrapReader, newAlcoholicMergePolicy, newAlcoholicMergePolicy, newBytesRef, newBytesRef, newBytesRef, newBytesRef, newBytesRef, newBytesRef, newDirectory, newDirectory, newDirectory, newDirectory, newDirectory, newField, newField, newFSDirectory, newFSDirectory, newIndexWriterConfig, newIndexWriterConfig, newIndexWriterConfig, newIOContext, newIOContext, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newMaybeVirusCheckingDirectory, newMaybeVirusCheckingFSDirectory, newMergePolicy, newMergePolicy, newMergePolicy, newMockDirectory, newMockDirectory, newMockDirectory, newMockFSDirectory, newMockFSDirectory, newSearcher, newSearcher, newSearcher, newSearcher, newStringField, newStringField, newStringField, newStringField, newTextField, newTextField, newTieredMergePolicy, newTieredMergePolicy, overrideDefaultQueryCache, overrideTestDefaultQueryCache, random, randomLocale, randomTimeZone, rarely, rarely, replaceMaxFailureRule, resetDefaultQueryCache, restoreCPUCoreCount, restoreIndexWriterMaxDocs, runWithRestrictedPermissions, setIndexWriterMaxDocs, setupCPUCoreCount, slowFileExists, usually, usually, wrapReader
-
Methods inherited from class org.junit.Assert
assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertThat, assertThat, assertThrows, assertThrows, assertTrue, assertTrue, fail, fail
-
-
-
-
Method Detail
-
addRandomFields
protected void addRandomFields(Document doc)
Add random fields to the provided document.
-
testOneNumber
public void testOneNumber() throws IOException
- Throws:
IOException
-
testOneFloat
public void testOneFloat() throws IOException
- Throws:
IOException
-
testTwoNumbers
public void testTwoNumbers() throws IOException
- Throws:
IOException
-
testTwoBinaryValues
public void testTwoBinaryValues() throws IOException
- Throws:
IOException
-
testVariouslyCompressibleBinaryValues
public void testVariouslyCompressibleBinaryValues() throws IOException
- Throws:
IOException
-
testTwoFieldsMixed
public void testTwoFieldsMixed() throws IOException
- Throws:
IOException
-
testThreeFieldsMixed
public void testThreeFieldsMixed() throws IOException
- Throws:
IOException
-
testThreeFieldsMixed2
public void testThreeFieldsMixed2() throws IOException
- Throws:
IOException
-
testTwoDocumentsNumeric
public void testTwoDocumentsNumeric() throws IOException
- Throws:
IOException
-
testTwoDocumentsMerged
public void testTwoDocumentsMerged() throws IOException
- Throws:
IOException
-
testBigNumericRange
public void testBigNumericRange() throws IOException
- Throws:
IOException
-
testBigNumericRange2
public void testBigNumericRange2() throws IOException
- Throws:
IOException
-
testBytes
public void testBytes() throws IOException
- Throws:
IOException
-
testBytesTwoDocumentsMerged
public void testBytesTwoDocumentsMerged() throws IOException
- Throws:
IOException
-
testBytesMergeAwayAllValues
public void testBytesMergeAwayAllValues() throws IOException
- Throws:
IOException
-
testSortedBytes
public void testSortedBytes() throws IOException
- Throws:
IOException
-
testSortedBytesTwoDocuments
public void testSortedBytesTwoDocuments() throws IOException
- Throws:
IOException
-
testSortedBytesThreeDocuments
public void testSortedBytesThreeDocuments() throws IOException
- Throws:
IOException
-
testSortedBytesTwoDocumentsMerged
public void testSortedBytesTwoDocumentsMerged() throws IOException
- Throws:
IOException
-
testSortedMergeAwayAllValues
public void testSortedMergeAwayAllValues() throws IOException
- Throws:
IOException
-
testBytesWithNewline
public void testBytesWithNewline() throws IOException
- Throws:
IOException
-
testMissingSortedBytes
public void testMissingSortedBytes() throws IOException
- Throws:
IOException
-
testSortedTermsEnum
public void testSortedTermsEnum() throws IOException
- Throws:
IOException
-
testEmptySortedBytes
public void testEmptySortedBytes() throws IOException
- Throws:
IOException
-
testEmptyBytes
public void testEmptyBytes() throws IOException
- Throws:
IOException
-
testVeryLargeButLegalBytes
public void testVeryLargeButLegalBytes() throws IOException
- Throws:
IOException
-
testVeryLargeButLegalSortedBytes
public void testVeryLargeButLegalSortedBytes() throws IOException
- Throws:
IOException
-
testCodecUsesOwnBytes
public void testCodecUsesOwnBytes() throws IOException
- Throws:
IOException
-
testCodecUsesOwnSortedBytes
public void testCodecUsesOwnSortedBytes() throws IOException
- Throws:
IOException
-
testDocValuesSimple
public void testDocValuesSimple() throws IOException
- Throws:
IOException
-
testRandomSortedBytes
public void testRandomSortedBytes() throws IOException
- Throws:
IOException
-
assertDVIterate
protected void assertDVIterate(Directory dir) throws IOException
- Throws:
IOException
-
testBooleanNumericsVsStoredFields
public void testBooleanNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseBooleanNumericsVsStoredFields
public void testSparseBooleanNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testByteNumericsVsStoredFields
public void testByteNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseByteNumericsVsStoredFields
public void testSparseByteNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testShortNumericsVsStoredFields
public void testShortNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseShortNumericsVsStoredFields
public void testSparseShortNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testIntNumericsVsStoredFields
public void testIntNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseIntNumericsVsStoredFields
public void testSparseIntNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testLongNumericsVsStoredFields
public void testLongNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseLongNumericsVsStoredFields
public void testSparseLongNumericsVsStoredFields() throws Exception
- Throws:
Exception
-
testBinaryFixedLengthVsStoredFields
public void testBinaryFixedLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseBinaryFixedLengthVsStoredFields
public void testSparseBinaryFixedLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testBinaryVariableLengthVsStoredFields
public void testBinaryVariableLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseBinaryVariableLengthVsStoredFields
public void testSparseBinaryVariableLengthVsStoredFields() throws Exception
- Throws:
Exception
-
doTestBinaryVariableLengthVsStoredFields
public void doTestBinaryVariableLengthVsStoredFields(double density) throws Exception
- Throws:
Exception
-
doTestSortedVsStoredFields
protected void doTestSortedVsStoredFields(int numDocs, double density, Supplier<byte[]> bytes) throws Exception
- Throws:
Exception
-
testSortedFixedLengthVsStoredFields
public void testSortedFixedLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseSortedFixedLengthVsStoredFields
public void testSparseSortedFixedLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedVariableLengthVsStoredFields
public void testSortedVariableLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testSparseSortedVariableLengthVsStoredFields
public void testSparseSortedVariableLengthVsStoredFields() throws Exception
- Throws:
Exception
-
doTestSortedVsStoredFields
protected void doTestSortedVsStoredFields(int numDocs, double density, int minLength, int maxLength) throws Exception
- Throws:
Exception
-
testSortedSetOneValue
public void testSortedSetOneValue() throws IOException
- Throws:
IOException
-
testSortedSetTwoFields
public void testSortedSetTwoFields() throws IOException
- Throws:
IOException
-
testSortedSetTwoDocumentsMerged
public void testSortedSetTwoDocumentsMerged() throws IOException
- Throws:
IOException
-
testSortedSetTwoValues
public void testSortedSetTwoValues() throws IOException
- Throws:
IOException
-
testSortedSetTwoValuesUnordered
public void testSortedSetTwoValuesUnordered() throws IOException
- Throws:
IOException
-
testSortedSetThreeValuesTwoDocs
public void testSortedSetThreeValuesTwoDocs() throws IOException
- Throws:
IOException
-
testSortedSetTwoDocumentsLastMissing
public void testSortedSetTwoDocumentsLastMissing() throws IOException
- Throws:
IOException
-
testSortedSetTwoDocumentsLastMissingMerge
public void testSortedSetTwoDocumentsLastMissingMerge() throws IOException
- Throws:
IOException
-
testSortedSetTwoDocumentsFirstMissing
public void testSortedSetTwoDocumentsFirstMissing() throws IOException
- Throws:
IOException
-
testSortedSetTwoDocumentsFirstMissingMerge
public void testSortedSetTwoDocumentsFirstMissingMerge() throws IOException
- Throws:
IOException
-
testSortedSetMergeAwayAllValues
public void testSortedSetMergeAwayAllValues() throws IOException
- Throws:
IOException
-
testSortedSetTermsEnum
public void testSortedSetTermsEnum() throws IOException
- Throws:
IOException
-
doTestSortedSetVsStoredFields
protected void doTestSortedSetVsStoredFields(int numDocs, int minLength, int maxLength, int maxValuesPerDoc, int maxUniqueValues) throws Exception
- Throws:
Exception
-
testSortedSetFixedLengthVsStoredFields
public void testSortedSetFixedLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedNumericsSingleValuedVsStoredFields
public void testSortedNumericsSingleValuedVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedNumericsSingleValuedMissingVsStoredFields
public void testSortedNumericsSingleValuedMissingVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedNumericsMultipleValuesVsStoredFields
public void testSortedNumericsMultipleValuesVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedNumericsFewUniqueSetsVsStoredFields
public void testSortedNumericsFewUniqueSetsVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedSetVariableLengthVsStoredFields
public void testSortedSetVariableLengthVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedSetFixedLengthSingleValuedVsStoredFields
public void testSortedSetFixedLengthSingleValuedVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedSetVariableLengthSingleValuedVsStoredFields
public void testSortedSetVariableLengthSingleValuedVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedSetFixedLengthFewUniqueSetsVsStoredFields
public void testSortedSetFixedLengthFewUniqueSetsVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedSetVariableLengthFewUniqueSetsVsStoredFields
public void testSortedSetVariableLengthFewUniqueSetsVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedSetVariableLengthManyValuesPerDocVsStoredFields
public void testSortedSetVariableLengthManyValuesPerDocVsStoredFields() throws Exception
- Throws:
Exception
-
testSortedSetFixedLengthManyValuesPerDocVsStoredFields
public void testSortedSetFixedLengthManyValuesPerDocVsStoredFields() throws Exception
- Throws:
Exception
-
testTwoNumbersOneMissing
public void testTwoNumbersOneMissing() throws IOException
- Throws:
IOException
-
testTwoNumbersOneMissingWithMerging
public void testTwoNumbersOneMissingWithMerging() throws IOException
- Throws:
IOException
-
testThreeNumbersOneMissingWithMerging
public void testThreeNumbersOneMissingWithMerging() throws IOException
- Throws:
IOException
-
testTwoBytesOneMissing
public void testTwoBytesOneMissing() throws IOException
- Throws:
IOException
-
testTwoBytesOneMissingWithMerging
public void testTwoBytesOneMissingWithMerging() throws IOException
- Throws:
IOException
-
testThreeBytesOneMissingWithMerging
public void testThreeBytesOneMissingWithMerging() throws IOException
- Throws:
IOException
-
testThreads
public void testThreads() throws Exception
Tests dv against stored fields with threads (binary/numeric/sorted, no missing)- Throws:
Exception
-
testThreads2
@Nightly public void testThreads2() throws Exception
Tests dv against stored fields with threads (all types + missing)- Throws:
Exception
-
testEmptyBinaryValueOnPageSizes
public void testEmptyBinaryValueOnPageSizes() throws Exception
- Throws:
Exception
-
testOneSortedNumber
public void testOneSortedNumber() throws IOException
- Throws:
IOException
-
testOneSortedNumberOneMissing
public void testOneSortedNumberOneMissing() throws IOException
- Throws:
IOException
-
testNumberMergeAwayAllValues
public void testNumberMergeAwayAllValues() throws IOException
- Throws:
IOException
-
testTwoSortedNumber
public void testTwoSortedNumber() throws IOException
- Throws:
IOException
-
testTwoSortedNumberSameValue
public void testTwoSortedNumberSameValue() throws IOException
- Throws:
IOException
-
testTwoSortedNumberOneMissing
public void testTwoSortedNumberOneMissing() throws IOException
- Throws:
IOException
-
testSortedNumberMerge
public void testSortedNumberMerge() throws IOException
- Throws:
IOException
-
testSortedNumberMergeAwayAllValues
public void testSortedNumberMergeAwayAllValues() throws IOException
- Throws:
IOException
-
testSortedEnumAdvanceIndependently
public void testSortedEnumAdvanceIndependently() throws IOException
- Throws:
IOException
-
testSortedSetEnumAdvanceIndependently
public void testSortedSetEnumAdvanceIndependently() throws IOException
- Throws:
IOException
-
doTestSortedSetEnumAdvanceIndependently
protected void doTestSortedSetEnumAdvanceIndependently(SortedSetDocValues dv) throws IOException
- Throws:
IOException
-
testSortedMergeAwayAllValuesLargeSegment
public void testSortedMergeAwayAllValuesLargeSegment() throws IOException
- Throws:
IOException
-
testSortedSetMergeAwayAllValuesLargeSegment
public void testSortedSetMergeAwayAllValuesLargeSegment() throws IOException
- Throws:
IOException
-
testNumericMergeAwayAllValuesLargeSegment
public void testNumericMergeAwayAllValuesLargeSegment() throws IOException
- Throws:
IOException
-
testSortedNumericMergeAwayAllValuesLargeSegment
public void testSortedNumericMergeAwayAllValuesLargeSegment() throws IOException
- Throws:
IOException
-
testBinaryMergeAwayAllValuesLargeSegment
public void testBinaryMergeAwayAllValuesLargeSegment() throws IOException
- Throws:
IOException
-
testRandomAdvanceNumeric
public void testRandomAdvanceNumeric() throws IOException
- Throws:
IOException
-
testRandomAdvanceBinary
public void testRandomAdvanceBinary() throws IOException
- Throws:
IOException
-
testHighOrdsSortedSetDV
@Nightly public void testHighOrdsSortedSetDV() throws Exception
Tests where a DVField uses a high number of packed bits to store its ords. See: https://issues.apache.org/jira/browse/LUCENE-10159- Throws:
Exception
-
codecAcceptsHugeBinaryValues
protected boolean codecAcceptsHugeBinaryValues(String field)
-
getCodec
protected abstract Codec getCodec()
Returns the codec to run tests against
-
getCreatedVersionMajor
protected int getCreatedVersionMajor()
Returns the major version that this codec is compatible with.
-
applyCreatedVersionMajor
protected final <D extends Directory> D applyCreatedVersionMajor(D d) throws IOException
Set the created version of the givenDirectory
and return it.- Throws:
IOException
-
setUp
public void setUp() throws Exception
Description copied from class:LuceneTestCase
For subclasses to override. Overrides must callsuper.setUp()
.- Overrides:
setUp
in classLuceneTestCase
- Throws:
Exception
-
tearDown
public void tearDown() throws Exception
Description copied from class:LuceneTestCase
For subclasses to override. Overrides must callsuper.tearDown()
.- Overrides:
tearDown
in classLuceneTestCase
- Throws:
Exception
-
excludedExtensionsFromByteCounts
protected Collection<String> excludedExtensionsFromByteCounts()
Return the list of extensions that should be excluded from byte counts when comparing indices that store the same content.
-
testMergeStability
public void testMergeStability() throws Exception
The purpose of this test is to make sure that bulk merge doesn't accumulate useless data over runs.- Throws:
Exception
-
mergeIsStable
protected boolean mergeIsStable()
-
testMultiClose
public void testMultiClose() throws IOException
Calls close multiple times on closeable codec apis- Throws:
IOException
-
testRandomExceptions
public void testRandomExceptions() throws Exception
Tests exception handling on write and openInput/createOutput- Throws:
Exception
-
shouldTestMergeInstance
protected boolean shouldTestMergeInstance()
Returnsfalse
if only the regular fields reader should be tested, andtrue
if only the merge instance should be tested.
-
maybeWrapWithMergingReader
protected final DirectoryReader maybeWrapWithMergingReader(DirectoryReader r) throws IOException
- Throws:
IOException
-
testCheckIntegrityReadsAllBytes
public void testCheckIntegrityReadsAllBytes() throws Exception
This test is a best effort at verifying that checkIntegrity doesn't miss any files. It tests that the combination of opening a reader and calling checkIntegrity on it reads all bytes of all files.- Throws:
Exception
-
-