Class BaseDocValuesFormatTestCase

  • Direct Known Subclasses:
    BaseCompressingDocValuesFormatTestCase

    public abstract class BaseDocValuesFormatTestCase
    extends LuceneTestCase
    Abstract class to do basic tests for a docvalues format. NOTE: This test focuses on the docvalues impl, nothing else. The [stretch] goal is for this test to be so thorough in testing a new DocValuesFormat that if this test passes, then all Lucene/Solr tests should also pass. Ie, if there is some bug in a given DocValuesFormat that this test fails to catch then this test needs to be improved!
    • Constructor Detail

      • BaseDocValuesFormatTestCase

        public BaseDocValuesFormatTestCase()
    • Method Detail

      • addRandomFields

        protected void addRandomFields​(Document doc)
        Add random fields to the provided document.
      • testVariouslyCompressibleBinaryValues

        public void testVariouslyCompressibleBinaryValues()
                                                   throws IOException
        Throws:
        IOException
      • testBytesTwoDocumentsMerged

        public void testBytesTwoDocumentsMerged()
                                         throws IOException
        Throws:
        IOException
      • testBytesMergeAwayAllValues

        public void testBytesMergeAwayAllValues()
                                         throws IOException
        Throws:
        IOException
      • testSortedBytesTwoDocuments

        public void testSortedBytesTwoDocuments()
                                         throws IOException
        Throws:
        IOException
      • testSortedBytesThreeDocuments

        public void testSortedBytesThreeDocuments()
                                           throws IOException
        Throws:
        IOException
      • testSortedBytesTwoDocumentsMerged

        public void testSortedBytesTwoDocumentsMerged()
                                               throws IOException
        Throws:
        IOException
      • testSortedMergeAwayAllValues

        public void testSortedMergeAwayAllValues()
                                          throws IOException
        Throws:
        IOException
      • testVeryLargeButLegalBytes

        public void testVeryLargeButLegalBytes()
                                        throws IOException
        Throws:
        IOException
      • testVeryLargeButLegalSortedBytes

        public void testVeryLargeButLegalSortedBytes()
                                              throws IOException
        Throws:
        IOException
      • testCodecUsesOwnSortedBytes

        public void testCodecUsesOwnSortedBytes()
                                         throws IOException
        Throws:
        IOException
      • testBooleanNumericsVsStoredFields

        public void testBooleanNumericsVsStoredFields()
                                               throws Exception
        Throws:
        Exception
      • testSparseBooleanNumericsVsStoredFields

        public void testSparseBooleanNumericsVsStoredFields()
                                                     throws Exception
        Throws:
        Exception
      • testByteNumericsVsStoredFields

        public void testByteNumericsVsStoredFields()
                                            throws Exception
        Throws:
        Exception
      • testSparseByteNumericsVsStoredFields

        public void testSparseByteNumericsVsStoredFields()
                                                  throws Exception
        Throws:
        Exception
      • testShortNumericsVsStoredFields

        public void testShortNumericsVsStoredFields()
                                             throws Exception
        Throws:
        Exception
      • testSparseShortNumericsVsStoredFields

        public void testSparseShortNumericsVsStoredFields()
                                                   throws Exception
        Throws:
        Exception
      • testIntNumericsVsStoredFields

        public void testIntNumericsVsStoredFields()
                                           throws Exception
        Throws:
        Exception
      • testSparseIntNumericsVsStoredFields

        public void testSparseIntNumericsVsStoredFields()
                                                 throws Exception
        Throws:
        Exception
      • testLongNumericsVsStoredFields

        public void testLongNumericsVsStoredFields()
                                            throws Exception
        Throws:
        Exception
      • testSparseLongNumericsVsStoredFields

        public void testSparseLongNumericsVsStoredFields()
                                                  throws Exception
        Throws:
        Exception
      • testBinaryFixedLengthVsStoredFields

        public void testBinaryFixedLengthVsStoredFields()
                                                 throws Exception
        Throws:
        Exception
      • testSparseBinaryFixedLengthVsStoredFields

        public void testSparseBinaryFixedLengthVsStoredFields()
                                                       throws Exception
        Throws:
        Exception
      • testBinaryVariableLengthVsStoredFields

        public void testBinaryVariableLengthVsStoredFields()
                                                    throws Exception
        Throws:
        Exception
      • testSparseBinaryVariableLengthVsStoredFields

        public void testSparseBinaryVariableLengthVsStoredFields()
                                                          throws Exception
        Throws:
        Exception
      • doTestBinaryVariableLengthVsStoredFields

        public void doTestBinaryVariableLengthVsStoredFields​(double density)
                                                      throws Exception
        Throws:
        Exception
      • doTestSortedVsStoredFields

        protected void doTestSortedVsStoredFields​(int numDocs,
                                                  double density,
                                                  Supplier<byte[]> bytes)
                                           throws Exception
        Throws:
        Exception
      • testSortedFixedLengthVsStoredFields

        public void testSortedFixedLengthVsStoredFields()
                                                 throws Exception
        Throws:
        Exception
      • testSparseSortedFixedLengthVsStoredFields

        public void testSparseSortedFixedLengthVsStoredFields()
                                                       throws Exception
        Throws:
        Exception
      • testSortedVariableLengthVsStoredFields

        public void testSortedVariableLengthVsStoredFields()
                                                    throws Exception
        Throws:
        Exception
      • testSparseSortedVariableLengthVsStoredFields

        public void testSparseSortedVariableLengthVsStoredFields()
                                                          throws Exception
        Throws:
        Exception
      • doTestSortedVsStoredFields

        protected void doTestSortedVsStoredFields​(int numDocs,
                                                  double density,
                                                  int minLength,
                                                  int maxLength)
                                           throws Exception
        Throws:
        Exception
      • testSortedSetTwoDocumentsMerged

        public void testSortedSetTwoDocumentsMerged()
                                             throws IOException
        Throws:
        IOException
      • testSortedSetTwoValuesUnordered

        public void testSortedSetTwoValuesUnordered()
                                             throws IOException
        Throws:
        IOException
      • testSortedSetThreeValuesTwoDocs

        public void testSortedSetThreeValuesTwoDocs()
                                             throws IOException
        Throws:
        IOException
      • testSortedSetTwoDocumentsLastMissing

        public void testSortedSetTwoDocumentsLastMissing()
                                                  throws IOException
        Throws:
        IOException
      • testSortedSetTwoDocumentsLastMissingMerge

        public void testSortedSetTwoDocumentsLastMissingMerge()
                                                       throws IOException
        Throws:
        IOException
      • testSortedSetTwoDocumentsFirstMissing

        public void testSortedSetTwoDocumentsFirstMissing()
                                                   throws IOException
        Throws:
        IOException
      • testSortedSetTwoDocumentsFirstMissingMerge

        public void testSortedSetTwoDocumentsFirstMissingMerge()
                                                        throws IOException
        Throws:
        IOException
      • testSortedSetMergeAwayAllValues

        public void testSortedSetMergeAwayAllValues()
                                             throws IOException
        Throws:
        IOException
      • doTestSortedSetVsStoredFields

        protected void doTestSortedSetVsStoredFields​(int numDocs,
                                                     int minLength,
                                                     int maxLength,
                                                     int maxValuesPerDoc,
                                                     int maxUniqueValues)
                                              throws Exception
        Throws:
        Exception
      • testSortedSetFixedLengthVsStoredFields

        public void testSortedSetFixedLengthVsStoredFields()
                                                    throws Exception
        Throws:
        Exception
      • testSortedNumericsSingleValuedVsStoredFields

        public void testSortedNumericsSingleValuedVsStoredFields()
                                                          throws Exception
        Throws:
        Exception
      • testSortedNumericsSingleValuedMissingVsStoredFields

        public void testSortedNumericsSingleValuedMissingVsStoredFields()
                                                                 throws Exception
        Throws:
        Exception
      • testSortedNumericsMultipleValuesVsStoredFields

        public void testSortedNumericsMultipleValuesVsStoredFields()
                                                            throws Exception
        Throws:
        Exception
      • testSortedNumericsFewUniqueSetsVsStoredFields

        public void testSortedNumericsFewUniqueSetsVsStoredFields()
                                                           throws Exception
        Throws:
        Exception
      • testSortedSetVariableLengthVsStoredFields

        public void testSortedSetVariableLengthVsStoredFields()
                                                       throws Exception
        Throws:
        Exception
      • testSortedSetFixedLengthSingleValuedVsStoredFields

        public void testSortedSetFixedLengthSingleValuedVsStoredFields()
                                                                throws Exception
        Throws:
        Exception
      • testSortedSetVariableLengthSingleValuedVsStoredFields

        public void testSortedSetVariableLengthSingleValuedVsStoredFields()
                                                                   throws Exception
        Throws:
        Exception
      • testSortedSetFixedLengthFewUniqueSetsVsStoredFields

        public void testSortedSetFixedLengthFewUniqueSetsVsStoredFields()
                                                                 throws Exception
        Throws:
        Exception
      • testSortedSetVariableLengthFewUniqueSetsVsStoredFields

        public void testSortedSetVariableLengthFewUniqueSetsVsStoredFields()
                                                                    throws Exception
        Throws:
        Exception
      • testSortedSetVariableLengthManyValuesPerDocVsStoredFields

        public void testSortedSetVariableLengthManyValuesPerDocVsStoredFields()
                                                                       throws Exception
        Throws:
        Exception
      • testSortedSetFixedLengthManyValuesPerDocVsStoredFields

        public void testSortedSetFixedLengthManyValuesPerDocVsStoredFields()
                                                                    throws Exception
        Throws:
        Exception
      • testGCDCompression

        public void testGCDCompression()
                                throws Exception
        Throws:
        Exception
      • testSparseGCDCompression

        public void testSparseGCDCompression()
                                      throws Exception
        Throws:
        Exception
      • testTwoNumbersOneMissingWithMerging

        public void testTwoNumbersOneMissingWithMerging()
                                                 throws IOException
        Throws:
        IOException
      • testThreeNumbersOneMissingWithMerging

        public void testThreeNumbersOneMissingWithMerging()
                                                   throws IOException
        Throws:
        IOException
      • testTwoBytesOneMissingWithMerging

        public void testTwoBytesOneMissingWithMerging()
                                               throws IOException
        Throws:
        IOException
      • testThreeBytesOneMissingWithMerging

        public void testThreeBytesOneMissingWithMerging()
                                                 throws IOException
        Throws:
        IOException
      • testThreads

        public void testThreads()
                         throws Exception
        Tests dv against stored fields with threads (binary/numeric/sorted, no missing)
        Throws:
        Exception
      • testThreads2

        @Nightly
        public void testThreads2()
                          throws Exception
        Tests dv against stored fields with threads (all types + missing)
        Throws:
        Exception
      • testEmptyBinaryValueOnPageSizes

        public void testEmptyBinaryValueOnPageSizes()
                                             throws Exception
        Throws:
        Exception
      • testOneSortedNumberOneMissing

        public void testOneSortedNumberOneMissing()
                                           throws IOException
        Throws:
        IOException
      • testNumberMergeAwayAllValues

        public void testNumberMergeAwayAllValues()
                                          throws IOException
        Throws:
        IOException
      • testTwoSortedNumberSameValue

        public void testTwoSortedNumberSameValue()
                                          throws IOException
        Throws:
        IOException
      • testTwoSortedNumberOneMissing

        public void testTwoSortedNumberOneMissing()
                                           throws IOException
        Throws:
        IOException
      • testSortedNumberMergeAwayAllValues

        public void testSortedNumberMergeAwayAllValues()
                                                throws IOException
        Throws:
        IOException
      • testSortedEnumAdvanceIndependently

        public void testSortedEnumAdvanceIndependently()
                                                throws IOException
        Throws:
        IOException
      • testSortedSetEnumAdvanceIndependently

        public void testSortedSetEnumAdvanceIndependently()
                                                   throws IOException
        Throws:
        IOException
      • testSortedMergeAwayAllValuesLargeSegment

        public void testSortedMergeAwayAllValuesLargeSegment()
                                                      throws IOException
        Throws:
        IOException
      • testSortedSetMergeAwayAllValuesLargeSegment

        public void testSortedSetMergeAwayAllValuesLargeSegment()
                                                         throws IOException
        Throws:
        IOException
      • testNumericMergeAwayAllValuesLargeSegment

        public void testNumericMergeAwayAllValuesLargeSegment()
                                                       throws IOException
        Throws:
        IOException
      • testSortedNumericMergeAwayAllValuesLargeSegment

        public void testSortedNumericMergeAwayAllValuesLargeSegment()
                                                             throws IOException
        Throws:
        IOException
      • testBinaryMergeAwayAllValuesLargeSegment

        public void testBinaryMergeAwayAllValuesLargeSegment()
                                                      throws IOException
        Throws:
        IOException
      • testHighOrdsSortedSetDV

        @Nightly
        public void testHighOrdsSortedSetDV()
                                     throws Exception
        Tests where a DVField uses a high number of packed bits to store its ords. See: https://issues.apache.org/jira/browse/LUCENE-10159
        Throws:
        Exception
      • codecAcceptsHugeBinaryValues

        protected boolean codecAcceptsHugeBinaryValues​(String field)
      • getCodec

        protected abstract Codec getCodec()
        Returns the codec to run tests against
      • getCreatedVersionMajor

        protected int getCreatedVersionMajor()
        Returns the major version that this codec is compatible with.
      • applyCreatedVersionMajor

        protected final <D extends Directory> D applyCreatedVersionMajor​(D d)
                                                                  throws IOException
        Set the created version of the given Directory and return it.
        Throws:
        IOException
      • excludedExtensionsFromByteCounts

        protected Collection<String> excludedExtensionsFromByteCounts()
        Return the list of extensions that should be excluded from byte counts when comparing indices that store the same content.
      • testMergeStability

        public void testMergeStability()
                                throws Exception
        The purpose of this test is to make sure that bulk merge doesn't accumulate useless data over runs.
        Throws:
        Exception
      • mergeIsStable

        protected boolean mergeIsStable()
      • testMultiClose

        public void testMultiClose()
                            throws IOException
        Calls close multiple times on closeable codec apis
        Throws:
        IOException
      • testRandomExceptions

        public void testRandomExceptions()
                                  throws Exception
        Tests exception handling on write and openInput/createOutput
        Throws:
        Exception
      • shouldTestMergeInstance

        protected boolean shouldTestMergeInstance()
        Returns false if only the regular fields reader should be tested, and true if only the merge instance should be tested.
      • testCheckIntegrityReadsAllBytes

        public void testCheckIntegrityReadsAllBytes()
                                             throws Exception
        This test is a best effort at verifying that checkIntegrity doesn't miss any files. It tests that the combination of opening a reader and calling checkIntegrity on it reads all bytes of all files.
        Throws:
        Exception