Skip navigation links

Package org.apache.lucene.search.similarities

This package contains the various ranking models that can be used in Lucene.

See: Description

Package org.apache.lucene.search.similarities Description

This package contains the various ranking models that can be used in Lucene. The abstract class Similarity serves as the base for ranking functions. For searching, users can employ the models already implemented or create their own by extending one of the classes in this package.

Table Of Contents

  1. Summary of the Ranking Methods
  2. Changing the Similarity

Summary of the Ranking Methods

BM25Similarity is an optimized implementation of the successful Okapi BM25 model.

ClassicSimilarity is the original Lucene scoring function. It is based on the Vector Space Model. For more information, see TFIDFSimilarity.

SimilarityBase provides a basic implementation of the Similarity contract and exposes a highly simplified interface, which makes it an ideal starting point for new ranking functions. Lucene ships the following methods built on SimilarityBase:

Since SimilarityBase is not optimized to the same extent as ClassicSimilarity and BM25Similarity, a difference in performance is to be expected when using the methods listed above. However, optimizations can always be implemented in subclasses; see below.

Changing Similarity

Chances are the available Similarities are sufficient for all your searching needs. However, in some applications it may be necessary to customize your Similarity implementation. For instance, some applications do not need to distinguish between shorter and longer documents and could set BM25's b parameter to 0.

To change Similarity, one must do so for both indexing and searching, and the changes must happen before either of these actions take place. Although in theory there is nothing stopping you from changing mid-stream, it just isn't well-defined what is going to happen.

To make this change, implement your own Similarity (likely you'll want to simply subclass SimilarityBase), and then register the new class by calling IndexWriterConfig.setSimilarity(Similarity) before indexing and IndexSearcher.setSimilarity(Similarity) before searching.

Tuning BM25Similarity

BM25Similarity has two parameters that may be tuned:

Extending SimilarityBase

The easiest way to quickly implement a new ranking method is to extend SimilarityBase, which provides basic implementations for the low level . Subclasses are only required to implement the SimilarityBase.score(BasicStats, double, double) and SimilarityBase.toString() methods.

Another option is to extend one of the frameworks based on SimilarityBase. These Similarities are implemented modularly, e.g. DFRSimilarity delegates computation of the three parts of its formula to the classes BasicModel, AfterEffect and Normalization. Instead of subclassing the Similarity, one can simply introduce a new basic model and tell DFRSimilarity to use it.

Skip navigation links

Copyright © 2000-2021 Apache Software Foundation. All Rights Reserved.