BM25Similarity (Lucene 7.0.1 API)

java.lang.Object
- org.apache.lucene.search.similarities.Similarity
- - org.apache.lucene.search.similarities.BM25Similarity

```
public class BM25Similarity
extends Similarity
```
BM25 Similarity. Introduced in Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. Okapi at TREC-3. In Proceedings of the Third Text REtrieval Conference (TREC 1994). Gaithersburg, USA, November 1994.

Nested Class Summary
- Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity
  Similarity.SimScorer, Similarity.SimWeight

Field Summary

Fields
Modifier and Type	Field and Description
`protected boolean`	`discountOverlaps` True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.

Constructor Summary

Constructors
Constructor and Description
`BM25Similarity()` BM25 with these default values: `k1 = 1.2` `b = 0.75`
`BM25Similarity(float k1, float b)` BM25 with the supplied parameter values.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected float`	`avgFieldLength(CollectionStatistics collectionStats)` The default implementation computes the average as `sumTotalTermFreq / docCount`, or returns `1` if the index does not store sumTotalTermFreq: any field that omits frequency information).
`long`	`computeNorm(FieldInvertState state)` Computes the normalization value for a field, given the accumulated state of term processing for this field (see `FieldInvertState`).
`Similarity.SimWeight`	`computeWeight(float boost, CollectionStatistics collectionStats, TermStatistics... termStats)` Compute any collection-level weight (e.g.
`float`	`getB()` Returns the `b` parameter
`boolean`	`getDiscountOverlaps()` Returns true if overlap tokens are discounted from the document's length.
`float`	`getK1()` Returns the `k1` parameter
`protected float`	`idf(long docFreq, long docCount)` Implemented as `log(1 + (docCount - docFreq + 0.5)/(docFreq + 0.5))`.
`Explanation`	`idfExplain(CollectionStatistics collectionStats, TermStatistics termStats)` Computes a score factor for a simple term and returns an explanation for that score factor.
`Explanation`	`idfExplain(CollectionStatistics collectionStats, TermStatistics[] termStats)` Computes a score factor for a phrase.
`protected float`	`scorePayload(int doc, int start, int end, BytesRef payload)` The default implementation returns `1`
`void`	`setDiscountOverlaps(boolean v)` Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm.
`Similarity.SimScorer`	`simScorer(Similarity.SimWeight stats, LeafReaderContext context)` Creates a new `Similarity.SimScorer` to score matching documents from a segment of the inverted index.
`protected float`	`sloppyFreq(int distance)` Implemented as `1 / (distance + 1)`.
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - discountOverlaps
```
protected boolean discountOverlaps
```
    True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.
- Constructor Detail
  - BM25Similarity
```
public BM25Similarity(float k1,
                      float b)
```
    BM25 with the supplied parameter values.
    
    Parameters:
    
    k1 - Controls non-linear term frequency normalization (saturation).
    
    b - Controls to what degree document length normalizes tf values.
    
    Throws:
    
    IllegalArgumentException - if k1 is infinite or negative, or if b is not within the range [0..1]
  - BM25Similarity
```
public BM25Similarity()
```
    BM25 with these default values:
    - k1 = 1.2
    - b = 0.75
- Method Detail
  - idf
```
protected float idf(long docFreq,
                    long docCount)
```
    Implemented as log(1 + (docCount - docFreq + 0.5)/(docFreq + 0.5)).
  - sloppyFreq
```
protected float sloppyFreq(int distance)
```
    Implemented as 1 / (distance + 1).
  - scorePayload
```
protected float scorePayload(int doc,
                             int start,
                             int end,
                             BytesRef payload)
```
    The default implementation returns 1
  - avgFieldLength
```
protected float avgFieldLength(CollectionStatistics collectionStats)
```
    The default implementation computes the average as sumTotalTermFreq / docCount, or returns 1 if the index does not store sumTotalTermFreq: any field that omits frequency information).
  - setDiscountOverlaps
```
public void setDiscountOverlaps(boolean v)
```
    Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm. By default this is true, meaning overlap tokens do not count when computing norms.
  - getDiscountOverlaps
```
public boolean getDiscountOverlaps()
```
    Returns true if overlap tokens are discounted from the document's length.
    
    See Also:
    
    setDiscountOverlaps(boolean)
  - computeNorm
```
public final long computeNorm(FieldInvertState state)
```
    Description copied from class: Similarity
    
    Computes the normalization value for a field, given the accumulated state of term processing for this field (see FieldInvertState).
    Matches in longer fields are less precise, so implementations of this method usually set smaller values when state.getLength() is large, and larger values when state.getLength() is small.
    
    Specified by:
    
    computeNorm in class Similarity
    
    Parameters:
    
    state - current processing state for this field
    
    Returns:
    
    computed norm value
  - idfExplain
```
public Explanation idfExplain(CollectionStatistics collectionStats,
                              TermStatistics termStats)
```
    Computes a score factor for a simple term and returns an explanation for that score factor.
    The default implementation uses:
```
 idf(docFreq, docCount);
 
```
    Note that CollectionStatistics.docCount() is used instead of IndexReader#numDocs() because also TermStatistics.docFreq() is used, and when the latter is inaccurate, so is CollectionStatistics.docCount(), and in the same direction. In addition, CollectionStatistics.docCount() does not skew when fields are sparse.
    Parameters:
    
    collectionStats - collection-level statistics
    
    termStats - term-level statistics for the term
    
    Returns:
    
    an Explain object that includes both an idf score factor and an explanation for the term.
  - idfExplain
```
public Explanation idfExplain(CollectionStatistics collectionStats,
                              TermStatistics[] termStats)
```
    Computes a score factor for a phrase.
    The default implementation sums the idf factor for each term in the phrase.
    
    Parameters:
    
    collectionStats - collection-level statistics
    
    termStats - term-level statistics for the terms in the phrase
    
    Returns:
    
    an Explain object that includes both an idf score factor for the phrase and an explanation for each term.
  - computeWeight
```
public final Similarity.SimWeight computeWeight(float boost,
                                                CollectionStatistics collectionStats,
                                                TermStatistics... termStats)
```
    Description copied from class: Similarity
    
    Compute any collection-level weight (e.g. IDF, average document length, etc) needed for scoring a query.
    
    Specified by:
    
    computeWeight in class Similarity
    
    Parameters:
    
    boost - a multiplicative factor to apply to the produces scores
    
    collectionStats - collection-level statistics, such as the number of tokens in the collection.
    
    termStats - term-level statistics, such as the document frequency of a term across the collection.
    
    Returns:
    
    SimWeight object with the information this Similarity needs to score a query.
  - simScorer
```
public final Similarity.SimScorer simScorer(Similarity.SimWeight stats,
                                            LeafReaderContext context)
                                     throws IOException
```
    Description copied from class: Similarity
    
    Creates a new Similarity.SimScorer to score matching documents from a segment of the inverted index.
    
    Specified by:
    
    simScorer in class Similarity
    
    Parameters:
    
    stats - collection information from Similarity.computeWeight(float, CollectionStatistics, TermStatistics...)
    
    context - segment of the inverted index to be scored.
    
    Returns:
    
    SloppySimScorer for scoring documents across context
    
    Throws:
    
    IOException - if there is a low-level I/O error
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - getK1
```
public final float getK1()
```
    Returns the k1 parameter
    
    See Also:
    
    BM25Similarity(float, float)
  - getB
```
public final float getB()
```
    Returns the b parameter
    
    See Also:
    
    BM25Similarity(float, float)

Class BM25Similarity

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

discountOverlaps

Constructor Detail

BM25Similarity

BM25Similarity

Method Detail

idf

sloppyFreq

scorePayload

avgFieldLength

setDiscountOverlaps

getDiscountOverlaps

computeNorm

idfExplain

idfExplain

computeWeight

simScorer

toString

getK1

getB