org.apache.lucene.search.similarities

Class ClassicSimilarity

java.lang.Object

org.apache.lucene.search.similarities.Similarity

org.apache.lucene.search.similarities.TFIDFSimilarity

org.apache.lucene.search.similarities.ClassicSimilarity

public class ClassicSimilarity
extends TFIDFSimilarity

Expert: Historical scoring implementation. You might want to consider using BM25Similarity instead, which is generally considered superior to TF-IDF.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity

Similarity.SimScorer, Similarity.SimWeight

Field Summary

Fields inherited from class org.apache.lucene.search.similarities.TFIDFSimilarity

discountOverlaps

Constructor Summary

Constructors

Constructor and Description
ClassicSimilarity() Sole constructor: parameter-free

Method Summary

Modifier and Type	Method and Description
float	idf(long docFreq, long docCount) Implemented as log((docCount+1)/(docFreq+1)) + 1.
Explanation	idfExplain(CollectionStatistics collectionStats, TermStatistics termStats) Computes a score factor for a simple term and returns an explanation for that score factor.
float	lengthNorm(int numTerms) Implemented as 1/sqrt(length).
float	scorePayload(int doc, int start, int end, BytesRef payload) The default implementation returns 1
float	sloppyFreq(int distance) Implemented as 1 / (distance + 1).
float	tf(float freq) Implemented as sqrt(freq).
String	toString()

Methods inherited from class org.apache.lucene.search.similarities.TFIDFSimilarity

computeNorm, computeWeight, getDiscountOverlaps, idfExplain, setDiscountOverlaps, simScorer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ClassicSimilarity

public ClassicSimilarity()

Sole constructor: parameter-free

Method Detail

lengthNorm

public float lengthNorm(int numTerms)

Implemented as 1/sqrt(length).

Specified by:

lengthNorm in class TFIDFSimilarity

Parameters:

numTerms - the number of terms in the field, optionally discounting overlaps

Returns:

a length normalization value

WARNING: This API is experimental and might change in incompatible ways in the next release.

tf

public float tf(float freq)

Implemented as sqrt(freq).

Specified by:

tf in class TFIDFSimilarity

Parameters:

freq - the frequency of a term within a document

Returns:

a score factor based on a term's within-document frequency

sloppyFreq

public float sloppyFreq(int distance)

Implemented as 1 / (distance + 1).

Specified by:

sloppyFreq in class TFIDFSimilarity

Parameters:

distance - the edit distance of this sloppy phrase match

Returns:

the frequency increment for this match

See Also:

PhraseQuery.getSlop()

scorePayload

public float scorePayload(int doc,
                          int start,
                          int end,
                          BytesRef payload)

The default implementation returns 1

Specified by:

scorePayload in class TFIDFSimilarity

Parameters:

doc - The docId currently being scored.

start - The start position of the payload

end - The end position of the payload

payload - The payload byte array to be scored

Returns:

An implementation dependent float to be used as a scoring factor

idfExplain

public Explanation idfExplain(CollectionStatistics collectionStats,
                              TermStatistics termStats)

Description copied from class: TFIDFSimilarity

Computes a score factor for a simple term and returns an explanation for that score factor.

The default implementation uses:

 idf(docFreq, docCount);
 

Note that CollectionStatistics.docCount() is used instead of IndexReader#numDocs() because also TermStatistics.docFreq() is used, and when the latter is inaccurate, so is CollectionStatistics.docCount(), and in the same direction. In addition, CollectionStatistics.docCount() does not skew when fields are sparse.

Overrides:

idfExplain in class TFIDFSimilarity

Parameters:

collectionStats - collection-level statistics

termStats - term-level statistics for the term

Returns:

an Explain object that includes both an idf score factor and an explanation for the term.

idf

public float idf(long docFreq,
                 long docCount)

Implemented as log((docCount+1)/(docFreq+1)) + 1.

Specified by:

idf in class TFIDFSimilarity

Parameters:

docFreq - the number of documents which contain the term

docCount - the total number of documents in the collection

Returns:

a score factor based on the term's document frequency

toString

public String toString()

Overrides:

toString in class Object