Package org.apache.lucene.search.similarities

Class LMJelinekMercerSimilarity

java.lang.Object

org.apache.lucene.search.similarities.Similarity

org.apache.lucene.search.similarities.SimilarityBase

org.apache.lucene.search.similarities.LMSimilarity

org.apache.lucene.search.similarities.LMJelinekMercerSimilarity

public class LMJelinekMercerSimilarity
extends LMSimilarity

Language model based on the Jelinek-Mercer smoothing method. From Chengxiang Zhai and John Lafferty. 2001. A study of smoothing methods for language models applied to Ad Hoc information retrieval. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '01). ACM, New York, NY, USA, 334-342.

The model has a single parameter, λ. According to said paper, the optimal value depends on both the collection and the query. The optimal value is around 0.1 for title queries and 0.7 for long queries.

Values should be between 0 (exclusive) and 1 (inclusive). Values near zero act score more like a conjunction (coordinate level matching), whereas values near 1 behave the opposite (more like pure disjunction).

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.LMSimilarity

LMSimilarity.CollectionModel, LMSimilarity.DefaultCollectionModel, LMSimilarity.LMStats

Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity

Similarity.SimScorer

Field Summary

Fields inherited from class org.apache.lucene.search.similarities.LMSimilarity

collectionModel

Constructor Summary

Constructors

Constructor	Description
LMJelinekMercerSimilarity(float lambda)	Instantiates with the specified λ parameter.
LMJelinekMercerSimilarity(LMSimilarity.CollectionModel collectionModel, boolean discountOverlaps, float lambda)	Instantiates with the specified collectionModel and parameters.
LMJelinekMercerSimilarity(LMSimilarity.CollectionModel collectionModel, float lambda)	Instantiates with the specified collectionModel and λ parameter.

Method Summary

Modifier and Type	Method	Description
protected void	explain(List<Explanation> subs, BasicStats stats, double freq, double docLen)	Subclasses should implement this method to explain the score.
protected Explanation	explain(BasicStats stats, Explanation freq, double docLen)	Explains the score.
float	getLambda()	Returns the λ parameter.
String	getName()	Returns the name of the LM method.
protected double	score(BasicStats stats, double freq, double docLen)	Scores the document doc.

Methods inherited from class org.apache.lucene.search.similarities.LMSimilarity

fillBasicStats, newStats, toString

Methods inherited from class org.apache.lucene.search.similarities.SimilarityBase

log2, scorer

Methods inherited from class org.apache.lucene.search.similarities.Similarity

computeNorm, getDiscountOverlaps

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LMJelinekMercerSimilarity

public LMJelinekMercerSimilarity(LMSimilarity.CollectionModel collectionModel,
                                 float lambda)

Instantiates with the specified collectionModel and λ parameter.

LMJelinekMercerSimilarity

public LMJelinekMercerSimilarity(LMSimilarity.CollectionModel collectionModel,
                                 boolean discountOverlaps,
                                 float lambda)

Instantiates with the specified collectionModel and parameters.

LMJelinekMercerSimilarity

public LMJelinekMercerSimilarity(float lambda)

Instantiates with the specified λ parameter.

Method Detail

score

protected double score(BasicStats stats,
                       double freq,
                       double docLen)

Description copied from class: SimilarityBase

Scores the document doc.

Subclasses must apply their scoring formula in this class.

Specified by:

score in class SimilarityBase

Parameters:

stats - the corpus level statistics.

freq - the term frequency.

docLen - the document length.

Returns:

the score.

explain

protected void explain(List<Explanation> subs,
                       BasicStats stats,
                       double freq,
                       double docLen)

Description copied from class: SimilarityBase

Subclasses should implement this method to explain the score. expl already contains the score, the name of the class and the doc id, as well as the term frequency and its explanation; subclasses can add additional clauses to explain details of their scoring formulae.

The default implementation does nothing.

Overrides:

explain in class LMSimilarity

Parameters:

subs - the list of details of the explanation to extend

stats - the corpus level statistics.

freq - the term frequency.

docLen - the document length.

explain

protected Explanation explain(BasicStats stats,
                              Explanation freq,
                              double docLen)

Description copied from class: SimilarityBase

Explains the score. The implementation here provides a basic explanation in the format score(name-of-similarity, doc=doc-id, freq=term-frequency), computed from:, and attaches the score (computed via the SimilarityBase.score(BasicStats, double, double) method) and the explanation for the term frequency. Subclasses content with this format may add additional details in SimilarityBase.explain(List, BasicStats, double, double).

Overrides:

explain in class SimilarityBase

Parameters:

stats - the corpus level statistics.

freq - the term frequency and its explanation.

docLen - the document length.

Returns:

the explanation.

getLambda

public float getLambda()

Returns the λ parameter.

getName

public String getName()

Description copied from class: LMSimilarity

Returns the name of the LM method. The values of the parameters should be included as well.

Used in LMSimilarity.toString().

Specified by:

getName in class LMSimilarity