org.apache.lucene.search.similarities
Class LMDirichletSimilarity

java.lang.Object
  extended by org.apache.lucene.search.similarities.Similarity
      extended by org.apache.lucene.search.similarities.SimilarityBase
          extended by org.apache.lucene.search.similarities.LMSimilarity
              extended by org.apache.lucene.search.similarities.LMDirichletSimilarity

public class LMDirichletSimilarity
extends LMSimilarity

Bayesian smoothing using Dirichlet priors. From Chengxiang Zhai and John Lafferty. 2001. A study of smoothing methods for language models applied to Ad Hoc information retrieval. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '01). ACM, New York, NY, USA, 334-342.

The formula as defined the paper assigns a negative score to documents that contain the term, but with fewer occurrences than predicted by the collection language model. The Lucene implementation returns 0 for such documents.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.LMSimilarity
LMSimilarity.CollectionModel, LMSimilarity.DefaultCollectionModel, LMSimilarity.LMStats
 
Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity
Similarity.ExactSimScorer, Similarity.SimWeight, Similarity.SloppySimScorer
 
Field Summary
 
Fields inherited from class org.apache.lucene.search.similarities.LMSimilarity
collectionModel
 
Fields inherited from class org.apache.lucene.search.similarities.SimilarityBase
discountOverlaps
 
Constructor Summary
LMDirichletSimilarity()
          Instantiates the similarity with the default μ value of 2000.
LMDirichletSimilarity(float mu)
          Instantiates the similarity with the provided μ parameter.
LMDirichletSimilarity(LMSimilarity.CollectionModel collectionModel)
          Instantiates the similarity with the default μ value of 2000.
LMDirichletSimilarity(LMSimilarity.CollectionModel collectionModel, float mu)
          Instantiates the similarity with the provided μ parameter.
 
Method Summary
protected  void explain(Explanation expl, BasicStats stats, int doc, float freq, float docLen)
          Subclasses should implement this method to explain the score.
 float getMu()
          Returns the μ parameter.
 String getName()
          Returns the name of the LM method.
protected  float score(BasicStats stats, float freq, float docLen)
          Scores the document doc.
 
Methods inherited from class org.apache.lucene.search.similarities.LMSimilarity
fillBasicStats, newStats, toString
 
Methods inherited from class org.apache.lucene.search.similarities.SimilarityBase
computeNorm, computeWeight, decodeNormValue, encodeNormValue, exactSimScorer, explain, getDiscountOverlaps, log2, setDiscountOverlaps, sloppySimScorer
 
Methods inherited from class org.apache.lucene.search.similarities.Similarity
coord, queryNorm
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

LMDirichletSimilarity

public LMDirichletSimilarity(LMSimilarity.CollectionModel collectionModel,
                             float mu)
Instantiates the similarity with the provided μ parameter.


LMDirichletSimilarity

public LMDirichletSimilarity(float mu)
Instantiates the similarity with the provided μ parameter.


LMDirichletSimilarity

public LMDirichletSimilarity(LMSimilarity.CollectionModel collectionModel)
Instantiates the similarity with the default μ value of 2000.


LMDirichletSimilarity

public LMDirichletSimilarity()
Instantiates the similarity with the default μ value of 2000.

Method Detail

score

protected float score(BasicStats stats,
                      float freq,
                      float docLen)
Description copied from class: SimilarityBase
Scores the document doc.

Subclasses must apply their scoring formula in this class.

Specified by:
score in class SimilarityBase
Parameters:
stats - the corpus level statistics.
freq - the term frequency.
docLen - the document length.
Returns:
the score.

explain

protected void explain(Explanation expl,
                       BasicStats stats,
                       int doc,
                       float freq,
                       float docLen)
Description copied from class: SimilarityBase
Subclasses should implement this method to explain the score. expl already contains the score, the name of the class and the doc id, as well as the term frequency and its explanation; subclasses can add additional clauses to explain details of their scoring formulae.

The default implementation does nothing.

Overrides:
explain in class LMSimilarity
Parameters:
expl - the explanation to extend with details.
stats - the corpus level statistics.
doc - the document id.
freq - the term frequency.
docLen - the document length.

getMu

public float getMu()
Returns the μ parameter.


getName

public String getName()
Description copied from class: LMSimilarity
Returns the name of the LM method. The values of the parameters should be included as well.

Used in LMSimilarity.toString()

.

Specified by:
getName in class LMSimilarity


Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.