Class DFRSimilarity
- java.lang.Object
-
- org.apache.lucene.search.similarities.Similarity
-
- org.apache.lucene.search.similarities.SimilarityBase
-
- org.apache.lucene.search.similarities.DFRSimilarity
-
public class DFRSimilarity extends SimilarityBase
Implements the divergence from randomness (DFR) framework introduced in Gianni Amati and Cornelis Joost Van Rijsbergen. 2002. Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. Inf. Syst. 20, 4 (October 2002), 357-389.The DFR scoring formula is composed of three separate components: the basic model, the aftereffect and an additional normalization component, represented by the classes
BasicModel
,AfterEffect
andNormalization
, respectively. The names of these classes were chosen to match the names of their counterparts in the Terrier IR engine.To construct a DFRSimilarity, you must specify the implementations for all three components of DFR:
BasicModel
: Basic model of information content:BasicModelG
: Geometric approximation of Bose-EinsteinBasicModelIn
: Inverse document frequencyBasicModelIne
: Inverse expected document frequency [mixture of Poisson and IDF]BasicModelIF
: Inverse term frequency [approximation of I(ne)]
AfterEffect
: First normalization of information gain:AfterEffectL
: Laplace's law of successionAfterEffectB
: Ratio of two Bernoulli processes
Normalization
: Second (length) normalization:NormalizationH1
: Uniform distribution of term frequencyNormalizationH2
: term frequency density inversely related to lengthNormalizationH3
: term frequency normalization provided by Dirichlet priorNormalizationZ
: term frequency normalization provided by a Zipfian relationNormalization.NoNormalization
: no second normalization
Note that qtf, the multiplicity of term-occurrence in the query, is not handled by this implementation.
Note that basic models BE (Limiting form of Bose-Einstein), P (Poisson approximation of the Binomial) and D (Divergence approximation of the Binomial) are not implemented because their formula couldn't be written in a way that makes scores non-decreasing with the normalized term frequency.
- See Also:
BasicModel
,AfterEffect
,Normalization
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity
Similarity.SimScorer
-
-
Field Summary
Fields Modifier and Type Field Description protected AfterEffect
afterEffect
The first normalization of the information content.protected BasicModel
basicModel
The basic model for information content.protected Normalization
normalization
The term frequency normalization.-
Fields inherited from class org.apache.lucene.search.similarities.SimilarityBase
discountOverlaps
-
-
Constructor Summary
Constructors Constructor Description DFRSimilarity(BasicModel basicModel, AfterEffect afterEffect, Normalization normalization)
Creates DFRSimilarity from the three components.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected void
explain(List<Explanation> subs, BasicStats stats, double freq, double docLen)
Subclasses should implement this method to explain the score.protected Explanation
explain(BasicStats stats, Explanation freq, double docLen)
Explains the score.AfterEffect
getAfterEffect()
Returns the first normalizationBasicModel
getBasicModel()
Returns the basic model of information contentNormalization
getNormalization()
Returns the second normalizationprotected double
score(BasicStats stats, double freq, double docLen)
Scores the documentdoc
.String
toString()
Subclasses must override this method to return the name of the Similarity and preferably the values of parameters (if any) as well.-
Methods inherited from class org.apache.lucene.search.similarities.SimilarityBase
computeNorm, fillBasicStats, getDiscountOverlaps, log2, newStats, scorer, setDiscountOverlaps
-
-
-
-
Field Detail
-
basicModel
protected final BasicModel basicModel
The basic model for information content.
-
afterEffect
protected final AfterEffect afterEffect
The first normalization of the information content.
-
normalization
protected final Normalization normalization
The term frequency normalization.
-
-
Constructor Detail
-
DFRSimilarity
public DFRSimilarity(BasicModel basicModel, AfterEffect afterEffect, Normalization normalization)
Creates DFRSimilarity from the three components.Note that
null
values are not allowed: if you want no normalization, instead passNormalization.NoNormalization
.- Parameters:
basicModel
- Basic model of information contentafterEffect
- First normalization of information gainnormalization
- Second (length) normalization
-
-
Method Detail
-
score
protected double score(BasicStats stats, double freq, double docLen)
Description copied from class:SimilarityBase
Scores the documentdoc
.Subclasses must apply their scoring formula in this class.
- Specified by:
score
in classSimilarityBase
- Parameters:
stats
- the corpus level statistics.freq
- the term frequency.docLen
- the document length.- Returns:
- the score.
-
explain
protected void explain(List<Explanation> subs, BasicStats stats, double freq, double docLen)
Description copied from class:SimilarityBase
Subclasses should implement this method to explain the score.expl
already contains the score, the name of the class and the doc id, as well as the term frequency and its explanation; subclasses can add additional clauses to explain details of their scoring formulae.The default implementation does nothing.
- Overrides:
explain
in classSimilarityBase
- Parameters:
subs
- the list of details of the explanation to extendstats
- the corpus level statistics.freq
- the term frequency.docLen
- the document length.
-
explain
protected Explanation explain(BasicStats stats, Explanation freq, double docLen)
Description copied from class:SimilarityBase
Explains the score. The implementation here provides a basic explanation in the format score(name-of-similarity, doc=doc-id, freq=term-frequency), computed from:, and attaches the score (computed via theSimilarityBase.score(BasicStats, double, double)
method) and the explanation for the term frequency. Subclasses content with this format may add additional details inSimilarityBase.explain(List, BasicStats, double, double)
.- Overrides:
explain
in classSimilarityBase
- Parameters:
stats
- the corpus level statistics.freq
- the term frequency and its explanation.docLen
- the document length.- Returns:
- the explanation.
-
toString
public String toString()
Description copied from class:SimilarityBase
Subclasses must override this method to return the name of the Similarity and preferably the values of parameters (if any) as well.- Specified by:
toString
in classSimilarityBase
-
getBasicModel
public BasicModel getBasicModel()
Returns the basic model of information content
-
getAfterEffect
public AfterEffect getAfterEffect()
Returns the first normalization
-
getNormalization
public Normalization getNormalization()
Returns the second normalization
-
-