public class SweetSpotSimilarity extends DefaultSimilarity
For lengthNorm, A global min/max can be specified to define the plateau of lengths that should all have a norm of 1.0. Below the min, and above the max the lengthNorm drops off in a sqrt function.
A per field min/max can be specified if different fields have different sweet spots.
For tf, baselineTf and hyperbolicTf functions are provided, which subclasses can choose between.
Similarity.ExactSimScorer, Similarity.SimWeight, Similarity.SloppySimScorer
discountOverlaps
Constructor and Description |
---|
SweetSpotSimilarity() |
Modifier and Type | Method and Description |
---|---|
float |
baselineTf(float freq)
Implemented as:
(x <= min) ? base : sqrt(x+(base**2)-min)
...but with a special case check for 0. |
float |
computeLengthNorm(int numTerms)
Implemented as:
1/sqrt( steepness * (abs(x-min) + abs(x-max) - (max-min)) + 1 )
. |
void |
computeNorm(FieldInvertState state,
Norm norm)
Implemented as
state.getBoost() *
computeLengthNorm(numTokens) where
numTokens does not count overlap tokens if
discountOverlaps is true by default or true for this
specific field. |
float |
hyperbolicTf(float freq)
Uses a hyperbolic tangent function that allows for a hard max...
|
void |
setBaselineTfFactors(float base,
float min)
Sets the baseline and minimum function variables for baselineTf
|
void |
setHyperbolicTfFactors(float min,
float max,
double base,
float xoffset)
Sets the function variables for the hyperbolicTf functions
|
void |
setLengthNormFactors(int min,
int max,
float steepness,
boolean discountOverlaps)
Sets the default function variables used by lengthNorm when no field
specific variables have been set.
|
float |
tf(int freq)
Delegates to baselineTf
|
coord, getDiscountOverlaps, idf, queryNorm, scorePayload, setDiscountOverlaps, sloppyFreq, tf, toString
computeWeight, decodeNormValue, encodeNormValue, exactSimScorer, idfExplain, idfExplain, sloppySimScorer
public void setBaselineTfFactors(float base, float min)
baselineTf(float)
public void setHyperbolicTfFactors(float min, float max, double base, float xoffset)
min
- the minimum tf value to ever be returned (default: 0.0)max
- the maximum tf value to ever be returned (default: 2.0)base
- the base value to be used in the exponential for the hyperbolic function (default: 1.3)xoffset
- the midpoint of the hyperbolic function (default: 10.0)hyperbolicTf(float)
public void setLengthNormFactors(int min, int max, float steepness, boolean discountOverlaps)
computeLengthNorm(int)
public void computeNorm(FieldInvertState state, Norm norm)
state.getBoost() *
computeLengthNorm(numTokens)
where
numTokens does not count overlap tokens if
discountOverlaps is true by default or true for this
specific field.computeNorm
in class DefaultSimilarity
public float computeLengthNorm(int numTerms)
1/sqrt( steepness * (abs(x-min) + abs(x-max) - (max-min)) + 1 )
.
This degrades to 1/sqrt(x)
when min and max are both 1 and
steepness is 0.5
:TODO: potential optimization is to just flat out return 1.0f if numTerms is between min and max.
public float tf(int freq)
tf
in class TFIDFSimilarity
baselineTf(float)
public float baselineTf(float freq)
(x <= min) ? base : sqrt(x+(base**2)-min)
...but with a special case check for 0.
This degrates to sqrt(x)
when min and base are both 0
public float hyperbolicTf(float freq)
tf(x)=min+(max-min)/2*(((base**(x-xoffset)-base**-(x-xoffset))/(base**(x-xoffset)+base**-(x-xoffset)))+1)
This code is provided as a convenience for subclasses that want to use a hyperbolic tf function.
Copyright © 2000-2012 Apache Software Foundation. All Rights Reserved.