Package org.apache.lucene.benchmark.quality

Class QualityStats

java.lang.Object

org.apache.lucene.benchmark.quality.QualityStats

public class QualityStats
extends Object

Results of quality benchmark run for a single query or for a set of queries.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	QualityStats.RecallPoint	A certain rank in which a relevant doc was found.

Field Summary

Fields

Modifier and Type	Field	Description
static int	MAX_POINTS	Number of points for which precision is computed.

Constructor Summary

Constructors

Constructor	Description
QualityStats(double maxGoodPoints, long searchTime)	Construct a QualityStats object with anticipated maximal number of relevant hits.

Method Summary

Modifier and Type	Method	Description
void	addResult(int n, boolean isRelevant, long docNameExtractTime)	Add a (possibly relevant) doc.
static QualityStats	average(QualityStats[] stats)	Create a QualityStats object that is the average of the input QualityStats objects.
double	getAvp()	Return the average precision at recall points.
long	getDocNamesExtractTime()	Returns the time it took to extract doc names for judging the measured query, in milliseconds.
double	getMaxGoodPoints()	Returns the maximal number of good points.
double	getMRR()	Returns the Mean reciprocal rank over the queries or RR for a single query.
double	getNumGoodPoints()	Returns the number of good points (only relevant points).
double	getNumPoints()	Returns the number of points (both relevant and irrelevant points).
double	getPrecisionAt(int n)	Return the precision at rank n: |{relevant hits within first n hits}| / n .
double	getRecall()	Return the recall: |{relevant hits found}| / |{relevant hits existing}|.
QualityStats.RecallPoint[]	getRecallPoints()	Returns the recallPoints.
long	getSearchTime()	Returns the search time in milliseconds for the measured query.
void	log(String title, int paddLines, PrintWriter logger, String prefix)	Log information on this QualityStats object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_POINTS

public static final int MAX_POINTS

Number of points for which precision is computed.

See Also:

Constant Field Values

Constructor Detail

QualityStats

public QualityStats(double maxGoodPoints,
                    long searchTime)

Construct a QualityStats object with anticipated maximal number of relevant hits.

Parameters:

maxGoodPoints - maximal possible relevant hits.

Method Detail

addResult

public void addResult(int n,
                      boolean isRelevant,
                      long docNameExtractTime)

Add a (possibly relevant) doc.

Parameters:

n - rank of the added doc (its ordinal position within the query results).

isRelevant - true if the added doc is relevant, false otherwise.

getPrecisionAt

public double getPrecisionAt(int n)

Return the precision at rank n: |{relevant hits within first n hits}| / n .

Parameters:

n - requested precision point, must be at least 1 and at most MAX_POINTS.

getAvp

public double getAvp()

Return the average precision at recall points.

getRecall

public double getRecall()

Return the recall: |{relevant hits found}| / |{relevant hits existing}|.

log

public void log(String title,
                int paddLines,
                PrintWriter logger,
                String prefix)

Log information on this QualityStats object.

Parameters:

logger - Logger.

prefix - prefix before each log line.

average

public static QualityStats average(QualityStats[] stats)

Create a QualityStats object that is the average of the input QualityStats objects.

Parameters:

stats - array of input stats to be averaged.

Returns:

an average over the input stats.

getDocNamesExtractTime

public long getDocNamesExtractTime()

Returns the time it took to extract doc names for judging the measured query, in milliseconds.

getMaxGoodPoints

public double getMaxGoodPoints()

Returns the maximal number of good points. This is the number of relevant docs known by the judge for the measured query.

getNumGoodPoints

public double getNumGoodPoints()

Returns the number of good points (only relevant points).

getNumPoints

public double getNumPoints()

Returns the number of points (both relevant and irrelevant points).

getRecallPoints

public QualityStats.RecallPoint[] getRecallPoints()

Returns the recallPoints.

getMRR

public double getMRR()

Returns the Mean reciprocal rank over the queries or RR for a single query.

Reciprocal rank is defined as 1/r where r is the rank of the first correct result, or 0 if there are no correct results within the top 5 results.

This follows the definition in Question Answering - CNLP at the TREC-10 Question Answering Track.

getSearchTime

public long getSearchTime()

Returns the search time in milliseconds for the measured query.