Package org.apache.lucene.benchmark.quality

Class QualityStats

java.lang.Object
org.apache.lucene.benchmark.quality.QualityStats
public class QualityStats extends Object
Results of quality benchmark run for a single query or for a set of queries.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    QualityStats.RecallPoint
    A certain rank in which a relevant doc was found.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    MAX_POINTS
    Number of points for which precision is computed.

  • Constructor Summary

    Constructors
    Constructor
    Description
    QualityStats(double maxGoodPoints, long searchTime)
    Construct a QualityStats object with anticipated maximal number of relevant hits.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    addResult(int n, boolean isRelevant, long docNameExtractTime)
    Add a (possibly relevant) doc.
    static QualityStats
    average(QualityStats[] stats)
    Create a QualityStats object that is the average of the input QualityStats objects.
    double
    getAvp()
    Return the average precision at recall points.
    long
    getDocNamesExtractTime()
    Returns the time it took to extract doc names for judging the measured query, in milliseconds.
    double
    getMaxGoodPoints()
    Returns the maximal number of good points.
    double
    getMRR()
    Returns the Mean reciprocal rank over the queries or RR for a single query.
    double
    getNumGoodPoints()
    Returns the number of good points (only relevant points).
    double
    getNumPoints()
    Returns the number of points (both relevant and irrelevant points).
    double
    getPrecisionAt(int n)
    Return the precision at rank n: |{relevant hits within first n hits}| / n .
    double
    getRecall()
    Return the recall: |{relevant hits found}| / |{relevant hits existing}|.
    QualityStats.RecallPoint[]
    getRecallPoints()
    Returns the recallPoints.
    long
    getSearchTime()
    Returns the search time in milliseconds for the measured query.
    void
    log(String title, int paddLines, PrintWriter logger, String prefix)
    Log information on this QualityStats object.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • MAX_POINTS

      public static final int MAX_POINTS
      Number of points for which precision is computed.
      See Also:

  • Constructor Details

    • QualityStats

      public QualityStats(double maxGoodPoints, long searchTime)
      Construct a QualityStats object with anticipated maximal number of relevant hits.
      Parameters:
      maxGoodPoints - maximal possible relevant hits.

  • Method Details

    • addResult

      public void addResult(int n, boolean isRelevant, long docNameExtractTime)
      Add a (possibly relevant) doc.
      Parameters:
      n - rank of the added doc (its ordinal position within the query results).
      isRelevant - true if the added doc is relevant, false otherwise.

    • getPrecisionAt

      public double getPrecisionAt(int n)
      Return the precision at rank n: |{relevant hits within first n hits}| / n .
      Parameters:
      n - requested precision point, must be at least 1 and at most MAX_POINTS.

    • getAvp

      public double getAvp()
      Return the average precision at recall points.

    • getRecall

      public double getRecall()
      Return the recall: |{relevant hits found}| / |{relevant hits existing}|.

    • log

      public void log(String title, int paddLines, PrintWriter logger, String prefix)
      Log information on this QualityStats object.
      Parameters:
      logger - Logger.
      prefix - prefix before each log line.

    • average

      public static QualityStats average(QualityStats[] stats)
      Create a QualityStats object that is the average of the input QualityStats objects.
      Parameters:
      stats - array of input stats to be averaged.
      Returns:
      an average over the input stats.

    • getDocNamesExtractTime

      public long getDocNamesExtractTime()
      Returns the time it took to extract doc names for judging the measured query, in milliseconds.

    • getMaxGoodPoints

      public double getMaxGoodPoints()
      Returns the maximal number of good points. This is the number of relevant docs known by the judge for the measured query.

    • getNumGoodPoints

      public double getNumGoodPoints()
      Returns the number of good points (only relevant points).

    • getNumPoints

      public double getNumPoints()
      Returns the number of points (both relevant and irrelevant points).

    • getRecallPoints

      public QualityStats.RecallPoint[] getRecallPoints()
      Returns the recallPoints.

    • getMRR

      public double getMRR()
      Returns the Mean reciprocal rank over the queries or RR for a single query.

      Reciprocal rank is defined as 1/r where r is the rank of the first correct result, or 0 if there are no correct results within the top 5 results.

      This follows the definition in Question Answering - CNLP at the TREC-10 Question Answering Track.

    • getSearchTime

      public long getSearchTime()
      Returns the search time in milliseconds for the measured query.