Class QualityStats

  • public class QualityStats
    extends Object
    Results of quality benchmark run for a single query or for a set of queries.
    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  QualityStats.RecallPoint
      A certain rank in which a relevant doc was found.
    • Field Summary

      Modifier and Type Field Description
      static int MAX_POINTS
      Number of points for which precision is computed.
    • Constructor Summary

      Constructor Description
      QualityStats​(double maxGoodPoints, long searchTime)
      Construct a QualityStats object with anticipated maximal number of relevant hits.
    • Field Detail

      • MAX_POINTS

        public static final int MAX_POINTS
        Number of points for which precision is computed.
        See Also:
        Constant Field Values
    • Constructor Detail

      • QualityStats

        public QualityStats​(double maxGoodPoints,
                            long searchTime)
        Construct a QualityStats object with anticipated maximal number of relevant hits.
        maxGoodPoints - maximal possible relevant hits.
    • Method Detail

      • addResult

        public void addResult​(int n,
                              boolean isRelevant,
                              long docNameExtractTime)
        Add a (possibly relevant) doc.
        n - rank of the added doc (its ordinal position within the query results).
        isRelevant - true if the added doc is relevant, false otherwise.
      • getPrecisionAt

        public double getPrecisionAt​(int n)
        Return the precision at rank n: |{relevant hits within first n hits}| / n .
        n - requested precision point, must be at least 1 and at most MAX_POINTS.
      • getAvp

        public double getAvp()
        Return the average precision at recall points.
      • getRecall

        public double getRecall()
        Return the recall: |{relevant hits found}| / |{relevant hits existing}|.
      • log

        public void log​(String title,
                        int paddLines,
                        PrintWriter logger,
                        String prefix)
        Log information on this QualityStats object.
        logger - Logger.
        prefix - prefix before each log line.
      • average

        public static QualityStats average​(QualityStats[] stats)
        Create a QualityStats object that is the average of the input QualityStats objects.
        stats - array of input stats to be averaged.
        an average over the input stats.
      • getDocNamesExtractTime

        public long getDocNamesExtractTime()
        Returns the time it took to extract doc names for judging the measured query, in milliseconds.
      • getMaxGoodPoints

        public double getMaxGoodPoints()
        Returns the maximal number of good points. This is the number of relevant docs known by the judge for the measured query.
      • getNumGoodPoints

        public double getNumGoodPoints()
        Returns the number of good points (only relevant points).
      • getNumPoints

        public double getNumPoints()
        Returns the number of points (both relevant and irrelevant points).
      • getMRR

        public double getMRR()
        Returns the Mean reciprocal rank over the queries or RR for a single query.

        Reciprocal rank is defined as 1/r where r is the rank of the first correct result, or 0 if there are no correct results within the top 5 results.

        This follows the definition in Question Answering - CNLP at the TREC-10 Question Answering Track.

      • getSearchTime

        public long getSearchTime()
        Returns the search time in milliseconds for the measured query.