Class TrecJudge

  • All Implemented Interfaces:

    public class TrecJudge
    extends Object
    implements Judge
    Judge if given document is relevant to given quality query, based on Trec format for judgements.
    • Constructor Detail

      • TrecJudge

        public TrecJudge​(BufferedReader reader)
                  throws IOException
        Constructor from a reader.

        Expected input format:

             qnum  0   doc-name     is-relevant
        Two sample lines:
             19    0   doc303       1
             19    0   doc7295      0
        reader - where judgements are read from.
        IOException - If there is a low-level I/O error.
    • Method Detail

      • isRelevant

        public boolean isRelevant​(String docName,
                                  QualityQuery query)
        Description copied from interface: Judge
        Judge if document docName is relevant for the given quality query.
        Specified by:
        isRelevant in interface Judge
        docName - name of doc tested for relevancy.
        query - tested quality query.
        true if relevant, false if not.
      • validateData

        public boolean validateData​(QualityQuery[] qq,
                                    PrintWriter logger)
        Description copied from interface: Judge
        Validate that queries and this Judge match each other. To be perfectly valid, this Judge must have some data for each and every input quality query, and must not have any data on any other quality query. Note: the quality benchmark run would not fail in case of imperfect validity, just a warning message would be logged.
        Specified by:
        validateData in interface Judge
        qq - quality queries to be validated.
        logger - if not null, validation issues are logged.
        true if perfectly valid, false if not.
      • maxRecall

        public int maxRecall​(QualityQuery query)
        Description copied from interface: Judge
        Return the maximal recall for the input quality query. It is the number of relevant docs this Judge "knows" for the query.
        Specified by:
        maxRecall in interface Judge
        query - the query whose maximal recall is needed.