| Interface | Description |
|---|---|
| Judge |
Judge if a document is relevant for a quality query.
|
| QualityQueryParser |
Parse a QualityQuery into a Lucene query.
|
| Class | Description |
|---|---|
| QualityBenchmark |
Main entry point for running a quality benchmark.
|
| QualityQuery |
A QualityQuery has an ID and some name-value pairs.
|
| QualityStats |
Results of quality benchmark run for a single query or for a set of queries.
|
| QualityStats.RecallPoint |
A certain rank in which a relevant doc was found.
|
This package allows to benchmark search quality of a Lucene application.
In order to use this package you should provide:
For benchmarking TREC collections with TREC QRels, take a look at the trec package.
Here is a sample code used to run the TREC 2006 queries 701-850 on the .Gov2 collection:
File topicsFile = new File("topics-701-850.txt");
File qrelsFile = new File("qrels-701-850.txt");
Searcher searcher = new IndexSearcher("index");
int maxResults = 1000;
String docNameField = "docname";
PrintWriter logger = new PrintWriter(System.out,true);
// use trec utilities to read trec topics into quality queries
TrecTopicsReader qReader = new TrecTopicsReader();
QualityQuery qqs[] = qReader.readQueries(new BufferedReader(new FileReader(topicsFile)));
// prepare judge, with trec utilities that read from a QRels file
Judge judge = new TrecJudge(new BufferedReader(new FileReader(qrelsFile)));
// validate topics & judgments match each other
judge.validateData(qqs, logger);
// set the parsing of quality queries into Lucene queries.
QualityQueryParser qqParser = new SimpleQQParser("title", "body");
// run the benchmark
QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher, docNameField);
SubmissionReport submitLog = null;
QualityStats stats[] = qrun.execute(maxResults, judge, submitLog, logger);
// print an avarage sum of the results
QualityStats avg = QualityStats.average(stats);
avg.log("SUMMARY",2,logger, " ");
Some immediate ways to modify this program to your needs are: