| Interface | Description |
|---|---|
| Encoder |
Encodes original text.
|
| Formatter |
Processes terms found in the original text, typically by applying some form
of mark-up to highlight terms in HTML search results pages.
|
| Fragmenter |
Implements the policy for breaking text into multiple fragments for
consideration by the
Highlighter class. |
| Scorer |
A Scorer is responsible for scoring a stream of tokens.
|
| Class | Description |
|---|---|
| DefaultEncoder |
Simple
Encoder implementation that does not modify the output |
| GradientFormatter |
Formats text with different color intensity depending on the score of the
term.
|
| Highlighter |
Marks up highlighted terms found in the best sections of
text, using configurable
Fragmenter, Scorer, Formatter,
Encoder and tokenizers. |
| NullFragmenter |
Fragmenter implementation which does not fragment the text. |
| OffsetLimitTokenFilter |
This TokenFilter limits the number of tokens while indexing by adding up the
current offset.
|
| PositionSpan |
Utility class to record Positions Spans
|
| QueryScorer |
Scorer implementation which scores text fragments by the number of
unique query terms found. |
| QueryTermExtractor |
Utility class used to extract the terms used in a query, plus any weights.
|
| QueryTermScorer |
Scorer implementation which scores text fragments by the number of
unique query terms found. |
| SimpleFragmenter |
Fragmenter implementation which breaks text up into same-size
fragments with no concerns over spotting sentence boundaries. |
| SimpleHTMLEncoder |
Simple
Encoder implementation to escape text for HTML output |
| SimpleHTMLFormatter |
Simple
Formatter implementation to highlight terms with a pre and
post tag. |
| SimpleSpanFragmenter |
Fragmenter implementation which breaks text up into same-size
fragments but does not split up Spans. |
| SpanGradientFormatter |
Formats text with different color intensity depending on the score of the
term using the span tag.
|
| TermVectorLeafReader |
Wraps a Terms with a
LeafReader, typically from term vectors. |
| TextFragment |
Low-level class used to record information about a section of a document
with a score.
|
| TokenGroup |
One, or several overlapping tokens, along with the score(s) and the scope of
the original text.
|
| TokenSources |
Convenience methods for obtaining a
TokenStream for use with the Highlighter - can obtain from
term vectors with offsets and positions or from an Analyzer re-parsing the stored content. |
| TokenStreamFromTermVector |
TokenStream created from a term vector field.
|
| WeightedSpanTerm |
Lightweight class to hold term, weight, and positions used for scoring this
term.
|
| WeightedSpanTermExtractor |
Class used to extract
WeightedSpanTerms from a Query based on whether
Terms from the Query are contained in a supplied TokenStream. |
| WeightedSpanTermExtractor.PositionCheckingMap<K> |
This class makes sure that if both position sensitive and insensitive
versions of the same term are added, the position insensitive one wins.
|
| WeightedTerm |
Lightweight class to hold term and a weight value used for scoring this term
|
| Exception | Description |
|---|---|
| InvalidTokenOffsetsException |
Exception thrown if TokenStream Tokens are incompatible with provided text
|
The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages. The Highlighter class is the central component and can be used to extract the most interesting sections of a piece of text and highlight them, with the help of Fragmenter, fragment Scorer, and Formatter classes.
//... Above, create documents with two fields, one with term vectors (tv) and one without (notv)
IndexSearcher searcher = new IndexSearcher(directory);
QueryParser parser = new QueryParser("notv", analyzer);
Query query = parser.parse("million");
TopDocs hits = searcher.search(query, 10);
SimpleHTMLFormatter htmlFormatter = new SimpleHTMLFormatter();
Highlighter highlighter = new Highlighter(htmlFormatter, new QueryScorer(query));
for (int i = 0; i < 10; i++) {
int id = hits.scoreDocs[i].doc;
Document doc = searcher.doc(id);
String text = doc.get("notv");
TokenStream tokenStream = TokenSources.getAnyTokenStream(searcher.getIndexReader(), id, "notv", analyzer);
TextFragment[] frag = highlighter.getBestTextFragments(tokenStream, text, false, 10);//highlighter.getBestFragments(tokenStream, text, 3, "...");
for (int j = 0; j < frag.length; j++) {
if ((frag[j] != null) && (frag[j].getScore() > 0)) {
System.out.println((frag[j].toString()));
}
}
//Term vector
text = doc.get("tv");
tokenStream = TokenSources.getAnyTokenStream(searcher.getIndexReader(), hits.scoreDocs[i].doc, "tv", analyzer);
frag = highlighter.getBestTextFragments(tokenStream, text, false, 10);
for (int j = 0; j < frag.length; j++) {
if ((frag[j] != null) && (frag[j].getScore() > 0)) {
System.out.println((frag[j].toString()));
}
}
System.out.println("-------------");
}
The highlighter takes a TokenStream as input. Until now these streams have typically been produced using an Analyzer but the new class TokenSources provides helper methods for obtaining TokenStreams from the new TermVector position support (see latest CVS version).
The new class GradientFormatter can use a scale of colors to highlight terms according to their score. A subtle use of color can help emphasise the reasons for matching (useful when doing "MoreLikeThis" queries and you want to see what the basis of the similarities are).
The QueryScorer class has constructors that use an IndexReader to derive the IDF (inverse document frequency) for each term in order to influence the score. This is useful for helping to extracting the most significant sections of a document and in supplying scores used by the new GradientFormatter to color significant words more strongly. The QueryScorer.getMaxTermWeight method is useful when passed to the GradientFormatter constructor to define the top score which is associated with the top color.
Copyright © 2000-2019 Apache Software Foundation. All Rights Reserved.