Overview (Lucene 4.0.0-BETA API)

Packages
Package	Description
org.apache.lucene	Top-level package.
org.apache.lucene.analysis	API and code to convert text into indexable/searchable tokens.
org.apache.lucene.analysis.tokenattributes	General-purpose attributes for text analysis.
org.apache.lucene.codecs	Codecs API: API for customization of the encoding and structure of the index.
org.apache.lucene.codecs.appending	Codec for on append-only outputs, such as plain output streams and append-only filesystems.
org.apache.lucene.codecs.bloom	Codec PostingsFormat for fast access to low-frequency terms such as primary key fields.
org.apache.lucene.codecs.intblock	Intblock: base support for fixed or variable length block integer encoders
org.apache.lucene.codecs.lucene3x	Codec to support Lucene 3.x indexes (readonly)
org.apache.lucene.codecs.lucene40	Lucene 4.0 file format.
org.apache.lucene.codecs.lucene40.values	Default DocValues implementation for Lucene 4.0 indexes.
org.apache.lucene.codecs.memory	Postings format that is read entirely into memory.
org.apache.lucene.codecs.perfield	Postings format that can delegate to different formats per-field.
org.apache.lucene.codecs.pulsing	Pulsing Codec: inlines low frequency terms' postings into terms dictionary.
org.apache.lucene.codecs.sep	Sep: base support for separate files (doc,frq,pos,skp,pyl)
org.apache.lucene.codecs.simpletext	Simpletext Codec: writes human readable postings.
org.apache.lucene.document	The logical representation of a `Document` for indexing and searching.
org.apache.lucene.index	Code to maintain and access indices.
org.apache.lucene.search	Code to search indices.
org.apache.lucene.search.payloads	The payloads package provides Query mechanisms for finding and using payloads.
org.apache.lucene.search.similarities	This package contains the various ranking models that can be used in Lucene.
org.apache.lucene.search.spans	The calculus of spans.
org.apache.lucene.store	Binary i/o API, used for all index data.
org.apache.lucene.util	Some utility classes.
org.apache.lucene.util.automaton	Finite-state automaton for regular expressions.
org.apache.lucene.util.fst	Finite state transducers
org.apache.lucene.util.hash	Hashing functions load-able via SPI service
org.apache.lucene.util.mutable	Comparable object wrappers
org.apache.lucene.util.packed	Packed integer arrays and streams.

Apache Lucene is a high-performance, full-featured text search engine library. Here's a simple example how to use Lucene for indexing and searching (using JUnit to check if the results are what we expect):

    Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_CURRENT);

    // Store the index in memory:
    Directory directory = new RAMDirectory();
    // To store an index on disk, use this instead:
    //Directory directory = FSDirectory.open("/tmp/testindex");
    IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_CURRENT, analyzer);
    IndexWriter iwriter = new IndexWriter(directory, config);
    Document doc = new Document();
    String text = "This is the text to be indexed.";
    doc.add(new Field("fieldname", text, TextField.TYPE_STORED));
    iwriter.addDocument(doc);
    iwriter.close();
    
    // Now search the index:
    DirectoryReader ireader = DirectoryReader.open(directory);
    IndexSearcher isearcher = new IndexSearcher(ireader);
    // Parse a simple query that searches for "text":
    QueryParser parser = new QueryParser(Version.LUCENE_CURRENT, "fieldname", analyzer);
    Query query = parser.parse("text");
    ScoreDoc[] hits = isearcher.search(query, null, 1000).scoreDocs;
    assertEquals(1, hits.length);
    // Iterate through the results:
    for (int i = 0; i < hits.length; i++) {
      Document hitDoc = isearcher.doc(hits[i].doc);
      assertEquals("This is the text to be indexed.", hitDoc.get("fieldname"));
    }
    ireader.close();
    directory.close();

The Lucene API is divided into several packages:

org.apache.lucene.analysis defines an abstract Analyzer API for converting text from a Reader into a TokenStream, an enumeration of token Attributes. A TokenStream can be composed by applying TokenFilters to the output of a Tokenizer. Tokenizers and TokenFilters are strung together and applied with an Analyzer. analyzers-common provides a number of Analyzer implementations, including StopAnalyzer and the grammar-based StandardAnalyzer.
org.apache.lucene.codecs provides an abstraction over the encoding and decoding of the inverted index structure, as well as different implementations that can be chosen depending upon application needs.
org.apache.lucene.document provides a simple Document class. A Document is simply a set of named Fields, whose values may be strings or instances of Reader.
org.apache.lucene.index provides two primary classes: IndexWriter, which creates and adds documents to indices; and IndexReader, which accesses the data in the index.
org.apache.lucene.search provides data structures to represent queries (ie TermQuery for individual words, PhraseQuery for phrases, and BooleanQuery for boolean combinations of queries) and the IndexSearcher which turns queries into TopDocs. A number of QueryParsers are provided for producing query structures from strings or xml.
org.apache.lucene.store defines an abstract class for storing persistent data, the Directory, which is a collection of named files written by an IndexOutput and read by an IndexInput. Multiple implementations are provided, including FSDirectory, which uses a file system directory to store files, and RAMDirectory which implements files as memory-resident data structures.
org.apache.lucene.util contains a few handy data structures and util classes, ie OpenBitSet and PriorityQueue.

To use Lucene, an application should:

Create Documents by adding Fields;
Create an IndexWriter and add documents to it with addDocument();
Call QueryParser.parse() to build a query from a string; and
Create an IndexSearcher and pass the query to its search() method.

Some simple examples of code which does this are:

IndexFiles.java creates an index for all the files contained in a directory.
SearchFiles.java prompts for queries and searches an index.

To demonstrate these, try something like:

> java -cp lucene-core.jar:lucene-demo.jar:lucene-analyzers-common.jar org.apache.lucene.demo.IndexFiles rec.food.recipes/soups
adding rec.food.recipes/soups/abalone-chowder
[ ... ]
> java -cp lucene-core.jar:lucene-demo.jar:lucene-analyzers-common.jar org.apache.lucene.demo.SearchFiles
Query: chowder
Searching for: chowder
34 total matching documents
1. rec.food.recipes/soups/spam-chowder
[ ... thirty-four documents contain the word "chowder" ... ]
Query: "clam chowder" AND Manhattan
Searching for: +"clam chowder" +manhattan
2 total matching documents
1. rec.food.recipes/soups/clam-chowder
[ ... two documents contain the phrase "clam chowder" and the word "manhattan" ... ]
[ Note: "+" and "-" are canonical, but "AND", "OR" and "NOT" may be used. ]

Lucene 4.0.0-BETA core API