A small search engine with spelling correction

The problem

Build a search engine over a collection of short documents that corrects misspelled queries, ranks the results and shows the passage that matched.

Requirements

Technologies

Step by step

  1. Write or collect the documents and a list of test queries with the documents they should find.
  2. Build the tokenizer, the inverted index and the TF-IDF vectors.
  3. Implement ranking and check it on the test queries.
  4. Add spelling correction for query words that are not in the vocabulary.
  5. Add snippets: find the sentence with the most query words and mark them.
  6. Measure how often the right document is in the top three, before and after spelling correction.

Expected result

A command-line search tool that forgives typos and explains its results, with a measured hit rate.

How it is evaluated

Ways to improve it