A product review analyser

The problem

Build a complete text classification pipeline in plain Python that labels product reviews as positive or negative, explains its decisions and measures how well it works.

Requirements

Technologies

Step by step

  1. Write the reviews, label them, and set aside a fifth of them as the test set.
  2. Write the tokenizer and normalizer and test them on awkward examples.
  3. Implement the lexicon scorer and evaluate it on the test set.
  4. Train Naive Bayes on the training set and evaluate it on the same test set.
  5. Add bigram features and compare the results.
  6. Read every mistake and write down why the model failed.

Expected result

A working sentiment classifier with an evaluation report that compares rules with learning and explains the typical errors.

How it is evaluated

Ways to improve it