Train Naive Bayes, run it on labelled test messages and measure how well it does.
What to build
A program that trains a classifier, lists its mistakes and reports accuracy, precision and recall.
Requirements
- Train multinomial Naive Bayes with add-one smoothing on the training lines (
label: text, lowercased, split on whitespace); test words never seen in training are ignored, and ties go to the alphabetically first label. - Classify every test line (also
label: text). For each wrong prediction, in input order, printmistake: TEXT -> PREDICTED, where TEXT is the test text with surrounding spaces removed. - Then print
accuracy A, and for each training label in alphabetical orderlabel precision P recall R, all with three decimal places (0.000 when a divisor is 0).
Input
Line 1: n. Then n training lines. Then m and m test lines.
Output
The mistakes, then the accuracy line, then one line per label.
Constraints
n ≤ 500; 1 ≤ m ≤ 200.
Example
Input:
4
spam: win money now
spam: free money
ham: see you at lunch
ham: lunch money
4
spam: win free money
ham: lunch at noon
ham: free lunch
spam: see you money
Output:
mistake: see you money -> ham
accuracy 0.750
ham precision 0.667 recall 1.000
spam precision 1.000 recall 0.500
Why it works
Accuracy alone can hide a class the model never predicts; precision shows how trustworthy a predicted label is and recall how many true cases are found, so together they show where the classifier fails.
Examples
4 spam: win money now spam: free money ham: see you at lunch ham: lunch money 4 spam: win free money ham: lunch at noon ham: free lunch spam: see you money
mistake: see you money -> ham accuracy 0.750 ham precision 0.667 recall 1.000 spam precision 1.000 recall 0.500
2 pos: great fun neg: awful dull 2 pos: fun neg: dull film
accuracy 1.000 neg precision 1.000 recall 1.000 pos precision 1.000 recall 1.000
Prompts inside input() are ignored when checking, so input("n: ") is fine.
Hints
Hint 1
Train exactly as in the medium exercise, then classify each test message and compare with its true label.
Hint 2
precision = correct predictions of the class ÷ all predictions of the class; recall = correct predictions of the class ÷ test messages truly in the class. Use 0 when the divisor is 0.
Solution
Show one correct solution (try it yourself first)
import math
n = int(input())
docs, counts, totals, vocab = {}, {}, {}, []
for _ in range(n):
label, text = input().split(":", 1)
label = label.strip()
docs[label] = docs.get(label, 0) + 1
counts.setdefault(label, {})
totals.setdefault(label, 0)
for w in text.lower().split():
counts[label][w] = counts[label].get(w, 0) + 1
totals[label] += 1
if w not in vocab:
vocab.append(w)
labels = sorted(docs)
def predict(text):
words = [w for w in text.lower().split() if w in vocab]
scores = {}
for label in labels:
s = math.log(docs[label] / n)
for w in words:
s += math.log((counts[label].get(w, 0) + 1) / (totals[label] + len(vocab)))
scores[label] = s
return max(labels, key=lambda c: scores[c])
results = []
for _ in range(int(input())):
actual, text = input().split(":", 1)
actual, text = actual.strip(), text.strip()
guess = predict(text)
results.append((actual, guess))
if guess != actual:
print(f"mistake: {text} -> {guess}")
print(f"accuracy {sum(1 for a, g in results if a == g) / len(results):.3f}")
for label in labels:
tp = sum(1 for a, g in results if a == label and g == label)
predicted = sum(1 for a, g in results if g == label)
actual = sum(1 for a, g in results if a == label)
p = tp / predicted if predicted else 0.0
r = tp / actual if actual else 0.0
print(f"{label} precision {p:.3f} recall {r:.3f}")