Compute the evaluation report that every classifier needs.
What to build
A program that prints precision, recall and F1 for each class, the macro F1 and the accuracy.
Requirements
- Each input pair is
actual predicted. The classes are every label that appears as actual or predicted, in alphabetical order. - For each class print
class: precision P recall R f1 F. Precision is 0 when the class is never predicted, recall is 0 when it never occurs, and F1 is 0 when P + R = 0. - Then print
macro f1 M(the average F1 over the classes) andaccuracy A. All numbers have three decimal places.
Input
Line 1: n. Then n lines actual predicted.
Output
One line per class, then two lines.
Constraints
1 ≤ n ≤ 10,000.
Example
Input:
6
pos pos
neg neg
pos pos
neg neg
neg neutral
neg neg
Output:
neg: precision 1.000 recall 0.750 f1 0.857
neutral: precision 0.000 recall 0.000 f1 0.000
pos: precision 1.000 recall 1.000 f1 1.000
macro f1 0.619
accuracy 0.833
Why it works
Per-class precision and recall show which kind of mistake a model makes, F1 combines them, and the macro average gives a small class the same weight as a large one, unlike accuracy.
Examples
6 pos pos neg neg pos pos neg neg neg neutral neg neg
neg: precision 1.000 recall 0.750 f1 0.857 neutral: precision 0.000 recall 0.000 f1 0.000 pos: precision 1.000 recall 1.000 f1 1.000 macro f1 0.619 accuracy 0.833
4 spam ham spam spam ham ham ham spam
ham: precision 0.500 recall 0.500 f1 0.500 spam: precision 0.500 recall 0.500 f1 0.500 macro f1 0.500 accuracy 0.500
Prompts inside input() are ignored when checking, so input("n: ") is fine.
Hints
Hint 1
For class c: true positives = pairs with actual c and predicted c; precision divides them by all predictions of c, recall by all actual c.
Hint 2
F1 = 2PR / (P + R), and 0 when P + R is 0; macro F1 is the plain average of the classes' F1.
Solution
Show one correct solution (try it yourself first)
n = int(input())
pairs = [input().split() for _ in range(n)]
labels = sorted({a for a, _ in pairs} | {p for _, p in pairs})
f1s = []
for c in labels:
tp = sum(1 for a, p in pairs if a == c and p == c)
predicted = sum(1 for _, p in pairs if p == c)
actual = sum(1 for a, _ in pairs if a == c)
precision = tp / predicted if predicted else 0.0
recall = tp / actual if actual else 0.0
f1 = 2 * precision * recall / (precision + recall) if precision + recall else 0.0
f1s.append(f1)
print(f"{c}: precision {precision:.3f} recall {recall:.3f} f1 {f1:.3f}")
print(f"macro f1 {sum(f1s) / len(f1s):.3f}")
print(f"accuracy {sum(1 for a, p in pairs if a == p) / n:.3f}")