Data science: Precision, recall and F1

Accuracy is not enough. Build the confusion matrix and the measures that show how a classifier really behaves.

What to build

A program that prints the confusion matrix, precision, recall and F1.

Requirements

  • Print TP a FP b FN c TN d.
  • Print precision P, recall R and F1 F, each to three decimals.
  • Print undefined instead of a number when precision's denominator (TP + FP) or recall's denominator (TP + FN) is 0; F1 is undefined when precision or recall is undefined, or when both are 0.

Input

Two lines with the same number of labels, each 0 or 1: the actual labels, then the predictions.

Output

Four lines.

Constraints

1 to 1000 labels per line.

Example

Input:

1 0 1 1 0 0 1 0
1 0 0 1 0 1 1 0

Output:

TP 3 FP 1 FN 1 TN 3
precision 0.750
recall 0.750
F1 0.750

Why it works

Precision divides by everything predicted positive and recall by everything actually positive, so they answer different questions; F1 is their harmonic mean, which is high only when both are.

Examples

Example 1 · Input
1 0 1 1 0 0 1 0
1 0 0 1 0 1 1 0
Expected output
TP 3 FP 1 FN 1 TN 3
precision 0.750
recall 0.750
F1 0.750
Example 2 · Input
1 1 0 0
1 0 0 0
Expected output
TP 1 FP 0 FN 1 TN 2
precision 1.000
recall 0.500
F1 0.667

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

Count TP, FP, FN and TN with one loop over zip(actual, predicted).

Hint 2

Check each denominator before dividing.

Hint 3

F1 = 2TP ÷ (2TP + FP + FN), but print it as undefined when precision or recall is undefined or both are 0.

Solution

Show one correct solution (try it yourself first)
actual = list(map(int, input().split()))
pred = list(map(int, input().split()))
tp = sum(1 for a, p in zip(actual, pred) if a == 1 and p == 1)
fp = sum(1 for a, p in zip(actual, pred) if a == 0 and p == 1)
fn = sum(1 for a, p in zip(actual, pred) if a == 1 and p == 0)
tn = sum(1 for a, p in zip(actual, pred) if a == 0 and p == 0)
print(f"TP {tp} FP {fp} FN {fn} TN {tn}")
precision = tp / (tp + fp) if tp + fp else None
recall = tp / (tp + fn) if tp + fn else None
print("precision", "undefined" if precision is None else f"{precision:.3f}")
print("recall", "undefined" if recall is None else f"{recall:.3f}")
if precision is None or recall is None or precision + recall == 0:
    print("F1 undefined")
else:
    print(f"F1 {2 * precision * recall / (precision + recall):.3f}")

Lesson: Evaluating classifiers →