Data science: Choose a decision threshold

A model gives each row a score between 0 and 1. Try several thresholds, measure each, and choose the best one by F1.

What to build

A program that evaluates a list of thresholds and picks the best.

Requirements

  • For each threshold, in order, print threshold T: TP a FP b FN c F1 F, with T exactly as given and F to three decimals.
  • Rows with a score greater than or equal to the threshold are predicted positive. When 2TP + FP + FN is 0, F1 is 0.
  • Finally print best threshold T: the threshold with the highest F1, taking the first one on a tie.

Input

Three lines: the scores, the actual labels (0 or 1) in the same order, and the thresholds to try.

Output

One line per threshold, then one more line.

Constraints

1 to 1000 rows; 1 to 20 thresholds; scores and thresholds are between 0 and 1.

Example

Input:

0.9 0.8 0.7 0.6 0.4 0.3 0.2 0.1
1 1 0 1 0 1 0 0
0.25 0.5 0.75

Output:

threshold 0.25: TP 4 FP 2 FN 0 F1 0.800
threshold 0.5: TP 3 FP 1 FN 1 F1 0.750
threshold 0.75: TP 2 FP 0 FN 2 F1 0.667
best threshold 0.25

Why it works

Each threshold turns the same scores into a different set of predictions, so recomputing the counts at every threshold shows the trade-off between false alarms and misses, and F1 picks a balance.

Examples

Example 1 · Input
0.9 0.8 0.7 0.6 0.4 0.3 0.2 0.1
1 1 0 1 0 1 0 0
0.25 0.5 0.75
Expected output
threshold 0.25: TP 4 FP 2 FN 0 F1 0.800
threshold 0.5: TP 3 FP 1 FN 1 F1 0.750
threshold 0.75: TP 2 FP 0 FN 2 F1 0.667
best threshold 0.25
Example 2 · Input
0.2 0.8
0 1
0.5 0.9
Expected output
threshold 0.5: TP 1 FP 0 FN 0 F1 1.000
threshold 0.9: TP 0 FP 0 FN 1 F1 0.000
best threshold 0.5

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

At threshold t, a row is predicted positive when its score is at least t.

Hint 2

F1 = 2TP ÷ (2TP + FP + FN); when that denominator is 0, use 0.

Hint 3

Keep the first threshold with the highest F1 by using a strict comparison.

Solution

Show one correct solution (try it yourself first)
scores = list(map(float, input().split()))
labels = list(map(int, input().split()))
raw = input().split()
best, best_f1 = None, -1
for t in raw:
    pred = [1 if s >= float(t) else 0 for s in scores]
    tp = sum(1 for a, p in zip(labels, pred) if a == 1 and p == 1)
    fp = sum(1 for a, p in zip(labels, pred) if a == 0 and p == 1)
    fn = sum(1 for a, p in zip(labels, pred) if a == 1 and p == 0)
    d = 2 * tp + fp + fn
    f1 = 2 * tp / d if d else 0
    print(f"threshold {t}: TP {tp} FP {fp} FN {fn} F1 {f1:.3f}")
    if f1 > best_f1:
        best, best_f1 = t, f1
print("best threshold", best)

Lesson: Evaluating classifiers →