A leak-free prediction pipeline

Put the course together: impute, standardize and predict, with every statistic taken from the training data only, so nothing leaks from the test set.

What to build

A program that fits the preparation steps on training data and applies them, with a given logistic model, to test data.

Requirements

  • Training median: the median of the known training values (mean of the two middle ones for an even count). Fill missing training values (?) with it, then compute the mean and the population σ of the filled values.
  • Print train median M, mean A, std S with 2 decimal places.
  • For each test row: fill a missing value with the training median, compute z (0 if σ is 0) and p = 1 / (1 + e^(−(w z + b))), and print z Z -> p P -> C, with Z to 2 decimals, P to 3 and C the predicted class (1 when p ≥ 0.5).
  • Finally print test accuracy A with 3 decimal places.

Input

Line 1: n, then n training lines x label. Then m, then m test lines x label. Then a line w b. Any x may be ?.

Output

m + 2 lines.

Constraints

1 ≤ n, m ≤ 500; at least one training x is known.

Example

Input:

5
10 0
20 0
? 1
40 1
50 1
3
35 1
? 0
12 0
2 0

Output:

train median 30.00, mean 30.00, std 14.14
z 0.35 -> p 0.670 -> 1
z 0.00 -> p 0.500 -> 1
z -1.27 -> p 0.073 -> 0
test accuracy 0.667

Why it works

Fitting the median, mean and σ on the training data only means the test rows are treated exactly like future data, so the test accuracy is an honest estimate.

Examples

Example 1 · Input
5
10 0
20 0
? 1
40 1
50 1
3
35 1
? 0
12 0
2 0
Expected output
train median 30.00, mean 30.00, std 14.14
z 0.35 -> p 0.670 -> 1
z 0.00 -> p 0.500 -> 1
z -1.27 -> p 0.073 -> 0
test accuracy 0.667
Example 2 · Input
2
1 0
3 1
1
5 1
1 0
Expected output
train median 2.00, mean 2.00, std 1.00
z 3.00 -> p 0.953 -> 1
test accuracy 1.000

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

Compute the median of the known training values, fill the training gaps with it, then the mean and σ of the filled training values. Never look at the test rows for these.

Hint 2

For each test row: fill a gap with the training median, z = (x − mean) / σ (0 if σ is 0), p = σ(w z + b).

Hint 3

Predict 1 when p ≥ 0.5.

Solution

Show one correct solution (try it yourself first)
import math

n = int(input())
train = [input().split() for _ in range(n)]
m = int(input())
test = [input().split() for _ in range(m)]
w, b = map(float, input().split())
known = sorted(float(x) for x, _ in train if x != "?")
k = len(known)
median = known[k // 2] if k % 2 else (known[k // 2 - 1] + known[k // 2]) / 2
filled = [median if x == "?" else float(x) for x, _ in train]
mean = sum(filled) / n
std = (sum((v - mean) ** 2 for v in filled) / n) ** 0.5
print(f"train median {median:.2f}, mean {mean:.2f}, std {std:.2f}")
right = 0
for x, label in test:
    value = median if x == "?" else float(x)
    z = (value - mean) / std if std > 0 else 0.0
    p = 1 / (1 + math.exp(-(w * z + b)))
    c = 1 if p >= 0.5 else 0
    right += c == int(label)
    print(f"z {z:.2f} -> p {p:.3f} -> {c}")
print(f"test accuracy {right / m:.3f}")

Lesson: Machine learning in practice →