Accuracy and recall for each group

An overall score can hide who bears a model's errors. Break the results down by group.

What to build

A program that reports a classifier's accuracy and recall for each group and overall.

Requirements

  • For each group, in alphabetical order, print group G: n N, accuracy A, recall R.
  • Then print all: n N, accuracy A, recall R for every row together.
  • A and R have 3 decimal places; R is - when there are no actual positives.

Input

Line 1: n. Then n lines group actual predicted, with actual and predicted 0 or 1.

Output

One line per group, then the overall line.

Constraints

1 ≤ n ≤ 1000; groups are single words.

Example

Input:

20
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 1
B 1 0
B 0 0
B 1 0
B 1 1
A 1 1
A 0 0
B 1 0
B 0 0

Output:

group A: n 14, accuracy 0.929, recall 1.000
group B: n 6, accuracy 0.500, recall 0.250
all: n 20, accuracy 0.800, recall 0.727

Why it works

Accuracy and recall computed separately for each group show whether the model's mistakes, especially missed positives, fall unevenly on some people.

Examples

Example 1 · Input
20
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 0
A 1 1
A 0 1
B 1 0
B 0 0
B 1 0
B 1 1
A 1 1
A 0 0
B 1 0
B 0 0
Expected output
group A: n 14, accuracy 0.929, recall 1.000
group B: n 6, accuracy 0.500, recall 0.250
all: n 20, accuracy 0.800, recall 0.727
Example 2 · Input
2
x 1 1
y 0 1
Expected output
group x: n 1, accuracy 1.000, recall 1.000
group y: n 1, accuracy 0.000, recall -
all: n 2, accuracy 0.500, recall 1.000

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

Collect the rows of each group; groups are reported in alphabetical order.

Hint 2

Recall is found positives (actual 1, predicted 1) divided by actual positives; print `-` if a group has no actual positives.

Solution

Show one correct solution (try it yourself first)
n = int(input())
rows = [input().split() for _ in range(n)]


def report(name, group):
    right = sum(1 for _, a, p in group if a == p)
    positives = [r for r in group if r[1] == "1"]
    found = sum(1 for r in positives if r[2] == "1")
    recall = f"{found / len(positives):.3f}" if positives else "-"
    print(f"{name}: n {len(group)}, accuracy {right / len(group):.3f}, recall {recall}")


for g in sorted({r[0] for r in rows}):
    report(f"group {g}", [r for r in rows if r[0] == g])
report("all", rows)

Lesson: Machine learning in practice →