Exact match accuracy

Exact match is the simplest score for short answers: the cleaned prediction must equal the cleaned reference.

What to build

A program that scores predictions against references with exact match.

Requirements

  • Clean both texts: lower-case them, remove the characters ., ,, ! and ?, and make the spaces single.
  • Print K: match or K: miss for each pair.
  • Print exact match: C/N = P% with P to one decimal place.

Input

Line 1: n. Then n lines of the form PREDICTION => REFERENCE.

Output

n + 1 lines.

Constraints

1 ≤ n ≤ 100.

Example

Input:

3
Paris. => paris
the  cat => the cat
a dog => the dog

Output:

1: match
2: match
3: miss
exact match: 2/3 = 66.7%

Why it works

Cleaning removes differences that do not matter, such as capitals and punctuation, so only a real difference in words counts as a miss.

Examples

Example 1 · Input
3
Paris. => paris
the  cat => the cat
a dog => the dog
Expected output
1: match
2: match
3: miss
exact match: 2/3 = 66.7%
Example 2 · Input
1
Yes! => no
Expected output
1: miss
exact match: 0/1 = 0.0%

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

Clean a text by lower-casing it, removing `.`, `,`, `!` and `?`, and joining its words with single spaces.

Hint 2

Split each line on ` => ` into the prediction and the reference.

Solution

Show one correct solution (try it yourself first)
n = int(input())


def norm(t):
    t = "".join(ch for ch in t.lower() if ch not in ".,!?")
    return " ".join(t.split())


hits = 0
for i in range(1, n + 1):
    pred, ref = input().split(" => ")
    ok = norm(pred) == norm(ref)
    hits += ok
    print(f"{i}: {'match' if ok else 'miss'}")
print(f"exact match: {hits}/{n} = {100 * hits / n:.1f}%")

Lesson: Evaluating generative models →