NLP: TF-IDF of one document

Compute every part of the TF-IDF formula for the words of one document.

What to build

A program that prints tf, idf and tf-idf for each distinct word of a chosen document.

Requirements

  • Lowercase the documents and split them on whitespace. Use tf = count ÷ length, idf = ln(N ÷ df) with the natural logarithm, and tf-idf = tf × idf.
  • For each distinct word of document i, in order of first appearance, print word tf idf tfidf, each number with four decimal places.

Input

Line 1: N. Then N documents. Last line: i, the number of the document to analyse (from 1).

Output

One line per distinct word of document i.

Constraints

N ≤ 100.

Example

Input:

2
the cat sat
the dog
1

Output:

the 0.3333 0.0000 0.0000
cat 0.3333 0.6931 0.2310
sat 0.3333 0.6931 0.2310

Why it works

tf rewards words used often in the document, idf rewards words found in few documents, and their product is high only for words that are both.

Examples

Example 1 · Input
2
the cat sat
the dog
1
Expected output
the 0.3333 0.0000 0.0000
cat 0.3333 0.6931 0.2310
sat 0.3333 0.6931 0.2310
Example 2 · Input
4
a b
a c
a d
b b c e
4
Expected output
b 0.5000 0.6931 0.3466
c 0.2500 0.6931 0.1733
e 0.2500 1.3863 0.3466

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

tf = count in the document ÷ the document's length; df = number of documents containing the word; idf = `math.log(n / df)`.

Hint 2

Keep the distinct words of the chosen document in order of first appearance.

Solution

Show one correct solution (try it yourself first)
import math

n = int(input())
docs = [input().lower().split() for _ in range(n)]
doc = docs[int(input()) - 1]
seen = []
for w in doc:
    if w not in seen:
        seen.append(w)
for w in seen:
    tf = doc.count(w) / len(doc)
    df = sum(1 for d in docs if w in d)
    idf = math.log(n / df)
    print(f"{w} {tf:.4f} {idf:.4f} {tf * idf:.4f}")

Lesson: TF-IDF: weighting words →