Deep learning: Sinusoidal positional encodings

Compute the position signals added to a transformer's inputs.

What to build

A program that computes the sinusoidal positional encodings for the first P positions.

Requirements

  • PE[p][2i] = sin(p / 10000^(2i / d)) and PE[p][2i + 1] = cos(p / 10000^(2i / d)), for positions p = 0 … P − 1; d is even.
  • Print one row per position, values with four decimal places separated by spaces.

Input

One line: P and d.

Output

P lines.

Constraints

P ≤ 100; d ≤ 64 and even.

Example

Input:

2 2

Output:

0.0000 1.0000
0.8415 0.5403

Why it works

Each pair of columns is a wave of a different frequency, so every position gets a unique pattern, and nearby positions get similar ones.

Examples

Example 1 · Input
2 2
Expected output
0.0000 1.0000
0.8415 0.5403
Example 2 · Input
1 4
Expected output
0.0000 1.0000 0.0000 1.0000

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

For column c: the frequency exponent uses i = c rounded down to an even number, angle = p / 10000^(i / d).

Hint 2

Even columns use math.sin(angle), odd columns use math.cos(angle).

Solution

Show one correct solution (try it yourself first)
import math

P, d = map(int, input().split())
for p in range(P):
    row = []
    for c in range(d):
        angle = p / 10000 ** ((c - c % 2) / d)
        row.append(math.sin(angle) if c % 2 == 0 else math.cos(angle))
    print(" ".join(f"{v + 0.0:.4f}" for v in row))

Lesson: Transformers →