Deep learning: Run a recurrent network

Watch a hidden state evolve along a sequence.

What to build

A program that runs a one-unit recurrent network over a sequence.

Requirements

  • The update is hₜ = f(wx × xₜ + wh × hₜ₋₁ + b) with h₀ = 0, where f is tanh or linear.
  • Print h1 = V, h2 = V, … with four decimal places.

Input

Line 1: the activation, wx, wh and b. Line 2: the inputs.

Output

One line per input.

Constraints

At most 1,000 inputs.

Example

Input:

linear 1 0.5 0
1 1 1

Output:

h1 = 1.0000
h2 = 1.5000
h3 = 1.7500

Why it works

The hidden state mixes each new input with a scaled copy of the previous state, so it carries a fading memory of everything seen so far.

Examples

Example 1 · Input
linear 1 0.5 0
1 1 1
Expected output
h1 = 1.0000
h2 = 1.5000
h3 = 1.7500
Example 2 · Input
tanh 1 0 0
0 1
Expected output
h1 = 0.0000
h2 = 0.7616

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

Start with h = 0; for each input, compute z = wx × x + wh × h + b and then h = f(z).

Hint 2

Print h after every step.

Solution

Show one correct solution (try it yourself first)
import math

activation, wx, wh, b = input().split()
wx, wh, b = float(wx), float(wh), float(b)
f = math.tanh if activation == "tanh" else (lambda z: z)
h = 0.0
for t, x in enumerate(map(float, input().split()), start=1):
    h = f(wx * x + wh * h + b)
    print(f"h{t} = {h + 0.0:.4f}")

Lesson: Recurrent networks and LSTMs →