Deep learning: Gradient descent with momentum

See how momentum builds up speed.

What to build

A program that minimizes f(x) = x² with momentum and prints x after every step.

Requirements

  • Start with v = 0. Each step computes g = 2x, sets v ← β v + g and x ← x − rate × v.
  • Print step I: x X with four decimal places.

Input

One line: the starting x, the learning rate, β and the number of steps.

Output

One line per step.

Constraints

At most 100 steps.

Example

Input:

1 0.1 0.9 3

Output:

step 1: x 0.8000
step 2: x 0.4600
step 3: x 0.0620

Why it works

Velocity adds up the recent gradients, so movement continues in a consistent direction even as the current gradient shrinks.

Examples

Example 1 · Input
1 0.1 0.9 3
Expected output
step 1: x 0.8000
step 2: x 0.4600
step 3: x 0.0620
Example 2 · Input
2 0.25 0 2
Expected output
step 1: x 1.0000
step 2: x 0.5000

Prompts inside input() are ignored when checking, so input("n: ") is fine.

Hints

Hint 1

For f(x) = x² the gradient is 2x.

Hint 2

Each step: v = β v + gradient, then x = x − rate × v.

Solution

Show one correct solution (try it yourself first)
x, rate, beta, steps = input().split()
x, rate, beta, steps = float(x), float(rate), float(beta), int(steps)
v = 0.0
for i in range(1, steps + 1):
    v = beta * v + 2 * x
    x = x - rate * v
    print(f"step {i}: x {x + 0.0:.4f}")

Lesson: Optimizers: momentum, RMSProp and Adam →