Do a full forward pass, backward pass and update for a two-layer network.
What to build
A program that performs one step of gradient descent for a network with one ReLU hidden layer and one linear output.
Requirements
- The network maps n inputs to h hidden ReLU units to one output. ReLU's derivative is 1 for a positive input and 0 otherwise (including 0). The loss is (ŷ − y)² for one example; every parameter p becomes p − rate × ∂L/∂p.
- Print
loss L(before the update), then the updatedW1:(rows separated by|),b1:,W2:andb2:, with four decimal places.
Input
Line 1: n and h. Line 2: x. Then h rows of W₁, then b₁, then W₂, then a line with b₂, y and the learning rate.
Output
Five lines.
Constraints
n, h ≤ 10.
Example
Input:
2 2
1 2
1 0
0 1
0 -3
1 1
0 3 0.1
Output:
loss 4.0000
W1: 1.4000 0.8000 | 0.0000 1.0000
b1: 0.4000 -3.0000
W2: 1.4000 1.0000
b2: 0.4000
Why it works
Backpropagation reuses the forward values to find every weight's effect on the loss in one backward pass, and gradient descent then moves each weight a small step to reduce the loss.
Examples
2 2 1 2 1 0 0 1 0 -3 1 1 0 3 0.1
loss 4.0000 W1: 1.4000 0.8000 | 0.0000 1.0000 b1: 0.4000 -3.0000 W2: 1.4000 1.0000 b2: 0.4000
1 1 2 0.5 0 2 1 2 0.25
loss 1.0000 W1: -1.5000 b1: -1.0000 W2: 1.5000 b2: 0.5000
Prompts inside input() are ignored when checking, so input("n: ") is fine.
Hints
Hint 1
Forward: z₁ = W₁x + b₁, h = ReLU(z₁), ŷ = W₂·h + b₂, loss = (ŷ − y)². Keep z₁ and h.
Hint 2
Backward: g = 2(ŷ − y); dW₂ = g·h, db₂ = g; dz₁ᵢ = g·W₂ᵢ if z₁ᵢ > 0 else 0; dW₁ᵢⱼ = dz₁ᵢ·xⱼ, db₁ = dz₁. Compute every gradient before updating any weight.
Solution
Show one correct solution (try it yourself first)
n, h = map(int, input().split())
x = list(map(float, input().split()))
W1 = [list(map(float, input().split())) for _ in range(h)]
b1 = list(map(float, input().split()))
W2 = list(map(float, input().split()))
b2, y, rate = map(float, input().split())
z1 = [sum(w * xi for w, xi in zip(row, x)) + bi for row, bi in zip(W1, b1)]
hidden = [max(0.0, z) for z in z1]
y_hat = sum(w * a for w, a in zip(W2, hidden)) + b2
g = 2 * (y_hat - y)
dW2 = [g * a for a in hidden]
db2 = g
dz1 = [g * w if z > 0 else 0.0 for w, z in zip(W2, z1)]
dW1 = [[d * xi for xi in x] for d in dz1]
db1 = dz1
W1 = [[w - rate * dw for w, dw in zip(row, drow)] for row, drow in zip(W1, dW1)]
b1 = [b - rate * d for b, d in zip(b1, db1)]
W2 = [w - rate * d for w, d in zip(W2, dW2)]
b2 = b2 - rate * db2
def fmt(values):
return " ".join(f"{v + 0.0:.4f}" for v in values)
print(f"loss {(y_hat - y) ** 2:.4f}")
print("W1: " + " | ".join(fmt(row) for row in W1))
print("b1: " + fmt(b1))
print("W2: " + fmt(W2))
print(f"b2: {b2:.4f}")