Train a small network on a whole dataset per update.
What to build
A program that trains a one-hidden-layer ReLU network with full-batch gradient descent and reports the loss each epoch.
Requirements
- The network maps n inputs to h ReLU units to one linear output; the loss is the mean of (ŷ − y)² over the N examples. ReLU's derivative is 1 for a positive input and 0 otherwise.
- For each epoch print
epoch E: loss L(four decimal places), the loss before that epoch's update.
Input
Line 1: n, h, N, the learning rate and the number of epochs. Then N lines: the inputs and the target. Then h rows of W₁, then b₁, then W₂, then b₂.
Output
One line per epoch.
Constraints
n, h ≤ 10; N ≤ 50; at most 100 epochs.
Example
Input:
1 1 2 0.05 2
1 2
2 4
1
0
1
0
Output:
epoch 1: loss 2.5000
epoch 2: loss 0.1495
Why it works
Averaging the gradient over every example gives the exact downhill direction of the mean loss, so each update improves the fit to the whole dataset at once.
Examples
1 1 2 0.05 2 1 2 2 4 1 0 1 0
epoch 1: loss 2.5000 epoch 2: loss 0.1495
1 1 1 0.1 2 2 0 1 0 1 0
epoch 1: loss 4.0000 epoch 2: loss 0.1600
Prompts inside input() are ignored when checking, so input("n: ") is fine.
Hints
Hint 1
Each epoch: compute the mean loss over all examples, then the gradients averaged over the examples, then update once.
Hint 2
For example i, g = 2(ŷ − y) / N; the gradients are sums over the examples of the single-example formulas with that g.
Solution
Show one correct solution (try it yourself first)
n, h, N, rate, epochs = input().split()
n, h, N, rate, epochs = int(n), int(h), int(N), float(rate), int(epochs)
data = []
for _ in range(N):
*x, y = map(float, input().split())
data.append((x, y))
W1 = [list(map(float, input().split())) for _ in range(h)]
b1 = list(map(float, input().split()))
W2 = list(map(float, input().split()))
b2 = float(input())
for epoch in range(1, epochs + 1):
dW1 = [[0.0] * n for _ in range(h)]
db1 = [0.0] * h
dW2 = [0.0] * h
db2 = 0.0
loss = 0.0
for x, y in data:
z = [sum(w * xi for w, xi in zip(row, x)) + b for row, b in zip(W1, b1)]
a = [max(0.0, v) for v in z]
y_hat = sum(w * v for w, v in zip(W2, a)) + b2
loss += (y_hat - y) ** 2 / N
g = 2 * (y_hat - y) / N
for j in range(h):
dW2[j] += g * a[j]
if z[j] > 0:
dz = g * W2[j]
db1[j] += dz
for i in range(n):
dW1[j][i] += dz * x[i]
db2 += g
print(f"epoch {epoch}: loss {loss:.4f}")
for j in range(h):
for i in range(n):
W1[j][i] -= rate * dW1[j][i]
b1[j] -= rate * db1[j]
W2[j] -= rate * dW2[j]
b2 -= rate * db2