Trace how much of a gradient survives the trip back through many layers.
What to build
A program that computes the gradient of the last activation of a chain of one-neuron layers with respect to every earlier activation.
Requirements
- Activations:
sigmoid(slope a(1 − a)),tanh(slope 1 − a²),reluorlinear(slope 1). Each layer has one weight and no bias. - Print
layer K: Gfor K from L down to 0, with G in scientific notation with three decimals (Python's.3e).
Input
Line 1: the activation and x. Line 2: the weights w₁ … w_L.
Output
L + 1 lines.
Constraints
L ≤ 100.
Example
Input:
tanh 0
0.5 0.5 0.5
Output:
layer 3: 1.000e+00
layer 2: 5.000e-01
layer 1: 2.500e-01
layer 0: 1.250e-01
Why it works
Backpropagation multiplies the gradient by each layer's weight and slope, so many small factors shrink it towards zero and many large ones blow it up.
Examples
Example 1 · Input
tanh 0 0.5 0.5 0.5
Expected output
layer 3: 1.000e+00 layer 2: 5.000e-01 layer 1: 2.500e-01 layer 0: 1.250e-01
Example 2 · Input
linear 1 2 2 2 2
Expected output
layer 4: 1.000e+00 layer 3: 2.000e+00 layer 2: 4.000e+00 layer 1: 8.000e+00 layer 0: 1.600e+01
Prompts inside input() are ignored when checking, so input("n: ") is fine.
Hints
Hint 1
Forward: a₀ = x, zₖ = wₖ aₖ₋₁, aₖ = f(zₖ); keep every z.
Hint 2
Backward: g_L = 1, then gₖ₋₁ = gₖ × f′(zₖ) × wₖ. ReLU's slope is 1 for z > 0 and 0 otherwise.
Solution
Show one correct solution (try it yourself first)
import math
activation, x = input().split()
a = float(x)
ws = list(map(float, input().split()))
f = {
"sigmoid": lambda z: 1 / (1 + math.exp(-z)),
"tanh": math.tanh,
"relu": lambda z: max(0.0, z),
"linear": lambda z: z,
}[activation]
slopes = []
for w in ws:
z = w * a
a = f(z)
if activation == "sigmoid":
slopes.append(a * (1 - a))
elif activation == "tanh":
slopes.append(1 - a * a)
elif activation == "relu":
slopes.append(1.0 if z > 0 else 0.0)
else:
slopes.append(1.0)
g = 1.0
print(f"layer {len(ws)}: {g:.3e}")
for k in range(len(ws), 0, -1):
g = g * slopes[k - 1] * ws[k - 1]
print(f"layer {k - 1}: {g + 0.0:.3e}")