Apply dropout exactly as frameworks do, in both directions.
What to build
A program that applies inverted dropout to a layer's activations and to the gradient flowing back.
Requirements
- The mask has 1 for a kept unit and 0 for a dropped one; keep is the probability of keeping a unit. In
trainmode use the mask and scale by 1/keep; intestmode pass everything through unchanged. - Print
forward:and the outputs, thenbackward:and the input gradients, all with four decimal places.
Input
Line 1: the mode and keep. Line 2: the activations. Line 3: the mask. Line 4: the gradients arriving from the next layer.
Output
Two lines.
Constraints
At most 1,000 units; 0 < keep ≤ 1.
Example
Input:
train 0.5
1 2 3 4
1 0 1 0
0.1 0.2 0.3 0.4
Output:
forward: 2.0000 0.0000 6.0000 0.0000
backward: 0.2000 0.0000 0.6000 0.0000
Why it works
Dividing by keep makes each unit's expected output the same as without dropout, so the network can run at inference with no dropout and no rescaling.
Examples
train 0.5 1 2 3 4 1 0 1 0 0.1 0.2 0.3 0.4
forward: 2.0000 0.0000 6.0000 0.0000 backward: 0.2000 0.0000 0.6000 0.0000
test 0.5 1 2 3 4 1 0 1 0 0.1 0.2 0.3 0.4
forward: 1.0000 2.0000 3.0000 4.0000 backward: 0.1000 0.2000 0.3000 0.4000
Prompts inside input() are ignored when checking, so input("n: ") is fine.
Hints
Hint 1
Forward: output = activation × mask / keep, so dropped units become 0.
Hint 2
Backward: gradient in = gradient out × mask / keep, in training; at inference both pass unchanged.
Solution
Show one correct solution (try it yourself first)
mode, keep = input().split()
keep = float(keep)
acts = list(map(float, input().split()))
mask = list(map(float, input().split()))
grads = list(map(float, input().split()))
if mode == "test":
mask, keep = [1.0] * len(acts), 1.0
print("forward: " + " ".join(f"{a * m / keep + 0.0:.4f}" for a, m in zip(acts, mask)))
print("backward: " + " ".join(f"{g * m / keep + 0.0:.4f}" for g, m in zip(grads, mask)))