Deep Learning: final assessment

This covers the whole course. Take it after the last module.

Final quiz

1. What is sigmoid(0)?
2. Which loss suits predicting a probability?
3. What is an epoch?
4. What does a 3 × 3 convolution with padding 1 and stride 1 do to the size of a feature map?
5. What carries information from step to step in an RNN?
6. Why must activation functions be non-linear?
7. What does backpropagation compute?
8. Why subtract the largest logit before softmax?
9. Which initialization suits ReLU layers?
10. What does early stopping keep?
11. Why do convolution layers need far fewer weights than dense layers on images?
12. Why does Adam's step size stay near the learning rate?
13. Why does layer norm suit transformers better than batch norm?
14. What makes very deep transformers trainable?
15. In attention, what decides the weights?
16. A 2-layer network has a dead ReLU unit. What does it pass back?

Practical assessment: Train a network from scratch

Without using hints or libraries, write a program that trains a two-layer neural network on a small classification dataset you create, and reports what it learned.

Final project

A digit classifier from scratch: Build and train a neural network in plain Python that recognizes small pixel images of the digits 0 to 9, and measure how well it works.

How it is evaluated

To complete this course