Convolutional and recurrent networks
Convolutions with padding, stride and gradients, pooling and complete CNNs, and recurrent networks, backpropagation through time and LSTMs.
About 5 hours of study
Learning objectives
- Compute convolutions and their gradients
- Track shapes and parameters through a CNN
- Run RNNs and LSTMs and backpropagate through time
Lessons
- 1
Convolutions Advanced
Slide a small kernel of weights across an image to detect local patterns such as edges. Compute convolutions by hand and in code, control the output size with padding and stride, and find the gradients a convolution layer needs for training.
- 2
Pooling and convolutional networks Advanced
Build a whole convolutional network: convolution layers with ReLU, pooling layers that shrink feature maps, then flattening and dense layers for the final decision. Track shapes and parameter counts through the network.
- 3
Recurrent networks and LSTMs Advanced
Process sequences such as text, sound and time series with a hidden state carried from step to step. Run a recurrent network, backpropagate through time, see why gradients vanish over long sequences, and how LSTM gates fix it.
Module practice
- Deep learning: A 1D convolutionEasy
- Deep learning: A 2D convolution with padding and strideMedium
- Deep learning: Gradients of a convolutionHard
- Deep learning: Max and average poolingEasy
- Deep learning: Shapes and parameters of a CNNMedium
- Deep learning: Forward pass of a small CNNHard
- Deep learning: Run a recurrent networkEasy
- Deep learning: One step of an LSTMMedium
- Deep learning: Backpropagation through timeHard