Convolutional and recurrent networks

Convolutions with padding, stride and gradients, pooling and complete CNNs, and recurrent networks, backpropagation through time and LSTMs.

About 5 hours of study

Learning objectives

Lessons

  1. 1

    Convolutions Advanced

    Slide a small kernel of weights across an image to detect local patterns such as edges. Compute convolutions by hand and in code, control the output size with padding and stride, and find the gradients a convolution layer needs for training.

  2. 2

    Pooling and convolutional networks Advanced

    Build a whole convolutional network: convolution layers with ReLU, pooling layers that shrink feature maps, then flattening and dense layers for the final decision. Track shapes and parameter counts through the network.

  3. 3

    Recurrent networks and LSTMs Advanced

    Process sequences such as text, sound and time series with a hidden state carried from step to step. Run a recurrent network, backpropagate through time, see why gradients vanish over long sequences, and how LSTM gates fix it.

Module practice

Module quiz

1. How many outputs does a valid convolution of 8 inputs with a kernel of 3 give?
2. What does 2 × 2 max pooling with stride 2 do to a 10 × 10 map?
3. What is the output width for width 28, kernel 3, padding 1, stride 1?
4. How many parameters does a conv layer with 4 kernels of 3 × 3 on 2 channels have?
5. Why are an RNN's weight gradients summed over time?
6. How does an LSTM keep information for many steps?

‹ Making training workAttention and transformers ›