Deep Learning

Neurons, backpropagation, optimizers, normalization, CNNs, RNNs, embeddings, attention and transformers, every network built from scratch in plain Python.

Level
Advanced
Time
About 25 hours
Modules
5
Lessons
15
Exercises
45

Deep learning trains networks of simple units to recognize images, understand speech and generate text. This course builds it from the ground up: a single neuron, losses and gradient descent, backpropagation, multi-layer networks and softmax classifiers, the optimizers and normalization layers that make training work, convolutional and recurrent networks, and finally embeddings, attention and the transformer.

There are no libraries and nothing is hidden. Every forward pass, gradient and update is plain Python that runs in your browser, on networks small enough to follow by hand. When you later use PyTorch or TensorFlow, you will know what each layer and each line of the training loop is doing.

What you will learn

  • Compute neurons, layers and whole networks by hand and in code
  • Derive and implement backpropagation and gradient descent
  • Train classifiers with softmax, cross-entropy and modern optimizers
  • Keep training stable with initialization, normalization and regularization
  • Implement convolutions, pooling and recurrent cells, with their gradients
  • Implement embeddings, attention and the parts of a transformer

Skills you will gain

  • Forward and backward passes
  • Optimization and training loops
  • Initialization, normalization and regularization
  • Convolutional and recurrent networks
  • Embeddings, attention and transformers

Before you start

  • Python: functions, lists, loops and classes (the Python course)
  • Comfort with algebra; derivatives are explained as they are needed
  • The Machine Learning course helps but is not required

Where it is used

  • Image classification and detection
  • Speech and time-series models
  • Language models and text generation
  • Recommendation and search with embeddings

Modules

Module 1: Neural network basics

Neurons and activation functions, losses and gradient descent, and backpropagation through computation graphs.

  1. 1

    Neurons, layers and activation functions Intermediate

    The building block of every neural network: a weighted sum of inputs plus a bias, passed through an activation function. Compute neurons and whole layers by hand and in Python, and compare sigmoid, tanh and ReLU.

  2. 2

    Loss functions and gradient descent Intermediate

    Measure how wrong a model is with a loss function, and reduce the loss by repeatedly stepping the parameters downhill along the gradient. Use mean squared error and cross-entropy, choose a learning rate, and check gradients numerically.

  3. 3

    Backpropagation Advanced

    Compute every gradient in a network efficiently by applying the chain rule backwards through the computation. Draw computation graphs, propagate gradients by hand, and build a tiny automatic-differentiation engine.

Module practice and quiz →

Module 2: Training networks

Multi-layer networks and the training loop, softmax classifiers with cross-entropy, and optimizers from momentum to Adam.

  1. 1

    Training a multi-layer network Intermediate

    Put neurons, losses, gradient descent and backpropagation together to train a network with a hidden layer. Solve XOR, which no single neuron can, and understand epochs, batches and why hidden layers matter.

  2. 2

    Softmax and multi-class classification Intermediate

    Turn a network's raw scores into probabilities over many classes with softmax, train with cross-entropy, and use the simple gradient that results. Compute softmax safely for large numbers.

  3. 3

    Optimizers: momentum, RMSProp and Adam Advanced

    Plain gradient descent zigzags in narrow valleys and crawls on flat ground. Learn how momentum, adaptive learning rates and Adam fix this, use mini-batches, and schedule the learning rate during training.

Module practice and quiz →

Module 3: Making training work

Initialization and vanishing gradients, batch and layer normalization, and regularization against overfitting.

  1. 1

    Initialization and vanishing gradients Advanced

    Why the starting values of the weights matter: signals and gradients shrink or explode as they pass through many layers. Use Xavier and He initialization, and see how activation functions affect gradient flow.

  2. 2

    Batch normalization and layer normalization Advanced

    Keep the values inside a network on a steady scale by normalizing them: subtract a mean, divide by a standard deviation, then rescale with learned parameters. Compare batch norm and layer norm, and handle training and inference.

  3. 3

    Overfitting and regularization Advanced

    A model that memorizes its training data fails on new data. Detect overfitting with a validation set, and prevent it with L2 weight decay, dropout and early stopping.

Module practice and quiz →

Module 4: Convolutional and recurrent networks

Convolutions with padding, stride and gradients, pooling and complete CNNs, and recurrent networks, backpropagation through time and LSTMs.

  1. 1

    Convolutions Advanced

    Slide a small kernel of weights across an image to detect local patterns such as edges. Compute convolutions by hand and in code, control the output size with padding and stride, and find the gradients a convolution layer needs for training.

  2. 2

    Pooling and convolutional networks Advanced

    Build a whole convolutional network: convolution layers with ReLU, pooling layers that shrink feature maps, then flattening and dense layers for the final decision. Track shapes and parameter counts through the network.

  3. 3

    Recurrent networks and LSTMs Advanced

    Process sequences such as text, sound and time series with a hidden state carried from step to step. Run a recurrent network, backpropagate through time, see why gradients vanish over long sequences, and how LSTM gates fix it.

Module practice and quiz →

Module 5: Attention and transformers

Embeddings and similarity, scaled dot-product and causal self-attention, and the transformer with positional encodings, parameter counts and decoding.

  1. 1

    Embeddings Advanced

    Represent words and other symbols as dense vectors whose directions carry meaning. Compare embeddings with cosine similarity, find nearest neighbours, solve analogies with vector arithmetic, and see how embedding tables are learned.

  2. 2

    Attention Advanced

    Let every position in a sequence look at every other position and take a weighted average of what it finds. Compute scaled dot-product attention, use self-attention and causal masks, and see why attention replaced recurrence.

  3. 3

    Transformers Advanced

    Assemble the architecture behind modern language models: token embeddings plus positional encodings, then blocks of multi-head self-attention and feed-forward layers with residual connections and layer normalization, and finally next-token prediction and decoding.

Module practice and quiz →

Projects

Final assessment →

What to learn next