Pooling and convolutional networks

Advanced

Introduction

One convolution finds simple patterns such as edges. A convolutional neural network stacks many: early layers find edges, middle layers combine them into textures and shapes, and late layers recognise parts and whole objects. Pooling layers between them shrink the feature maps, and dense layers at the end turn the features into class scores. This is the architecture of classic image classifiers from LeNet to VGG and ResNet.

How it works

Max pooling slides a window, usually 2 × 2 with stride 2, over a feature map and keeps the largest value in each window; average pooling keeps the mean. Pooling has no weights. It halves the width and height, so later layers are cheaper, and because a maximum does not care exactly where in its window a feature appeared, the network becomes tolerant to small shifts of the object.

A convolution layer with F kernels reads C input channels and outputs F channels. Each kernel spans all C channels, so it has C × k × k weights plus one bias:

parameters = F × (C × k × k) + F

and the output has F channels of size ⌊(n + 2p − k) / s⌋ + 1.

A typical network looks like:

  1. conv → ReLU → pool, repeated, with more channels and smaller maps each time;
  2. flatten the final maps into a vector of length channels × height × width;
  3. dense layers ending in a softmax over the classes.

Each value deep in the network depends on a growing patch of the image, its receptive field: a stack of 3 × 3 convolutions grows it by 2 pixels per layer, and each pooling doubles it. That is how small kernels end up recognizing large objects.

Syntax

def max_pool(fmap, k=2, s=2):
    rows = (len(fmap) - k) // s + 1
    cols = (len(fmap[0]) - k) // s + 1
    return [[max(fmap[i * s + a][j * s + b] for a in range(k) for b in range(k))
             for j in range(cols)] for i in range(rows)]

Example

image = [
    [0, 0, 0, 0, 0, 0],
    [0, 9, 9, 9, 9, 0],
    [0, 9, 0, 0, 9, 0],
    [0, 9, 0, 0, 9, 0],
    [0, 9, 9, 9, 9, 0],
    [0, 0, 0, 0, 0, 0],
]
kernels = {"corner": [[1, 1], [1, -1]], "edge": [[1, -1], [1, -1]]}


def conv(img, k):
    n = len(img) - len(k) + 1
    return [[sum(img[i + a][j + b] * k[a][b] for a in range(2) for b in range(2)) for j in range(n)] for i in range(n)]


def relu(fmap):
    return [[max(0, v) for v in row] for row in fmap]


def max_pool(fmap, size=2):
    n = len(fmap) // size
    return [[max(fmap[i * size + a][j * size + b] for a in range(size) for b in range(size)) for j in range(n)] for i in range(n)]


features = []
for name, k in kernels.items():
    pooled = max_pool(relu(conv(image, k)))
    print(f"{name} filter after conv, ReLU and 2 x 2 max pooling:", pooled)
    features += [v for row in pooled for v in row]
print("flattened features:", features)
weights = {"square": [1] * 9 + [0] * 9, "blank": [-1] * 18}
scores = {label: sum(w * f for w, f in zip(ws, features)) for label, ws in weights.items()}
print("dense layer scores:", scores, "-> prediction:", max(scores, key=scores.get))
Open in compiler

Line by line

  1. The 6 × 6 image contains the outline of a square.
  2. Two hand-made 2 × 2 kernels look for corners and for vertical edges; each gives a 5 × 5 map.
  3. ReLU removes negative responses, and 2 × 2 max pooling shrinks each map to 2 × 2, keeping the strongest response in each quarter.
  4. Flattening joins the two pooled maps into 8 features.
  5. A dense layer scores "square" and "blank" from the features and predicts "square".

Expected result

corner filter after conv, ReLU and 2 x 2 max pooling: [[27, 18], [18, 9]]
edge filter after conv, ReLU and 2 x 2 max pooling: [[9, 0], [18, 0]]
flattened features: [27, 18, 18, 9, 9, 0, 18, 0]
dense layer scores: {'square': 99, 'blank': -99} -> prediction: square

In the real world

ResNet-50, a standard image model, has about 25 million parameters in 50 layers of convolutions with batch norm and residual connections. Phone apps use small CNNs such as MobileNet that replace standard convolutions with cheaper variants. Many vision models now use transformers, but convolutions remain common, alone or combined with attention.

Common mistakes

  • Losing track of shapes, so the dense layer expects the wrong number of inputs.
  • Forgetting that pooling has no parameters, or counting the bias per input channel instead of per kernel.
  • Pooling too early and too much, throwing away the detail needed.
  • Flattening in a different order from the one the dense weights were trained with.

Best practices

  • Write a table of every layer's output shape and parameter count before training.
  • Double the channels each time pooling halves the size, a common balanced design.
  • Use padding to keep sizes simple, such as 3 × 3 kernels with padding 1.

Summary

CNNs stack convolution, ReLU and pooling layers, then flatten into dense layers. Pooling summarizes windows by their maximum or average, shrinking maps and adding tolerance to small shifts. Shapes follow the size formula, and a convolution layer has F × (C × k × k) + F parameters.

Quick check

1. What does 2 × 2 max pooling with stride 2 do to a 16 × 16 feature map?
2. How many parameters does a pooling layer have?
3. How many parameters does a conv layer with 16 kernels of 3 × 3 on 8 channels have?
4. Why does max pooling add tolerance to small shifts?

Key takeaways

  • A typical CNN repeats convolution, ReLU and pooling, then flattens into dense layers.
  • Pooling halves the width and height, cutting computation and adding tolerance to small shifts.
  • A convolution layer with F kernels of size k × k on C channels has F × (C × k × k) + F parameters.

Practice problems for this lesson

My notes

Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.