Resizing and interpolation

Intermediate

Introduction

Every image you see online has been resized: thumbnails are shrunk, photos are fitted to the screen, and neural networks demand inputs of exactly 224 × 224 or 640 × 640 pixels. Resizing means creating pixels at positions that did not exist in the original, so it needs interpolation: a rule for estimating values between known pixels. The rule you choose decides whether the result looks blocky, smooth or full of strange patterns.

How it works

Work backwards. For an output pixel at column d, ask: which position x in the source image does it correspond to? Then sample the source there. Going this way (output to input) guarantees that every output pixel gets exactly one value, with no gaps.

Nearest neighbour. The simplest mapping is x = d × (source width ÷ output width), rounded down; in whole numbers, d * sw // w. Each output pixel copies one source pixel, so enlarging repeats pixels in blocks. It is very fast and keeps exact values, which is what you want for label masks, where blending label 1 and label 3 into 2 would be wrong.

Pixel-centre alignment. Think of source pixel i as a small square from i to i + 1 with its centre at i + 0.5. The centre of output pixel d sits at (d + 0.5) × scale in source units, where scale = source size ÷ output size, so its position in pixel indices is

x = (d + 0.5) × scale − 0.5

Near the borders x can fall below 0 or above size − 1; clamp it into the image. This is the convention OpenCV uses, and it keeps the image from shifting by half a pixel.

Bilinear interpolation. If x lies between columns x₀ and x₀ + 1, with fraction f = x − x₀, blend the two: (1 − f) × left + f × right. Do this on the two rows around y, then blend those two results by the vertical fraction. Four pixels contribute, weighted by how close they are. Bilinear results are smooth, but sharp edges become slightly soft. Bicubic interpolation uses 16 pixels and is a little sharper.

Shrinking and aliasing. When shrinking by a large factor, sampling one point per output pixel skips most of the input. Fine stripes or textures can then turn into coarse false patterns (aliasing, the moiré you see on photographed screens). The fix is to average all the source pixels each output pixel covers (area averaging), or to blur before sampling.

Example

src = [
    [0, 100, 200],
    [100, 200, 100],
    [200, 100, 0],
]


def nearest(img, h, w):
    sh, sw = len(img), len(img[0])
    return [[img[y * sh // h][x * sw // w] for x in range(w)] for y in range(h)]


def positions(n_out, n_in):
    # For each output pixel: the two source pixels around its centre and how far it lies between them.
    out = []
    for d in range(n_out):
        s = min(max((d + 0.5) * n_in / n_out - 0.5, 0.0), n_in - 1.0)
        i = int(s)
        out.append((i, min(i + 1, n_in - 1), s - i))
    return out


def bilinear(img, h, w):
    xs = positions(w, len(img[0]))
    out = []
    for y0, y1, fy in positions(h, len(img)):
        top, bottom = img[y0], img[y1]
        out.append([int((top[a] * (1 - f) + top[b] * f) * (1 - fy) + (bottom[a] * (1 - f) + bottom[b] * f) * fy + 0.5) for a, b, f in xs])
    return out


print("where the 6 new columns sample the old 3:", positions(6, 3))
print("nearest neighbour, 3 x 3 to 6 x 6:")
for row in nearest(src, 6, 6):
    print("  ", row)
print("bilinear, 3 x 3 to 6 x 6:")
for row in bilinear(src, 6, 6):
    print("  ", row)
Open in compiler

Line by line

  1. src is a 3 × 3 image with a bright diagonal from the top-right to the bottom-left.
  2. nearest maps each output index with y * sh // h, so each source pixel becomes a 2 × 2 block.
  3. positions applies (d + 0.5) × 3/6 − 0.5 and clamps: the six new columns sit at 0 (clamped from −0.25), 0.25, 0.75, 1.25, 1.75 and 2 (clamped from 2.25).
  4. Each position is stored as the left index, the right index and the fraction between them, so the bilinear loop is just two blends.
  5. Bilinear output row 0 goes 0, 25, 75, 125, 175, 200: a smooth ramp instead of the nearest-neighbour steps 0, 0, 100, 100, 200, 200.
  6. Adding 0.5 before int rounds each blended value to the nearest whole number (half values such as 137.5 round up to 138).

Expected result

where the 6 new columns sample the old 3: [(0, 1, 0.0), (0, 1, 0.25), (0, 1, 0.75), (1, 2, 0.25), (1, 2, 0.75), (2, 2, 0.0)]
nearest neighbour, 3 x 3 to 6 x 6:
   [0, 0, 100, 100, 200, 200]
   [0, 0, 100, 100, 200, 200]
   [100, 100, 200, 200, 100, 100]
   [100, 100, 200, 200, 100, 100]
   [200, 200, 100, 100, 0, 0]
   [200, 200, 100, 100, 0, 0]
bilinear, 3 x 3 to 6 x 6:
   [0, 25, 75, 125, 175, 200]
   [25, 50, 100, 138, 163, 175]
   [75, 100, 150, 163, 138, 125]
   [125, 138, 163, 150, 100, 75]
   [175, 163, 138, 100, 50, 25]
   [200, 175, 125, 75, 25, 0]

In the real world

Phones resize photos for previews, web pages request images at several sizes, and every image classifier resizes its input. Training data is often resized with bilinear interpolation, while segmentation masks use nearest neighbour. In OpenCV (shown as text; it does not run on this site), note that the size is given as (width, height):

import cv2

big = cv2.resize(img, (640, 480), interpolation=cv2.INTER_LINEAR)
blocky = cv2.resize(mask, (640, 480), interpolation=cv2.INTER_NEAREST)
small = cv2.resize(img, (160, 120), interpolation=cv2.INTER_AREA)   # averages: best for shrinking

Common mistakes

  • Mapping forwards (input to output), which leaves holes when enlarging.
  • Ignoring pixel centres, which shifts the whole image by half a pixel.
  • Interpolating label masks, which invents labels that do not exist; use nearest neighbour.
  • Shrinking a detailed image by sampling single points, which causes aliasing.
  • Passing (height, width) to a function that expects (width, height).

Best practices

  • Enlarge with bilinear or bicubic; shrink with area averaging.
  • Use nearest neighbour only for masks, labels and pixel art.
  • Keep the aspect ratio unless you mean to stretch; pad if a fixed size is required.
  • Check a resized image's size and a few values against what you expect.

Summary

Resizing maps every output pixel back to a source position, x = (d + 0.5) × scale − 0.5 with pixel-centre alignment, clamped at the borders. Nearest neighbour copies the closest pixel and gives blocks; bilinear blends the four surrounding pixels by distance and gives smooth results. Shrinking should average what it drops, otherwise aliasing creates false patterns.

Quick check

1. Why does resizing map output pixels back to the input rather than input pixels forward?
2. A position falls at x = 2.25 between pixels with values 40 and 80. What does linear interpolation give?
3. Which interpolation should be used to resize a segmentation mask of class labels?
4. What causes aliasing when shrinking an image?

Key takeaways

  • Resizing works backwards: for each output pixel, find its position in the source image and sample there.
  • Bilinear interpolation blends the four nearest pixels and gives smooth results; nearest neighbour keeps hard edges and blocks.
  • Shrinking should average the pixels it drops, or fine detail turns into aliasing.

Practice problems for this lesson

My notes

Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.