Affine transforms: shift, scale, rotate and warp

Intermediate

Introduction

Straightening a scanned page, aligning two photos, augmenting training data with small rotations, or showing a map that turns with your phone: all of these move pixels to new positions. Most of these movements are affine transforms, which combine shifting, scaling, rotating and shearing. They are all described by the same six numbers, and this lesson shows how to build them, combine them and apply them to an image.

How it works

The matrix. An affine transform sends a point (x, y) to

x' = a x + b y + tx y' = c x + d y + ty

written as a 2 × 3 matrix [[a, b, tx], [c, d, ty]]. Special cases:

  • Translation by (tx, ty): [[1, 0, tx], [0, 1, ty]].
  • Scaling by (sx, sy): [[sx, 0, 0], [0, sy, 0]].
  • Rotation by θ about the origin: [[cos θ, −sin θ, 0], [sin θ, cos θ, 0]]. Because image y points down, a positive θ turns the picture clockwise on screen.
  • Shear: [[1, k, 0], [0, 1, 0]] slides each row sideways in proportion to y.

Composition. Add a third row [0, 0, 1] to get 3 × 3 matrices; then doing M₁ first and M₂ second is the single matrix M₂ × M₁. Order matters: scaling then shifting is not the same as shifting then scaling, because in the second case the shift gets scaled too.

Rotating about a centre. To rotate about (cx, cy) instead of the top-left corner: shift the centre to the origin, rotate, shift back: M = T(cx, cy) × R(θ) × T(−cx, −cy).

Inverse. For [[a, b, tx], [c, d, ty]] with det = a d − b c ≠ 0, the inverse has the 2 × 2 part [[d, −b], [−c, a]] / det and the translation −(that part) × (tx, ty).

Warping an image. The tempting way, moving each input pixel to its new position, leaves holes when the image is enlarged or rotated, because some output pixels receive nothing. Instead use backward mapping: for every output pixel (x, y), apply the inverse transform to find the source position, then sample it, with nearest neighbour (round to the closest pixel) or bilinear interpolation. Positions outside the source get a fill value, usually black.

Transforms that also make parallel lines meet, like photographing a page at an angle, need a 3 × 3 perspective matrix (a homography); the backward-mapping idea is the same.

Example

import math


def rotation(degrees, cx, cy):
    # Rotate about (cx, cy). With y pointing down, a positive angle turns the picture clockwise on screen.
    a = round(math.radians(degrees), 12)
    c, s = round(math.cos(a), 12), round(math.sin(a), 12)
    return [[c, -s, cx - c * cx + s * cy], [s, c, cy - s * cx - c * cy]]


def invert(M):
    (a, b, tx), (c, d, ty) = M
    det = a * d - b * c
    ia, ib, ic, id_ = d / det, -b / det, -c / det, a / det
    return [[ia, ib, -(ia * tx + ib * ty)], [ic, id_, -(ic * tx + id_ * ty)]]


def warp(img, M):
    # Backward mapping: for each output pixel, find where it comes from in the input.
    h, w = len(img), len(img[0])
    (a, b, tx), (c, d, ty) = invert(M)
    out = []
    for y in range(h):
        row = ""
        for x in range(w):
            sx = math.floor(a * x + b * y + tx + 0.5)  # nearest pixel
            sy = math.floor(c * x + d * y + ty + 0.5)
            row += img[sy][sx] if 0 <= sx < w and 0 <= sy < h else "."
        out.append(row)
    return out


img = [
    "#....",
    "#....",
    "#....",
    "###..",
    ".....",
]
M = rotation(90, 2, 2)
print("rotation matrix:", M)
x, y = 0, 3
print("pixel (x 0, y 3) moves to", (M[0][0] * x + M[0][1] * y + M[0][2], M[1][0] * x + M[1][1] * y + M[1][2]))
turned = warp(img, M)
moved = warp(img, [[1, 0, 2], [0, 1, -1]])
print("original   rotated 90   shifted right 2, up 1")
for a, b, c in zip(img, turned, moved):
    print("  ", a, "     ", b, "     ", c)
Open in compiler

Line by line

  1. rotation builds T(cx, cy) R(θ) T(−cx, −cy) directly. The cosine and sine are rounded to 12 decimal places, so cos 90° is exactly 0 rather than 6 × 10⁻¹⁷.
  2. For 90° about (2, 2) the matrix is [[0, −1, 4], [1, 0, 0]]: x' = 4 − y and y' = x.
  3. The bottom of the L's upright, at x 0 and y 3, moves to x 1, y 0: the left end of the top row.
  4. invert computes the inverse matrix from the determinant, here 1.
  5. warp loops over output pixels, maps each back with the inverse, rounds to the nearest pixel and copies it, or writes . if it falls outside.
  6. The L turns clockwise on screen: its upright becomes a bar along the top and its foot points down.
  7. The second warp is a plain translation, two pixels right and one up; the top row of the L falls off the image.

Expected result

rotation matrix: [[-0.0, -1.0, 4.0], [1.0, -0.0, 0.0]]
pixel (x 0, y 3) moves to (1.0, 0.0)
original   rotated 90   shifted right 2, up 1
   #....       .####       ..#..
   #....       .#...       ..#..
   #....       .#...       ..###
   ###..       .....       .....
   .....       .....       .....

In the real world

Phone scanners straighten documents, panorama apps align overlapping photos, and data augmentation rotates and shifts training images by small random amounts. In OpenCV (shown as text; it does not run on this site), getRotationMatrix2D uses the opposite sign convention, so a positive angle turns the picture counter-clockwise on screen; warpAffine takes the forward matrix and inverts it internally:

import cv2

h, w = img.shape[:2]
M = cv2.getRotationMatrix2D((w / 2, h / 2), 30, 1.0)   # 30 degrees counter-clockwise, scale 1
turned = cv2.warpAffine(img, M, (w, h))                 # size as (width, height)
back = cv2.invertAffineTransform(M)

Common mistakes

  • Composing in the wrong order. M₂ × M₁ applies M₁ first.
  • Rotating about the corner when you meant the centre.
  • Forward mapping an image, which leaves holes.
  • Mixing angle conventions between your maths and a library's, so rotations go the wrong way.
  • Using degrees in math.cos, which expects radians.

Best practices

  • Build transforms from named pieces (translate, rotate, scale) and multiply them, rather than typing six numbers.
  • Test a transform on a few points, such as the corners, before warping a whole image.
  • Use backward mapping with bilinear sampling for photos and nearest neighbour for masks.
  • Keep the inverse matrix if you will need to map results back to the original image.

Summary

An affine transform is a 2 × 3 matrix that combines translation, scaling, rotation and shear. Composition is matrix multiplication, with the first transform on the right, and rotation about a centre is shift, rotate, shift back. Images are warped by backward mapping: each output pixel looks up its source position with the inverse matrix, so no pixel is left empty.

Quick check

1. Which matrix moves every point 3 to the right and 2 down?
2. You scale by 2 and then translate by (5, 0). What is the combined matrix's translation?
3. Why are images warped by backward mapping?
4. How do you rotate an image about its centre (cx, cy)?

Key takeaways

  • One 2 × 3 matrix describes any combination of shift, scale, rotation and shear.
  • To compose, multiply 3 × 3 matrices with the transform applied first on the right: M = M₂ M₁.
  • Warp images backwards with the inverse matrix, so every output pixel gets a value.

Practice problems for this lesson

My notes

Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.