Thresholding and Otsu's method

Intermediate

Introduction

Many vision tasks need a simple answer for every pixel: is this part of the object or part of the background? A scanner wants ink versus paper, a microscope program wants cells versus fluid, a factory camera wants a part versus the conveyor belt. Thresholding gives that answer by comparing each pixel with a level. It is the simplest form of segmentation, and with a well-chosen threshold it is fast and remarkably effective.

How it works

Global threshold. Pick a level T. Pixels brighter than T become foreground (1, or 255, or #), the rest background:

mask(r, c) = 1 if img(r, c) > T else 0

An inverse threshold does the opposite, marking pixels at or below T. Use it for dark objects on a light background, such as text on paper.

Choosing T with Otsu's method. If the histogram has two humps, one for the background and one for the object, a good T lies in the valley between them. Otsu's method (1979) finds it automatically. For every candidate T it splits the pixels into a dark class (≤ T) and a bright class (> T), with n₀ and n₁ pixels and mean brightness μ₀ and μ₁, and computes the between-class variance

σ²_between = (n₀ / N) (n₁ / N) (μ₀ − μ₁)²

This is large when both classes are substantial and their means are far apart. Otsu keeps the T with the largest value. (Equivalently, it minimises the spread inside each class.) Only levels that actually occur need to be tried, and with running sums each candidate costs a few additions.

Adaptive thresholds. A photo of a page lit from one side has bright paper on the left and grey paper on the right; the ink on the left may be brighter than the paper on the right, so no single T works. An adaptive threshold compares each pixel with its own neighbourhood instead: a pixel is ink if it is darker than the mean of the k × k window around it by more than a constant C. Slow changes in lighting affect the pixel and its neighbours alike, so they cancel out.

Cleaning the mask. Thresholds leave specks and small holes. The next lesson removes them with morphology, and the one after counts and measures the remaining objects.

Example

img = [
    [20, 25, 30, 30, 25, 20],
    [25, 160, 170, 180, 30, 25],
    [30, 170, 200, 190, 35, 30],
    [30, 180, 190, 170, 30, 25],
    [25, 30, 35, 30, 25, 20],
    [20, 25, 30, 25, 20, 20],
]
counts = {}
for row in img:
    for v in row:
        counts[v] = counts.get(v, 0) + 1
levels = sorted(counts)
N = sum(counts.values())
total = sum([v * n for v, n in counts.items()])
print("histogram:", {v: counts[v] for v in levels})

best_t, best_var = None, -1.0
n0 = s0 = 0
for t in levels[:-1]:
    n0 += counts[t]
    s0 += t * counts[t]
    n1, s1 = N - n0, total - s0
    var = n0 * n1 * (s0 / n0 - s1 / n1) ** 2 / N ** 2
    print(f"  t = {t:3}: dark {n0:2} pixels, mean {s0 / n0:6.2f}; bright {n1:2}, mean {s1 / n1:6.2f}; variance {var:8.2f}")
    if var > best_var:
        best_t, best_var = t, var
print("Otsu threshold:", best_t)
for row in img:
    print("  ", "".join(["#" if v > best_t else "." for v in row]))
Open in compiler

Line by line

  1. The 6 × 6 image is a bright object (160 to 200) on a dark background (20 to 35).
  2. counts is the histogram: how many pixels have each level. There are nine distinct levels in two clear groups.
  3. The loop tries each level as T, keeping running totals n0 and s0 for the dark class; the bright class is whatever is left.
  4. For each T it prints both classes' sizes and means and the between-class variance.
  5. The variance peaks at T = 35, where the dark class is exactly the background (mean 26.48) and the bright class exactly the object (mean 178.89).
  6. Thresholds inside the object (160 or more) put object pixels into the dark class, so the means move closer and the variance falls.
  7. The mask marks the 3 × 3 object with #.

Expected result

histogram: {20: 6, 25: 9, 30: 10, 35: 2, 160: 1, 170: 3, 180: 2, 190: 2, 200: 1}
  t =  20: dark  6 pixels, mean  20.00; bright 30, mean  73.50; variance   397.53
  t =  25: dark 15 pixels, mean  23.00; bright 21, mean  94.29; variance  1235.12
  t =  30: dark 25 pixels, mean  25.80; bright 11, mean 152.73; variance  3418.52
  t =  35: dark 27 pixels, mean  26.48; bright  9, mean 178.89; variance  4355.25
  t = 160: dark 28 pixels, mean  31.25; bright  8, mean 181.25; variance  3888.89
  t = 170: dark 31 pixels, mean  44.68; bright  5, mean 188.00; variance  2456.72
  t = 180: dark 33 pixels, mean  52.88; bright  3, mean 193.33; variance  1506.96
  t = 190: dark 35 pixels, mean  60.71; bright  1, mean 200.00; variance   523.93
Otsu threshold: 35
   ......
   .###..
   .###..
   .###..
   ......
   ......

In the real world

Document scanners, barcode readers and licence-plate readers all threshold before reading. Medical software thresholds X-rays to separate bone from tissue. In OpenCV (shown as text; it does not run on this site), passing THRESH_OTSU makes threshold ignore the given level and compute Otsu's:

import cv2

t, mask = cv2.threshold(grey, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
ink = cv2.adaptiveThreshold(grey, 255, cv2.ADAPTIVE_THRESH_MEAN_C,
                            cv2.THRESH_BINARY_INV, 11, 2)   # 11 x 11 window, C = 2

Common mistakes

  • Using a fixed T for every image. Lighting changes, and the threshold must follow it.
  • Using Otsu on a histogram with one hump. It still returns a T, but the split is meaningless.
  • Forgetting the inverse threshold for dark objects on a light background.
  • Using a global threshold on unevenly lit pages, which turns whole regions black.
  • Choosing an adaptive window smaller than the objects, which marks only their edges.

Best practices

  • Look at the histogram: two clear humps mean a global threshold will work.
  • Blur slightly before thresholding to reduce speckle.
  • Use adaptive thresholds for documents and photos with shadows; make the window larger than a character or object.
  • Clean the mask with morphology afterwards.

Summary

A threshold turns a greyscale image into a binary mask: pixels above T are foreground, or below T for an inverse threshold. Otsu's method tries every level and keeps the one with the largest between-class variance, which is the valley between two histogram humps. When lighting varies across the image, an adaptive threshold compares each pixel with its local mean minus a constant.

Quick check

1. What does Otsu's method maximise?
2. Which threshold suits dark text on white paper?
3. Why can a single threshold fail on a page lit from one side?
4. An adaptive threshold compares each pixel with...

Key takeaways

  • A threshold turns brightness into a yes/no mask; inverse thresholds mark dark objects instead of bright ones.
  • Otsu tries every threshold and keeps the one that best separates the histogram into two groups.
  • Uneven lighting defeats any single threshold; comparing each pixel with its local mean fixes that.

Practice problems for this lesson

My notes

Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.