Introduction
A classifier says "there is a dog". A self-driving car needs more: where every pedestrian is, how big, and which pixels belong to the road. Object detection finds objects and draws a bounding box around each one; segmentation labels every pixel. Modern systems use deep networks for both, but they all rely on a few simple calculations to clean up their output and to measure how good they are. Those calculations are the subject of this lesson.
How it works
Boxes and IoU. A box is (x1, y1, x2, y2), its top-left and bottom-right corners. Two boxes overlap in the rectangle from max(x1) to min(x2) and from max(y1) to min(y2); if either width is negative, the overlap is 0. The intersection over union is
IoU = overlap area ÷ (area A + area B − overlap area)
It is 1 for identical boxes and 0 for boxes that do not touch. A box shifted by a tenth of its size still scores about 0.8, while half-overlapping boxes score only about 0.33, so IoU is a strict measure of agreement.
Non-maximum suppression. A detector scores thousands of candidate boxes, and a real object usually gets several high-scoring, overlapping boxes. Greedy NMS sorts boxes by score, keeps the best one, removes every remaining box whose IoU with it is above a threshold (often 0.5), and repeats with the next surviving box. In practice it runs separately for each class.
Scoring a detector. Compare predictions with the ground truth (boxes drawn by people). Go through predictions from the highest score down. A prediction is a true positive if it overlaps a ground-truth box that has not been matched yet with IoU at least the threshold (0.5 in the PASCAL VOC benchmark); that box is then used up. Otherwise it is a false positive: a wrong place, or a duplicate of an object already found. After each prediction:
- precision = true positives so far ÷ predictions so far (how many guesses were right);
- recall = true positives so far ÷ ground-truth objects (how many objects were found).
Lowering the score threshold raises recall but usually lowers precision. Average precision (AP) summarises the whole curve: first make precision non-increasing by replacing each value with the best precision at that recall or higher, then add up precision × each increase in recall. mAP averages AP over classes; the COCO benchmark also averages over IoU thresholds from 0.5 to 0.95.
Segmentation. Semantic segmentation labels each pixel with a class (road, car, sky). Instance segmentation also separates objects of the same class (car 1, car 2). Masks are compared with the same idea as boxes: mask IoU = overlapping pixels ÷ pixels in either mask, and the Dice coefficient = 2 × overlap ÷ (pixels in A + pixels in B), popular in medical imaging. Dice is always at least as large as IoU for the same pair of masks.
Example
def iou(a, b):
w = min(a[2], b[2]) - max(a[0], b[0])
h = min(a[3], b[3]) - max(a[1], b[1])
inter = max(0, w) * max(0, h)
union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter
return inter / union
# (x1, y1, x2, y2), score: three guesses at a cat, two at a dog and one stray box
boxes = [
((10, 10, 50, 50), 0.9),
((12, 14, 52, 54), 0.8),
((60, 20, 90, 60), 0.75),
((8, 8, 48, 46), 0.7),
((62, 22, 92, 58), 0.6),
((30, 70, 40, 80), 0.3),
]
print("IoU of box 0 and box 1:", round(iou(boxes[0][0], boxes[1][0]), 3))
order = sorted(range(len(boxes)), key=lambda i: -boxes[i][1])
kept = []
for i in order:
if all([iou(boxes[i][0], boxes[j][0]) <= 0.5 for j in kept]):
kept.append(i)
print("kept after non-maximum suppression:", kept)
truth = ["..##..", ".####.", ".####.", "..##.."]
guess = ["......", ".###..", ".####.", ".####."]
both = sum([a == b == "#" for ra, rb in zip(truth, guess) for a, b in zip(ra, rb)])
t = sum([row.count("#") for row in truth])
g = sum([row.count("#") for row in guess])
print(f"mask overlap {both}, truth {t}, guess {g}: IoU {both / (t + g - both):.3f}, Dice {2 * both / (t + g):.3f}")Line by line
ioucomputes the overlap width and height, clamps negatives to 0, and divides by the union.- Six scored boxes: three around one object, two around another and one weak stray box.
- Boxes 0 and 1 are slightly shifted copies and have IoU 0.747.
- NMS visits boxes by decreasing score. Box 0 is kept; box 1 overlaps it too much; box 2 is on the other object and is kept; box 3 overlaps box 0; box 4 overlaps box 2; box 5 overlaps nothing and is kept.
- In a real pipeline the weak box 5 would also be dropped by a minimum score, which is a separate setting.
- The two 4 × 6 masks share 9 pixels; the truth has 12 and the guess 11, so IoU = 9 ÷ 14 = 0.643 and Dice = 18 ÷ 23 = 0.783.
Expected result
IoU of box 0 and box 1: 0.747
kept after non-maximum suppression: [0, 2, 5]
mask overlap 9, truth 12, guess 11: IoU 0.643, Dice 0.783
In the real world
Two-stage detectors (Faster R-CNN, 2015) propose regions and then classify them; one-stage detectors (YOLO, 2016, and SSD) predict boxes directly and are fast enough for video. U-Net (2015) is a classic segmentation network, especially in medicine, and Mask R-CNN (2017) adds a mask branch to Faster R-CNN for instance segmentation. All of them use IoU, NMS and AP. In PyTorch's torchvision (shown as text; it does not run on this site):
import torch
from torchvision.ops import box_iou, nms
boxes = torch.tensor([[10, 10, 50, 50], [12, 14, 52, 54], [60, 20, 90, 60]], dtype=torch.float)
scores = torch.tensor([0.9, 0.8, 0.75])
print(box_iou(boxes, boxes)) # 3 x 3 matrix of IoUs
print(nms(boxes, scores, 0.5)) # indices kept, highest score first: tensor([0, 2])
Common mistakes
- Mixing box formats: (x1, y1, x2, y2) versus (x, y, width, height) gives wrong IoUs without any error.
- Forgetting to clamp a negative overlap to 0, which gives a negative intersection.
- Running NMS across classes, so a person box suppresses an overlapping bicycle box.
- Counting duplicates as correct: each ground-truth object can be matched only once.
- Comparing AP numbers computed at different IoU thresholds or with different benchmark rules.
Best practices
- Convert all boxes to one format as soon as you load them, and write it down.
- Tune the NMS threshold and the minimum score separately; crowded scenes need a higher NMS threshold.
- Report precision, recall and AP together, and say which IoU threshold you used.
- For segmentation, report IoU or Dice per class, since small classes are easily hidden by an average.
Summary
IoU divides the overlap of two boxes or masks by their union and measures agreement from 0 to 1. Greedy NMS keeps the highest-scoring box and removes boxes that overlap it too much. A detector is scored by matching predictions in score order to unmatched ground truth with enough IoU, giving precision, recall and the area under their curve, AP. Segmentation labels pixels and is scored with mask IoU or Dice.
Quick check
Key takeaways
- IoU measures how well two boxes or masks overlap, from 0 (none) to 1 (identical).
- Detectors produce many overlapping guesses; NMS keeps the best box for each object.
- A detection counts as correct only if it overlaps an unmatched ground-truth object enough; AP summarises precision over all recall levels.
Practice problems for this lesson
My notes
Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.