Computer Vision
Topic 4 ยท Computer Vision ยท โฑ about 8 h

Object Detection

Boxes and their formats, IoU, anchors and one- vs two-stage detectors, non-maximum suppression, mean average precision, and the modern YOLO and DETR families.

Show:
In this topic
Beginner

Boxes, IoU and what a detector outputs

A detector outputs, for each object, a bounding box, a class and a confidence score. Boxes come in several formats, a constant source of bugs: corners (x1, y1, x2, y2) ("Pascal VOC"), (x, y, w, h) from the top-left ("COCO"), or normalised centre (cx, cy, w, h) ("YOLO"). Always check which one a dataset and a library use.

The basic measure of how well two boxes match is intersection over union:

IoU(A,B)=area(AโˆฉB)area(AโˆชB)โˆˆ[0,1].\text{IoU}(A,B)=\frac{\text{area}(A\cap B)}{\text{area}(A\cup B)}\in[0,1].

A prediction usually counts as correct if IoU with a true box of the same class is at least 0.5 (stricter benchmarks average over 0.5 to 0.95).

Beginner

Watch it: From hundreds of boxes to one per object: scores, IoU and NMS

A short narrated animation. Press โ–ถ Play, or step through it with โฎ โญ. Tick ๐Ÿ”Š narrate to hear each step.

Interactive: this part runs in your browser.

Beginner

How detectors work

A detector is a CNN or transformer backbone (often pretrained on ImageNet) plus a neck that combines features at several scales (a feature pyramid, so small and large objects are both visible) and a head that predicts boxes and classes.

  • Two-stage (Faster R-CNN): first propose candidate regions, then classify and refine each. Accurate, slower.
  • One-stage (YOLO, SSD, RetinaNet): predict boxes densely at every location of the feature maps in one pass. Fast; modern YOLO versions are both fast and accurate, and the usual first choice in industry.
  • Anchors: older detectors predict offsets from preset boxes of several sizes and shapes at each location; many newer ones are anchor-free, predicting the object's centre and size directly.
  • DETR (detection transformer) treats detection as predicting a set of boxes, matched one-to-one to the true objects, so it needs no NMS.
Beginner

Non-maximum suppression

Dense detectors produce many overlapping boxes for the same object. Non-maximum suppression (NMS) cleans them up: sort boxes by score; keep the highest; remove every remaining box that overlaps it with IoU above a threshold (often 0.5); repeat with the next highest. NMS runs per class. Too low a threshold merges neighbouring objects (touching cells become one); too high leaves duplicates.

Try it

Try it: IoU and NMS

A detector found many candidate boxes for three touching cells. Change the score and IoU thresholds and watch which boxes survive NMS.

Interactive: this part runs in your browser.

Beginner

Evaluating detectors: mAP

Detection is evaluated with average precision (AP). For one class: sort all predicted boxes over the whole test set by score; walk down the list marking each as a true positive (matches an unmatched true box with IoU โ‰ฅ 0.5) or false positive; compute precision and recall after each; AP is the area under the precision-recall curve. mAP averages AP over classes; COCO's "mAP@[.5:.95]" also averages over IoU thresholds from 0.5 to 0.95, rewarding tight boxes.

For a real project, also report what users care about: recall at the operating threshold, false positives per image, and performance on small objects and per site or device.

Try it

Code example: IoU, NMS and average precision

A worked example: read it step by step (every part is explained), then run it and experiment.

Interactive: this part runs in your browser.

Code

Code: fine-tune and evaluate a YOLO detector

Ultralytics in a few lines (notebook 10 uses torchvision's Faster R-CNN to show the internals).

Python ยท Ultralytics
from ultralytics import YOLO
model = YOLO("yolov8n.pt")                    # small pretrained detector
model.train(data="forms.yaml", epochs=50, imgsz=1024, batch=16)   # forms.yaml lists image folders + class names
metrics = model.val()                         # mAP50, mAP50-95, per-class AP
print(metrics.box.map50, metrics.box.map)
results = model.predict("new_form.jpg", conf=0.25, iou=0.5)       # score threshold and NMS IoU
for b in results[0].boxes: print(b.xyxy.tolist(), b.cls.item(), b.conf.item())
This code needs Ultralytics, which cannot run inside a web page. To run it, press Copy, then paste it into a new notebook in Google Colab โ†— (free; Runtime โ†’ Change runtime type โ†’ GPU for deep learning).
Check yourself

Quiz

1. Box A = (0, 0, 2, 2), box B = (1, 1, 3, 3) in corner format. IoU?

2. A dataset uses (x, y, w, h) and your code assumes (x1, y1, x2, y2). The symptom isโ€ฆ

3. What does NMS remove?

4. Touching cells are being merged into one detection. Which change helps?

5. COCO mAP@[.5:.95] differs from [email protected] because itโ€ฆ

6. Which family is typically the first choice for fast industrial detection?

Go deeper

๐Ÿ“š Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready