Computer Vision: Basic

Feature Points and Camera Calibration

Overview

Feature Detection Harris, SIFT, ORB Corner Detection Scale Invariance Feature Description Descriptor BRIEF, FREAK Rotation Invariance Feature Matching Nearest Neighbor Search RANSAC Outlier Rejection Camera Calibration Checkerboard Intrinsic / Extrinsic Params Lens Distortion Correction Find corresponding points across images and estimate the geometric properties of the camera

The Basic level covers methods for detecting distinctive points in images and establishing correspondences between images. It also treats calibration techniques for estimating the intrinsic and extrinsic parameters of a camera.

These techniques form the foundation of applications such as 3D reconstruction, SLAM, image stitching (panorama creation), and augmented reality (AR). Recovering geometric relationships from feature correspondences is the starting point for many advanced tasks.

Learning Objectives

  • Understand the major feature detectors (Harris, SIFT, ORB)
  • Understand the role and types of feature descriptors
  • Understand robust estimation with RANSAC
  • Be able to perform camera calibration

Table of Contents

  1. Chapter 1 Corner Detection

    Harris corner detection, Shi-Tomasi

  2. Chapter 2 SIFT Features

    Scale space, DoG, descriptors

  3. Chapter 3 ORB and Binary Features

    FAST, BRIEF, ORB

  4. Chapter 4 Feature Matching

    Brute-Force, FLANN, ratio test

  5. Chapter 5 RANSAC

    Robust estimation, outlier rejection

  6. Chapter 6 Camera Calibration

    Zhang's method, distortion coefficients

  7. Chapter 7 Lens Distortion Correction

    Radial distortion, tangential distortion

  8. Chapter 8 Exercises

    Wrap-up of the Basic level

Supplementary Reading: 10 Illustrated Pieces

In addition to the 8 main chapters, we provide a collection of 10 columns on filtering, edges, pyramids, calibration, and distortion.

1.The Principle of Gaussian Smoothing
2.Median vs Bilateral
3.Sobel vs Canny
4.Line Detection with the Hough Transform
5.The Idea of Image Pyramids
6.Scale Selection in SIFT
7.Intuition for Homography
8.Reprojection Error and Calibration Evaluation
9.Lens Aberration and Distortion Models
10.Rolling Shutter and Motion Blur

Prerequisites

  • The content of Computer Vision: Introduction
  • Calculus (gradients, Hessian matrix)
  • Linear algebra (eigenvalue decomposition, SVD)
  • Probability (basic concepts; helpful for understanding RANSAC)

Basic Concepts

Harris Corner Detection

Construct the structure tensor $M$ from image gradients:

$$M = \begin{pmatrix} \displaystyle\sum I_x^2 & \displaystyle\sum I_x I_y \\ \displaystyle\sum I_x I_y & \displaystyle\sum I_y^2 \end{pmatrix}$$

Corner response function: $R = \det(M) - k(\text{trace}(M))^2$

Scale Space and DoG

By progressively blurring an image to build a Gaussian pyramid and taking the difference between adjacent blurred images (Difference of Gaussian, DoG), one detects candidate scale-invariant feature points. Through this mechanism, SIFT obtains feature points that are robust to scaling.

RANSAC

Randomly select a sample and estimate a model:

  1. Estimate a model from the minimal number of samples
  2. Count inliers (points that fit the model)
  3. Adopt the model with the most inliers

Lens Distortion Model

Radial distortion:

$$x' = x(1 + k_1 r^2 + k_2 r^4 + k_3 r^6)$$

$r^2 = x^2 + y^2$, where $k_i$ are the distortion coefficients

Reading Reading

Before diving into the equations, a story to relax and build intuition. We look at "why a computer picks corners as feature points" through the familiar analogy of sliding a window.

Frequently Asked Questions (FAQ)

Q1. How does Harris corner detection work?

It constructs the structure tensor $M$ from image gradients and evaluates the corner response function $R = \det(M) - k(\text{trace}(M))^2$. Points with large $R$ are those where the intensity changes significantly no matter which direction the window is shifted — that is, corners.

Q2. What is the difference between SIFT and ORB?

SIFT detects scale-invariant keypoints using scale space and DoG, and produces real-valued descriptors from gradient histograms. ORB combines FAST corner detection with a BRIEF-based binary descriptor and adds rotation invariance. SIFT is more accurate, while ORB is much faster and suited to real-time use.

Q3. Why is RANSAC needed?

Feature matches always contain wrong correspondences (outliers), so least-squares estimation over all points is dragged off by the outliers and fails. RANSAC randomly picks a minimal sample, estimates a model, and keeps the model that explains the most inliers, giving an estimate that is robust to outliers.

Q4. What does camera calibration estimate?

It estimates the intrinsic parameters such as focal length and principal point, the extrinsic parameters describing the camera pose, and the distortion coefficients for radial and tangential lens distortion. Zhang's method, which images a known planar checkerboard from several viewpoints, is a representative approach.