Image Processing Intermediate

Analysis and Feature Extraction

About This Level

The intermediate level covers image analysis and feature-extraction techniques that pull structure and features out of an image: noise removal and region operations via morphological processing, region partitioning via segmentation, shape-feature extraction via contour analysis, and classical (pre-deep-learning) object detection with HOG and Haar-like features.

Note that keypoint detection (SIFT, ORB) and camera models belong to Computer Vision ("image → 3D / semantic understanding"), where they are covered systematically.

Learning Objectives

  • Understand morphological operations (dilation, erosion, opening, closing)
  • Master the various image segmentation methods
  • Learn contour detection and shape analysis (area, moments, convex hull)
  • Understand classical object detection with HOG and Haar-like features

Diagram: Morphological Processing

Original With noise Dilation Region grows Erosion Region shrinks Opening Erosion → Dilation Removes small noise Operate on a binary image with a structuring element (kernel): noise removal, hole filling, contour extraction.
Figure 1: Morphological processing (schematic) — operating on a binary image with a structuring element. Dilation grows the region, erosion shrinks it, and opening (erosion then dilation) removes small noise. The exact result depends on the shape and size of the structuring element.

Table of Contents

1. Morphological Processing

Region operations on binary images.

  • Dilation and erosion
  • Opening and closing
  • Choosing the structuring element
  • Morphological gradient (contour extraction)

2. Image Segmentation

Methods for region partitioning.

  • Thresholding (Otsu's method)
  • Region growing
  • Watershed algorithm
  • GrabCut

3. Contour Detection and Analysis

Shape features.

  • Contour detection and approximation
  • Area, perimeter, centroid
  • Convex hull and moments
  • Shape matching

4. Keypoint Detection → Computer Vision

Keypoint detection, descriptors, and matching are foundational to Computer Vision and are covered systematically there.

→ See Computer Vision (Basic)

5. Object Detection (Classical)

Pre-machine-learning methods.

  • HOG features
  • Haar-like features
  • Cascade classifiers
  • Template matching

6. Camera Models and Geometry → Computer Vision

Camera models and projective geometry are foundational theory of Computer Vision and are covered systematically there.

→ See Computer Vision (Introduction)

Diagram: The HOG Feature

Input image Gradients Cell division 8×8 px / cell (8×10) Orientation histogram histogram of the red cell 0° 90° 180° 9 bins (0–180°)
Figure 2: The HOG feature — gradients of the input image (a person silhouette with arms lowered at 45°) are computed with the Sobel operator. Each cell's dominant gradient orientation is drawn as a double-headed arrow (orientation is unsigned, 0–180°, so it has heads on both ends; the vertical edges of the torso and legs give horizontal, the head and shoulders give vertical, and the 45° arms give diagonal orientations). The image is then split into 8×8-pixel cells (8×10). The key point of HOG is that the histogram is built per cell, not over the whole image: here one cell over the hand on the viewer's right (red box = the same cell as Figure 3) is highlighted, and its orientation histogram is shown (concentrated in the diagonal bin centered at 130°). Figure 3 shows how it is built. Normalizing the per-cell histograms per block and concatenating them yields the HOG feature vector.

The role of cell division is to summarize gradient orientation as a local distribution — which edge directions are common, and where. For each pixel in a cell, compute the gradient magnitude $m=\sqrt{g_x^2+g_y^2}$ and orientation $\theta=\arctan(g_y/g_x)$ (unsigned, 0–180°), then cast a vote of weight $m$ into the bin that $\theta$ falls in (votes near a bin boundary are split linearly between the two neighboring bins). Accumulating over all 8×8=64 pixels gives that cell's 9-dimensional histogram. Building one per cell, normalizing per block, and concatenating yields the HOG feature vector.

One cell (8×8 px) per-pixel gradients (arrow = direction) 9 orientation bins 0° 90° 180° sort by orientation (0–180°) vote by magnitude Histogram of this cell 10 30 50 70 90 110 130 150 170 degrees (bin centers)
Figure 3: Turning the gradients inside a cell into an orientation histogram — for every pixel of one cell (8×8 px) we take the gradient direction and magnitude, sort the directions into 9 orientation bins (0–180°), and vote by magnitude, giving that cell's histogram. The data shown are from a cell over the hand on the viewer's right (the arm's 45° diagonal edge), so the gradients concentrate in the diagonal bin (centered at 130°).

Key Methods and Formulas

Otsu's Thresholding

Automatically choose the threshold that maximizes the between-class variance: $\sigma_b^2(t) = \omega_0(t)\omega_1(t)[\mu_0(t) - \mu_1(t)]^2$.

The HOG Feature

Concatenate the per-cell histograms of oriented gradients and apply block normalization to form a feature vector robust to illumination changes.

Supplementary Reading

Readings on advanced analysis and feature-extraction topics, centered on diagrams.

Distance Transform

Understand the distance transform as a map of each foreground pixel's distance to the nearest background, and its uses.

Skeletonization (Thinning)

Understand the skeleton as a 1-pixel-wide central axis representing shape while preserving connectivity.

Adaptive Thresholding

Understand how a global threshold fails under uneven illumination and how a local threshold rescues it.

Superpixels (SLIC)

Understand superpixels as an over-segmentation into small boundary-aligned regions that serve as a preprocessing step.

Active Contour Models (Snakes)

Understand a snake as a curve that minimizes the energy of smoothness plus image edges.

Markov Random Fields and Images

Understand the MRF as energy minimization of data fidelity plus neighborhood smoothness.

Texture Analysis (GLCM and LBP)

Understand GLCM and LBP as two representative ways to turn texture into numeric features.

Fourier Descriptors

Understand how a Fourier transform of a contour represents shape with a few coefficients, usable for matching.

Top-hat Transform and Morphological Reconstruction

Understand top-hat as background-unevenness removal and reconstruction as shape recovery from markers.

Region Segmentation by Color Clustering

Understand how region segmentation by color clustering works, and how to choose the color space and k.

Hough Transform

Understand how lines and circles are detected by "voting" in a parameter space.

Prerequisites

  • The basic-level content (filtering, edge detection)
  • Linear algebra (eigenvalues, matrix decomposition)
  • Foundations of probability and statistics

Frequently Asked Questions

What does the intermediate image processing level cover?

Morphological processing (dilation, erosion, opening, closing), image segmentation (Otsu thresholding, region growing, watershed, GrabCut), contour detection and shape analysis (area, moments, convex hull), and classical object detection with HOG and Haar-like features. Keypoint detection and camera models are covered in Computer Vision.

What is morphological processing?

It manipulates the regions of (mainly binary) images using a structuring element (kernel). Dilation grows regions and erosion shrinks them. Opening (erosion then dilation) removes small noise, while closing (dilation then erosion) fills holes and gaps.

How does the HOG feature capture an object?

The image is divided into small cells, and a histogram of oriented gradients is built within each cell. Normalizing these per block and concatenating them yields the HOG descriptor, a shape (edge-orientation) representation robust to illumination changes, widely used for pedestrian detection.