Image Processing Basics

Filtering and Transforms

About This Level

The basic level covers the core filtering techniques of image processing. You learn smoothing and sharpening by spatial filtering, edge detection, and processing in the frequency domain via the Fourier transform.

Learning Objectives

  • Understand the principle of the convolution operation
  • Use the various filters (mean, Gaussian, Sobel, and so on) with confidence
  • Understand the principle and implementation of edge detection
  • Learn the Fourier transform and processing in the frequency domain
  • Understand the types of noise and the appropriate removal methods

Diagram: The Convolution Operation

Input image 50 60 55 52 100 58 48 54 51 Kernel (3×3) 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 Mean filter Output 59 Σ (pixel × kernel value) = (50+60+55+52+100+58+48+54+51) / 9 = 528 / 9 ≈ 59 Slide the kernel over the image, computing at each position
Figure 1: Convolution — sliding the kernel over the image and outputting the sum of products of pixel and kernel values at each position (this form is correlation, without flipping the kernel; the averaging kernel is symmetric, so the result equals convolution).

Table of Contents

1. Introduction to Spatial Filtering (in preparation)

Fundamentals of convolution.

  • What convolution is
  • The kernel (filter) concept
  • Border handling
  • Implementation in OpenCV

2. Smoothing Filters (in preparation)

Noise reduction and blurring.

  • Mean filter
  • Gaussian filter
  • Median filter
  • Bilateral filter

3. Sharpening Filters (in preparation)

Edge enhancement.

  • Laplacian filter
  • Unsharp masking
  • High-boost filtering
  • Applications of sharpening

4. Edge Detection (in preparation)

Extracting contours.

  • Sobel filter
  • Prewitt filter
  • Canny edge detection
  • Applications of edge detection

5. Fourier Transform (in preparation)

Frequency-domain processing.

  • Principles of the 2D DFT
  • Frequency spectrum
  • Low-pass / high-pass filters
  • Band-pass filters

6. Noise Reduction (in preparation)

Techniques for image quality.

  • Types of noise (Gaussian, S&P)
  • Removal with spatial filters
  • Removal with frequency filters
  • Non-local means (NLM)

Diagram: Edge Detection

Original Rectangular object Sobel X (horizontal) Detects vertical edges Sobel Y (vertical) Detects horizontal edges Combined |∇I| Sobel X: [-1,0,1; -2,0,2; -1,0,1] Sobel Y: [-1,-2,-1; 0,0,0; 1,2,1]
Figure 2: Edge detection with the Sobel filter — combining the horizontal and vertical derivatives, contours are obtained from the gradient magnitude $|\nabla I|$.

Diagram: The Fourier Transform

Spatial domain Periodic pattern 2D FFT ℱ Frequency domain Frequency components Filter H(u,v) 2D iFFT ℱ⁻¹ Output Low-pass example (blurred)
Figure 3: Frequency-domain filtering via the 2D Fourier transform — the spatial-domain image is transformed to the frequency domain, a filter $H(u,v)$ is applied, and it is transformed back (the output shows a low-pass filter as an example).

Key Theorems and Formulas

2D Convolution

The output image $g(x,y)$ is obtained as the convolution of the input image $f(x,y)$ with the kernel $h(x,y)$:

$$g(x,y) = \displaystyle\sum_{i=-k}^{k} \displaystyle\sum_{j=-k}^{k} f(x-i, y-j) \cdot h(i,j)$$

In practice, most implementations call the cross-correlation $\sum_{i,j} f(x+i, y+j)\, h(i,j)$ “convolution”; it does not flip the kernel. The two coincide for symmetric kernels (mean, Gaussian), but differ for asymmetric kernels such as Sobel. In fact, most image-processing libraries — including OpenCV’s filter2D — compute cross-correlation rather than true convolution (see Convolution vs. Correlation).

The Sobel Operator

Gradient magnitude: $|\nabla I| = \sqrt{G_x^2 + G_y^2}$; gradient direction: $\theta = \operatorname{atan2}(G_y, G_x)$ (which also handles $G_x = 0$ and the quadrant correctly).

The Convolution Theorem

Convolution in the spatial domain corresponds to multiplication in the frequency domain: $\mathcal{F}\{f * g\} = \mathcal{F}\{f\} \cdot \mathcal{F}\{g\}$.

Supplementary Reading

Readings that explore the "why" of filters and transforms one step at a time, with diagrams and equations (all in preparation).

Convolution vs. Correlation

Understand the single difference between convolution and correlation (flipping the kernel) and its consequences.

Border Handling (Padding) Methods

Understand the problem of a kernel running off the edge, and four representative padding schemes.

Integral Images and the Box Filter

Understand why an integral image can compute the sum over any rectangle in constant time.

Image Gradients and Derivative Filters

Understand how edges correspond to peaks of the first derivative and zero-crossings of the second.

DoG and LoG (Derivatives of a Gaussian)

Understand how LoG and DoG are "smoothing + second derivative" and can be used for blob detection.

Gabor Filters

Understand how a Gabor filter is a "Gaussian window × sinusoid", selective in orientation and frequency.

Reading a 2D FFT Spectrum

Learn to read what the center, the periphery, and orientation mean in a 2D FFT spectrum image.

The Sampling Theorem and Aliasing

Understand the Nyquist condition and the aliasing (folding) that occurs when it is violated.

Noise Models and Statistics

Understand the statistical properties of representative noise types and the filters suited to each.

Ringing and the Gibbs Phenomenon

Understand how ringing originates from the Gibbs phenomenon and can be suppressed with a smooth window.

Prerequisites

  • The introductory-level content (pixels, color spaces, histograms)
  • Foundations of linear algebra (matrix operations)
  • Basic operation of Python / OpenCV (the corresponding introductory chapter is in preparation)

Frequently Asked Questions

What does the basic image processing level cover?

Centered on spatial filtering by convolution, it covers smoothing (mean, Gaussian, median, bilateral), sharpening (Laplacian, unsharp masking), edge detection (Sobel, Prewitt, Canny), frequency-domain processing via the Fourier transform, and noise reduction.

How does spatial filtering differ from frequency-domain filtering?

Spatial filtering applies a convolution kernel directly to a pixel neighborhood. Frequency-domain filtering uses the Fourier transform to separate frequency components, applies a low-pass or high-pass filter, and transforms back. By the convolution theorem $\mathcal{F}\{f * g\} = \mathcal{F}\{f\} \cdot \mathcal{F}\{g\}$, for linear shift-invariant filters, convolution in the spatial domain corresponds to multiplication in the frequency domain. Multiplying via the FFT, however, gives circular convolution, so the result matches the spatial-domain one only if the border handling (such as zero padding) is made consistent. Nonlinear filters such as the median and bilateral filters cannot be written as a multiplication in the frequency domain.

What does edge detection actually compute?

It computes the image intensity gradient. The Sobel filter estimates the horizontal and vertical derivatives $G_x, G_y$, and locations where the gradient magnitude $\sqrt{G_x^2 + G_y^2}$ is large are treated as edges. The Canny method combines smoothing, gradient computation, non-maximum suppression, and hysteresis thresholding, which tends to produce thin edges (they are not guaranteed to be connected and can break depending on the thresholds).