Wiener Filter

Frequency-Domain and Time-Domain Approaches with Wirtinger Derivative

What Is the Wiener Filter?

The Wiener filter is a linear filter that optimally estimates the original signal from noisy observations in the minimum mean square error (MMSE) sense. Developed by Norbert Wiener in the 1940s, it is widely used in signal processing, image processing, and communications.

X(ω) Original signal H(ω) Degradation filter + N(ω) Y(ω) Degraded signal G(ω) Wiener filter
Figure: Signal degradation model and restoration by the Wiener filter

Solution of the Wiener Filter

Frequency domain:

$$G(\omega) = \frac{H^*(\omega) P_S(\omega)}{|H(\omega)|^2 P_S(\omega) + P_N(\omega)}$$

Time domain (Wiener-Hopf equation):

$$\boldsymbol{w}_{\rm opt} = \boldsymbol{R}^{-1}\boldsymbol{p}$$

Table of Contents

This series derives the Wiener filter through multiple approaches. Arriving at the same result by different methods deepens understanding of the underlying theory.

Part 1: Fundamental Derivations

  1. Chapter 1 Frequency-Domain Derivation

    Three approaches: orthogonality principle, completing the square, and Wirtinger derivative

  2. Chapter 2 Time-Domain Derivation

    Derivation of the Wiener-Hopf equation via the orthogonality principle and Wirtinger derivative

  3. Chapter 3 Relationship Between Frequency and Time Domains

    Correspondence between the two domains and their essential equivalence

Part 2: Advanced Derivation Methods

  1. Chapter 4 Advanced Derivation Methods

    Real/imaginary decomposition, variational method, projection theorem, maximum likelihood estimation, spectral factorization, and LMS convergence point

Part 3: Practical Aspects and Related Theory

  1. Chapter 5 Practical Considerations

    Non-causality, regularization, numerical stability, Levinson-Durbin algorithm, and spectral estimation

  2. Chapter 6 Related Theory

    Relationship with the Kalman filter, connections to machine learning, and comparison of derivation methods

Part 4: Advanced Topics

  1. Chapter 7 Causal Wiener Filter

    Spectral factorization, Wiener-Hopf decomposition, and real-time processing

  2. Chapter 8 Smoothing, Filtering, and Prediction

    Wiener's three classical problems and performance comparison

  3. Chapter 9 Multichannel Wiener Filter

    MIMO extension, microphone arrays, and beamforming

  4. Chapter 10 Applications

    Echo cancellation, channel equalization, speech enhancement, image restoration, and Python implementation

Appendix

  1. Appendix Supplementary Materials

    Wirtinger derivative formulas, optimality for Gaussian signals, references, and trivia

Why Study Multiple Derivation Approaches

  1. Multifaceted understanding: Arriving at the same conclusion through different approaches provides deeper insight into the essence of the theory
  2. Improved applicability: Enables choosing the most suitable derivation method depending on the nature of the problem
  3. Mathematical breadth: Offers exposure to various mathematical techniques including functional analysis, probability theory, and optimization theory
  4. Historical perspective: Provides understanding of how derivation methods have evolved alongside the development of the field

The main approaches are as follows:

  1. Frequency-domain derivation: completing the square, orthogonality principle, Wirtinger derivative
  2. Time-domain derivation: orthogonality principle, Wirtinger derivative
  3. Other derivation methods: real/imaginary decomposition, variational method, projection theorem, maximum likelihood estimation, spectral factorization, LMS convergence point

These are different representations of the same theory and ultimately yield the equivalent result (the Wiener-Hopf equation).