$L^2$ Space and Hilbert Space

Introduction

To understand the convergence of Fourier series rigorously, one needs the language of function spaces. The $L^2$ space is the prototypical example of a Hilbert space, and a Fourier series can be understood as an expansion in an orthonormal basis of that space.

The $L^2$ Space

Here the integral is understood in the Lebesgue sense. Strictly speaking, functions that are equal almost everywhere are identified with one another.

The $L^2$ norm

$$\|f\|_{L^2} = \left(\displaystyle\int_I |f(x)|^2\,dx\right)^{1/2}$$

The $L^2$ inner product

$$\langle f, g \rangle = \displaystyle\int_I f(x)\overline{g(x)}\,dx$$

With respect to this inner product, $\|f\|_{L^2} = \sqrt{\langle f, f \rangle}$.

Hilbert Space

A Hilbert space is a complete vector space equipped with an inner product.

Axioms of a Hilbert space

1. It is a vector space.
2. An inner product $\langle \cdot, \cdot \rangle$ is defined.
3. It is complete with respect to the norm induced by the inner product (every Cauchy sequence converges).

Orthonormal Basis

Definition

A subset $\{e_n\}$ of a Hilbert space $H$ is an orthonormal system if:

$$\langle e_m, e_n \rangle = \delta_{mn} = \begin{cases} 1 & (m = n) \\ 0 & (m \neq n) \end{cases}$$

It is an orthonormal basis (complete orthonormal system) if, in addition, every $f \in H$ can be written as

$$f = \displaystyle\sum_{n} \langle f, e_n \rangle e_n.$$

Here the coefficient $c_n = \langle f, e_n \rangle$ is called the ($n$-th) expansion coefficient of $f$.

The partial sum $S_N$

Taking only the first $N$ terms of the infinite sum above gives the partial sum,

$$S_N = \displaystyle\sum_{n=1}^{N} \langle f, e_n \rangle e_n = \sum_{n=1}^{N} c_n e_n.$$

As the number of terms $N$ increases, $S_N$ approaches $f$, and in the limit $N \to \infty$ it coincides with $f$ (in the $L^2$ sense described in the next section). The figure below shows concretely how close the partial sum $S_3 = c_1 e_1 + c_2 e_2 + c_3 e_3$ is to the original function.

Interpretation of a Fourier series

On $L^2([-\pi, \pi])$,

$$e_n(x) = \dfrac{1}{\sqrt{2\pi}}e^{inx} \quad (n \in \mathbb{Z})$$

forms an orthonormal basis. A Fourier series is precisely the expansion with respect to this basis.

Top: a function $f$ (here a triangle wave) is decomposed into orthonormal-basis components $\bigl(f = c_1 e_1 + c_2 e_2 + c_3 e_3 + \cdots,\ c_n=\langle f, e_n\rangle\bigr)$; reflecting its smoothness, higher-order components decay in amplitude ever more rapidly. Bottom: the partial sum $S_3$ of just the first three terms already almost coincides with the original $f$, so one can see with the eye that "$f$ equals the superposition of its basis components."

$L^2$ Convergence

This is equivalent to Parseval's identity:

$$\|f\|_{L^2}^2 = \displaystyle\sum_{n=-\infty}^{\infty}|c_n|^2.$$

Best Approximation

The partial sum of the Fourier series, $S_N(x) = \displaystyle\sum_{n=-N}^{N}c_n e^{inx}$, gives the best approximation to $f$ among all trigonometric polynomials of the same form:

This means that $S_N$ is the orthogonal projection of $f$ onto the space of all trigonometric polynomials of degree $N$.

Sobolev Spaces: Measuring Smoothness by Fourier Coefficients

Why $L^2$ alone is not enough

The $L^2$ norm can measure the "size" of a function, but not its "smoothness." For instance, $\cos x$ and $\cos(1000x)$ have equal $L^2$ norms, yet the latter oscillates far more violently. Within the $L^2$ framework one cannot even decide whether a function is differentiable—indeed, the differentiation operator is unbounded on $L^2$ (differentiating $\cos(nx)$ multiplies its amplitude by $n$, which diverges as $n\to\infty$). We need a new yardstick that measures "how many times a function can be differentiated." This is the motivation for Sobolev spaces.

Key observation: differentiation = multiplication by $in$ in Fourier space

Differentiating $f(x) = \sum_n c_n e^{inx}$ term by term gives

$$f'(x) = \sum_n (in)\,c_n e^{inx},$$

so differentiation is nothing but multiplying each coefficient by $in$. Applying Parseval's identity to $f'$,

$$\|f'\|_{L^2}^2 = \sum_n n^2\,|c_n|^2.$$

Thus the statement "$f$ is $k$-times differentiable (in the $L^2$ sense)" is equivalent to the algebraic condition

$$\sum_n n^{2k}\,|c_n|^2 < \infty.$$

The intuition smoothness = how fast the Fourier coefficients decay is thereby made quantitative.

The Sobolev space $H^s$

Generalizing this idea to a number of derivatives $s$ (not restricted to integers, but any real number) yields the Sobolev space.

The weight $(1+n^2)^s$ heavily penalizes high-frequency coefficients (large $|n|$). The larger $s$ is, the faster a decay it demands, so functions in $H^s$ are smoother. In particular, $s=0$ gives $H^0 = L^2$, and $s=1$ means that both $f$ and its (weak) derivative $f'$ belong to $L^2$ (the derivative here is understood in the weak sense discussed below). Here $s$ is a continuous parameter for the degree of smoothness, and its strength is that it can handle even fractional-order differentiability.

Smoothness ⇄ decay dictionary (worked examples)

Lining up representative waveforms—including the triangle wave used in the figure of this article—makes the correspondence clear.

One reads off the correspondence: the faster a function's coefficients decay, the larger the $s$ for which it lies in $H^s$, and the smoother it is.

Why it is useful ①: smoothness can be "predicted"

Comparing this against the table above reveals its power. The triangle wave belongs to $H^s$ for all $s<3/2$, so it exceeds the threshold $1/2$, and the theorem lets us predict that it is continuous (indeed it is, and the figure shows it joined up without breaks). The square wave, on the other hand, belongs only for $s<1/2$ and fails to reach the threshold, so continuity is not guaranteed (indeed it has jumps). Without examining a function directly, we can decide its smoothness from the decay of its Fourier coefficients alone.

Why it is useful ②: it fixes the rate of convergence of approximations

When $f \in H^s$, the $L^2$ error of the partial sum $S_N$ is bounded by

$$\|f - S_N\|_{L^2} \;\le\; \frac{C}{N^{s}}\,\|f\|_{H^s}$$

($C$ is a constant independent of $f$; the smoother $f$ is—the larger $s$—the faster it decays). This is the quantitative explanation of why, in the figure above, the triangle wave ($s<3/2$) was reproduced almost exactly by just three terms. Conversely, the square wave ($s<1/2$) converges slowly and requires many terms. The accuracy estimate for a Fourier series or a spectral method is determined by the smoothness $s$ of the function.

Why it is useful ③: the foundation for partial differential equations

Even for a function that cannot be differentiated in the classical sense, the framework of $H^s$ gives meaning to "differentiation" (the weak derivative). For this reason Sobolev spaces are the standard stage on which one discusses the existence, uniqueness, and regularity of solutions to partial differential equations such as the heat equation and the wave equation. Taking one step beyond $L^2$ so as to measure "smoothness" is what opens the door to a wide range of applications in analysis.

Summary

• The $L^2$ space: the space of square-integrable functions, equipped with an inner product and a norm.
• $L^2$ is a Hilbert space (a complete space with an inner product).
• A Fourier series is an expansion in an orthonormal basis of $L^2$.
• $L^2$ convergence is more general and easier to work with than pointwise convergence.
• The partial sum gives the best approximation (the orthogonal projection).
• The Sobolev space $H^s$ measures smoothness by the decay of Fourier coefficients, providing a test for continuity, an estimate of approximation error, and the foundation for partial differential equations.

Distilled into a single pair of formulas, for an orthonormal basis $\{e_n\}$ of $L^2$ we have

$$f = \sum_{n} \langle f, e_n \rangle e_n, \qquad \|f\|_{L^2}^2 = \sum_n |\langle f, e_n \rangle|^2$$

(the basis expansion and Parseval's identity).