Introduction to Functional Analysis
From finite to infinite dimensions: the world of function spaces
Introduction (undergraduate years 1–2)
About this chapter
Functional analysis is the mathematics that studies infinite-dimensional linear spaces, of which function spaces are the prototypical example. For instance, the solution of a differential equation is itself a "function," so we need an infinite-dimensional space that regards each function as a single point in order to search for that solution. This chapter begins with the motivating question, "why do we need to consider infinite-dimensional spaces?", and builds an intuitive understanding of how finite-dimensional vector spaces differ from infinite-dimensional function spaces. Rigorous definitions come from the basic level onward; here we use concrete examples to bridge into the world of infinite dimensions.
Prerequisites
- Basics of linear algebra (vectors, inner products, linear maps)
- Calculus (functions, the basics of integration)
- Convergence of sequences and the notion of a limit
Contents
1. Why infinite dimensions
Problems that finite dimensions cannot handle.
- Differential equations and function spaces
- Fourier series and infinitely many basis elements
- The state space of quantum mechanics
2. Functions as vectors
Seeing a function as a "point."
- Addition and scalar multiplication of functions
- The function space $C[a,b]$
- Similarities and differences with finite dimensions
3. Distance and convergence
How to measure the "closeness" of functions.
- Pointwise and uniform convergence
- The sup norm $\|f\|_\infty$
- The $L^2$ norm $\|f\|_2$
4. The inner product of functions
The "angle" and orthogonality of functions.
- $\displaystyle\langle f, g \rangle = \int f(x)\,g(x)\,dx$
- Orthogonal families of functions
- Orthogonality of the trigonometric system
5. What is completeness
A space "without holes."
- Cauchy sequences and limits
- $\mathbb{Q}$ vs $\mathbb{R}$: an example of completion
- $C[0,1]$ is not complete in the $L^2$ norm
6. Differences from finite dimensions
Phenomena unique to infinite dimensions.
- A closed bounded set need not be compact
- Extending the notion of dimension
- Unbounded (discontinuous) linear maps
7. Examples of linear operators
Maps from functions to functions.
- The differentiation operator $D: f \mapsto f'$
- Integral operators
- The shift operator
8. Applications of functional analysis
Where this theory is used.
- Weak solutions of partial differential equations
- The mathematical formulation of quantum mechanics
- Signal processing and Fourier analysis
Key concepts
The intuition behind a norm
A generalization of the "length" of a vector. For a function $f$:
- Sup norm: $\|f\|_\infty = \sup_{x} |f(x)|$ (measured by the maximum value)
- $L^2$ norm: $\displaystyle\|f\|_2 = \sqrt{\int |f(x)|^2\,dx}$ (measured by the mean square)
The intuition behind an inner product
A generalization of the "angle" of a vector. With $\displaystyle\langle f, g \rangle = \int f(x)\,g(x)\,dx$, when $\langle f, g \rangle = 0$ the functions $f$ and $g$ are "orthogonal."
The intuition behind completeness
The property that "any sequence that ought to converge actually does." The rationals $\mathbb{Q}$ are not complete (the limit of a rational sequence converging to $\sqrt{2}$ is not in $\mathbb{Q}$), whereas the reals $\mathbb{R}$ are complete.
The intuition behind a linear operator
A linear operator is the extension of a matrix to infinite dimensions. It is a linear map from functions to functions, such as the differentiation operator $D: f \mapsto f'$ or an integral operator. Unlike in finite dimensions, in infinite dimensions there also appear operators that are not continuous (not bounded).
Applications you can understand at this level
Fourier series
Represent a periodic function as an infinite sum of trigonometric functions. The families $\sin(nx)$ and $\cos(nx)$ form an "orthogonal basis" in the $L^2$ sense. This is the infinite-dimensional version of "expansion in an orthonormal basis" from finite dimensions.
Quantum mechanics
The state of a particle is represented by a wave function $\psi(x)$, an element of the Hilbert space $L^2$. Observables are represented by linear operators (Hermitian operators). The Schrödinger equation is an operator equation on the space $L^2$.
Signal processing
Signals such as audio and images can be treated as functions and analyzed in the space $L^2$. The Fourier transform, the wavelet transform, and the like can be understood as isometries on $L^2$.
Differential equations
The completeness of a function space is used in an essential way to prove the existence and uniqueness of solutions to partial differential equations. The theory of "weak solutions" is developed in Sobolev spaces (an extension of $L^2$).
Study tips
- Contrast with finite dimensions: keep the attitude of extending your intuition from $\mathbb{R}^n$ to infinite dimensions.
- Multiple norms: be aware that putting different norms on the same set yields different spaces.
- The importance of completeness: since being able to take limits is fundamental to analysis, always check for completeness.
- Understand through examples: confirm abstract definitions with concrete examples such as $C[0,1]$ or $L^2$.
References
Reading
Seeing a function as a single point [Reading]
Regard a whole function as a single point floating in a space. A relaxed take on the boldest idea in functional analysis, told through the analogy with vectors and everyday examples.
Fourier is a coordinate system in infinite dimensions [Reading]
The Fourier series that unravels a complex wave into $\sin$ and $\cos$ was, in fact, the act of projecting a function onto infinitely many orthogonal axes and reading off its "coordinates" via the inner product — a casual look at that viewpoint.
The comfort of a space without holes [Reading]
A sequence that ought to converge really does — completeness meant a "space without holes." Starting from the story of the rationals and the reals, a relaxed look at why functional analysis cares so much about completeness.