Chapter 6: Fundamentals of Eigenvalues and Eigenvectors

1. Why Eigenvalues and Eigenvectors Matter

Seeking the "Essence" of a Matrix

A matrix $A$ represents a linear transformation, but when $A$ is given, what is the "essence" of that transformation?

Consider, for example, the following matrix:

How does this matrix transform the vector

Let us draw arrows connecting each vector $\boldsymbol{v}$ to its image

$\boldsymbol{v}' = A \boldsymbol{v}$ under the transformation.

Figure 1: Transformation by matrix $\boldsymbol{A}$ — arrows from each point $\boldsymbol{v}$ to $\boldsymbol{v}' = \boldsymbol{A}\boldsymbol{v}$

A "Special Direction" Visible in the Figure

Looking at Figure 1, most arrows change direction depending on their location, but arrows along the diagonal 45° line shrink straight toward the origin. Although the coarse grid makes it hard to see clearly, arrows near the 120° line appear to stretch away from the origin. Vectors along these two lines seem to only scale without changing direction under the transformation.

Such a vector, with scaling factor $\lambda$, can be written as:

If $\lambda > 1$ the vector stretches; if $0 < \lambda < 1$ it shrinks.

In this example there are two such directions — 45° and 120°:

There should be two pairs $\{\boldsymbol{v}_1, \lambda_1\}$ and $\{\boldsymbol{v}_2, \lambda_2\}$. But how can we find them from the matrix $A$? We will unravel this question step by step below.

2. Naming the Special Directions — Definition of Eigenvalues and Eigenvectors

Let us give formal names to the "direction-preserving vectors" and their "scaling factors" found in Section 1.

2.1 Definition

2.2 Etymology

"Eigen" is German for "own" or "characteristic." An eigenvector represents a direction that is "characteristic of" the matrix.

2.3 Important Remarks

• An eigenvector must be a nonzero vector. $\boldsymbol{v} = \boldsymbol{0}$ always satisfies $A\boldsymbol{0} = \lambda\boldsymbol{0}$, but is not called an eigenvector.
• For an eigenvector $\boldsymbol{v}$, any nonzero scalar multiple $c\boldsymbol{v}$ is also an eigenvector for the same eigenvalue (since $A(c\boldsymbol{v}) = cA\boldsymbol{v} = c\lambda\boldsymbol{v} = \lambda(c\boldsymbol{v})$). Hence eigenvectors are not unique — only their direction matters.

3. Backing Up Our Observations — Geometric Meaning

Now that we have the definition, let us revisit the phenomena observed in Section 1.

3.1 Vectors That Preserve Direction

3.2 Sign and Magnitude of Eigenvalues

• $\lambda > 1$: stretches in the eigenvector direction
• $0 < \lambda < 1$: shrinks in the eigenvector direction
• $\lambda < 0$: reverses direction (and scales by $|\lambda|$)
• $\lambda = 1$: eigenvector direction is unchanged
• $\lambda = 0$: eigenvector direction collapses to the origin

In Figure 1 of Section 1, arrows in the 120° direction moved away from the origin ($\lambda > 1$, stretching), while arrows in the 45° direction approached the origin ($0 < \lambda < 1$, shrinking) — exactly matching the classification above.

3.3 Does Every Matrix Have Eigenvectors? — Rotation Matrices

Consider a 2D rotation matrix (for $\theta \neq 0, \pi$):

This matrix has no real eigenvalues. A rotation (with $\theta \neq 0, \pi$) changes the direction of every vector, so no direction-preserving vector exists.

Over the complex numbers, however, eigenvalues $e^{\pm i\theta}$ do exist. Thus some matrices have no real eigenvectors. How, then, do we systematically find eigenvalues and eigenvectors for a given matrix?

4. Finding $\lambda$ and $\boldsymbol{v}$ — The Characteristic Equation

The geometric meaning is clear. But how do we actually compute eigenvalues and eigenvectors from a given matrix?

4.1 The Characteristic Equation

Rearranging $A\boldsymbol{v} = \lambda\boldsymbol{v}$ gives

If $A - \lambda I$ were invertible, we could multiply both sides by its inverse to get

which forces $\boldsymbol{v} = \boldsymbol{0}$. But by definition $\boldsymbol{v} \neq \boldsymbol{0}$, so $(A - \lambda I)$ must be singular, meaning $\det(A - \lambda I) = 0$.

This equation is called the characteristic equation. Expanding $\det(A - \lambda I)$ yields a polynomial in $\lambda$ called the characteristic polynomial (of degree $n$ for an $n \times n$ matrix).

4.2 A Concrete Example

Find the eigenvalues of $A = \begin{pmatrix} 3 & 1 \\ 0 & 2 \end{pmatrix}$.

From $\det(A - \lambda I) = 0$ we get $\lambda = 2$ or $\lambda = 3$.

4.3 Computing the Eigenvectors

For $\lambda = 2$:

From $x + y = 0$, the eigenvectors are $\boldsymbol{v} = t\begin{pmatrix} 1 \\ -1 \end{pmatrix}$ ($t \neq 0$).

For $\lambda = 3$:

From $y = 0$, the eigenvectors are $\boldsymbol{v} = t\begin{pmatrix} 1 \\ 0 \end{pmatrix}$ ($t \neq 0$).

Figure 2 below shows the displacement $\boldsymbol{v} \to A\boldsymbol{v}$ for points on the two eigenvector directions (dashed lines) and for other points (arrow lengths scaled to 0.2× actual for clarity). Colored arrows along the dashed lines stay on those lines — confirming that the direction does not change — while gray arrows elsewhere do change direction.

4.4 Finding the Eigenvalues of the Matrix from Section 1

Using the characteristic equation, let us find the eigenvalues of the matrix $A = \begin{pmatrix} 0.88 & -0.18 \\ -0.32 & 1.02 \end{pmatrix}$ from Section 1.

By the quadratic formula:

Hence $\lambda_1 = 1.2$ and $\lambda_2 = 0.7$. Let us also find the eigenvectors.

For $\lambda_1 = 1.2$:

From $-0.32x - 0.18y = 0$ we get $x = -\dfrac{9}{16}y$, so the eigenvectors are $\boldsymbol{v}_1 = t\begin{pmatrix} -9 \\ 16 \end{pmatrix}$ ($t \neq 0$). The direction angle is $\arctan\!\left(\dfrac{16}{-9}\right) \approx 120°$.

For $\lambda_2 = 0.7$:

From $0.18x - 0.18y = 0$ we get $x = y$, so the eigenvectors are $\boldsymbol{v}_2 = t\begin{pmatrix} 1 \\ 1 \end{pmatrix}$ ($t \neq 0$). The direction angle is $\arctan\!\left(\dfrac{1}{1}\right) = 45°$.

Since $\lambda_1 = 1.2 > 1$ the 120° direction stretches, and since $0 < \lambda_2 = 0.7 < 1$ the 45° direction shrinks — the computed eigenvalues and eigenvectors match the phenomena observed in Figure 1 of Section 1 exactly.

5. The Span of Eigenvectors — Eigenspaces

In Section 4 we found that the eigenvectors of $A = \begin{pmatrix} 3 & 1 \\ 0 & 2 \end{pmatrix}$ for eigenvalue $\lambda = 2$ are $\boldsymbol{v} = t\begin{pmatrix} 1 \\ -1 \end{pmatrix}$ ($t \neq 0$). Whether $t = 1$, $t = -3$, or $t = 0.5$, all give eigenvectors — as we saw in Section 2, any nonzero scalar multiple of an eigenvector is again an eigenvector.

The eigenvector is therefore not unique; the entire line through the origin in the direction of $\begin{pmatrix} 1 \\ -1 \end{pmatrix}$ (excluding the origin itself) consists of eigenvectors. If we conveniently include the zero vector, this set is precisely a line through the origin — a 1-dimensional subspace of $\mathbb{R}^2$.

The word "space" often evokes images of 2D planes or 3D solids, but in linear algebra a line through the origin (1-dimensional) is a perfectly valid subspace. Any set closed under addition and scalar multiplication qualifies as a "space" regardless of dimension. Let us name this set.

5.1 Definition

That is, it is the set of all vectors $\boldsymbol{v}$ satisfying $A\boldsymbol{v} = \lambda\boldsymbol{v}$, including the zero vector. Since $A\boldsymbol{v} = \lambda\boldsymbol{v}$ is equivalent to $(A - \lambda I)\boldsymbol{v} = \boldsymbol{0}$, the eigenspace is the solution space (null space) of $(A - \lambda I)\boldsymbol{v} = \boldsymbol{0}$, written $\ker(A - \lambda I)$:

In the example above, $E_2 = \left\{ t\begin{pmatrix} 1 \\ -1 \end{pmatrix} \;\middle|\; t \in \mathbb{R} \right\}$ is a 1-dimensional subspace (a line). Similarly, $E_3 = \left\{ t\begin{pmatrix} 1 \\ 0 \end{pmatrix} \;\middle|\; t \in \mathbb{R} \right\}$ is also a 1-dimensional subspace.

5.2 Dimension of Eigenspaces

In the example above both eigenspaces were 1-dimensional (lines). In general, however, a single eigenvalue may have multiple linearly independent eigenvectors, making the eigenspace 2-dimensional (a plane) or higher.

The dimension of an eigenspace is called the geometric multiplicity. The number of times an eigenvalue appears as a root of the characteristic polynomial is the algebraic multiplicity. The inequality geometric multiplicity $\leq$ algebraic multiplicity always holds.

5.3 What Is an Eigenspace Good For?

One might wonder: "If an eigenvalue gives the scaling factor and an eigenvector gives the direction, why do we need the eigenspace?" The answer is that a single eigenvector is sometimes insufficient.

When an eigenvalue $\lambda$ has multiplicity 2 or higher, the corresponding eigenvectors may span an entire plane or higher-dimensional space. Consider the 3D diagonal matrix:

The eigenvectors for eigenvalue $\lambda = 3$ are not just $\begin{pmatrix}1\\0\\0\end{pmatrix}$. The vector $\begin{pmatrix}0\\1\\0\end{pmatrix}$ is also an eigenvector, and every linear combination $s\begin{pmatrix}1\\0\\0\end{pmatrix} + t\begin{pmatrix}0\\1\\0\end{pmatrix}$ is likewise an eigenvector. The eigenspace $E_3$ is the entire $xy$-plane (2-dimensional), and every direction in this plane is uniformly stretched by a factor of 3.

Listing only $\begin{pmatrix}1\\0\\0\end{pmatrix}$ as "the" eigenvector loses the information that the entire plane is scaled by 3. Knowing the eigenspace reveals the full extent of directions that scale at the same rate.

What Determines Diagonalizability

The dimension of eigenspaces determines whether a matrix can be diagonalized.

• For every eigenvalue, geometric multiplicity $=$ algebraic multiplicity → diagonalizable
• For at least one eigenvalue, geometric multiplicity $<$ algebraic multiplicity → not diagonalizable

For example, the matrix below has eigenvalue $\lambda = 3$ with algebraic multiplicity 2, but its eigenspace is only 1-dimensional:

Solving $(B - 3I)\boldsymbol{v} = \boldsymbol{0}$ gives only $\boldsymbol{v} = t\begin{pmatrix}1\\0\end{pmatrix}$. Since two independent eigenvectors are not available, $B$ cannot be diagonalized. This "insufficient span of the eigenspace" reveals the matrix's structural complexity — the need for a Jordan normal form.

6. Caveats

6.1 Eigenvectors Are Not Unique

Any nonzero scalar multiple of an eigenvector is also an eigenvector. For comparison purposes, eigenvectors are typically normalized to unit length.

6.2 Repeated Eigenvalues

When eigenvalues are repeated, the number of corresponding independent eigenvectors (geometric multiplicity) can be less than the algebraic multiplicity. In that case the matrix is not diagonalizable.

6.3 Numerical Stability

Eigenvalue computation for large matrices can be numerically unstable. In practice, iterative methods such as the QR algorithm or the power method are used.

6.4 Generalized Eigenvalue Problem

The generalized eigenvalue problem of the form $A\boldsymbol{v} = \lambda B\boldsymbol{v}$ is also important and arises in engineering applications such as the finite element method.

7. Summary and Next Steps

We now have a fundamental answer to the question posed in Section 1 — "What is the essence of a matrix?" A matrix's action can be understood as scaling along eigenvector directions, with eigenvalues providing the scaling factors.

Appendix: Interactive Matrix Image Transformation

Freely adjust the entries of matrix $A$ and observe how the image is deformed. For each pixel position $\boldsymbol{v}'$ in the output image, the source coordinate is computed as $\boldsymbol{v} = A^{-1}\boldsymbol{v}'$, and the corresponding color is drawn.