4.4 Eigenvalues and eigenvectors

This is the lesson the rest of the bookshelf has been quietly assuming. The “eigenvalues of A” you have already met in the ODE phase plane (Foundations 5.4), in modal analysis of PDEs (Foundations 6.5), in Helmholtz cavity modes (Foundations 6.7), and in the time-independent Schrödinger equation (Foundations 6.8) — all of those are instances of the same idea: a matrix or operator has special directions it merely stretches or contracts rather than rotates. Find those directions, and the linear map’s behaviour becomes transparent.

The defining equation

A non-zero vector $\mathbf{v}$ is an eigenvector of the matrix $A$ if it satisfies

A \mathbf{v} \;=\; \lambda \mathbf{v}

for some scalar $\lambda$ . The number $\lambda$ is the corresponding eigenvalue. The equation says: when $A$ acts on $\mathbf{v}$ , the result is $\mathbf{v}$ itself scaled by $\lambda$ — no rotation, no twisting, just a stretch by factor $\lambda$ (or a contraction if $|\lambda| < 1$ , or a flip if $\lambda < 0$ ).

Most vectors are not eigenvectors: most directions get rotated when $A$ acts. The eigenvectors are the special directions left invariant up to scaling. Their existence and structure are what make many matrix problems tractable.

Why eigenvectors matter

If $A$ is an $n \times n$ matrix with $n$ linearly independent eigenvectors $\mathbf{v}_1, \ldots, \mathbf{v}_n$ with eigenvalues $\lambda_1, \ldots, \lambda_n$ , then any vector $\mathbf{x}$ can be written as a linear combination

\mathbf{x} \;=\; c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_n \mathbf{v}_n.

Applying $A$ to both sides and using linearity plus the eigenvalue equation:

A \mathbf{x} \;=\; c_1 \lambda_1 \mathbf{v}_1 + c_2 \lambda_2 \mathbf{v}_2 + \cdots + c_n \lambda_n \mathbf{v}_n.

The action of $A$ on $\mathbf{x}$ has become componentwise multiplication in the eigenvector basis — each component $c_k$ just gets multiplied by its eigenvalue $\lambda_k$ . This is the diagonalisation trick: in the right basis, $A$ looks like a diagonal matrix, and diagonal matrices are trivial to power, exponentiate, or invert.

This is the deep mathematical reason mode expansions work in PDEs (Foundations 6.5). The modes are the eigenfunctions of the spatial differential operator; the operator acts on a sum-over-modes by multiplying each mode by its eigenvalue. The same algebraic move that makes a finite-dimensional matrix easy to handle makes an infinite-dimensional linear differential operator solvable.

Finding eigenvalues

To find the eigenvalues of a given matrix $A$ , rearrange the defining equation:

A \mathbf{v} = \lambda \mathbf{v} \quad\Longleftrightarrow\quad (A - \lambda I) \mathbf{v} = \mathbf{0}.

We want this to have a non-zero solution $\mathbf{v}$ (eigenvectors aren’t allowed to be the zero vector — that would be trivial). A square linear system has a non-zero solution exactly when its matrix is singular, i.e. has determinant zero. So the eigenvalues are the roots of

\boxed{\;\det(A - \lambda I) \;=\; 0,\;}

which is a polynomial equation in $\lambda$ of degree $n$ — the characteristic polynomial of $A$ . An $n \times n$ matrix therefore has at most $n$ eigenvalues (counted with multiplicity). Once each eigenvalue is known, the corresponding eigenvector is found by solving the homogeneous linear system $(A - \lambda I) \mathbf{v} = \mathbf{0}$ — Gaussian elimination from 4.3, with a singular matrix on purpose.

For a $2 \times 2$ matrix the characteristic polynomial is a quadratic:

\det(A - \lambda I) \;=\; \lambda^2 - (\mathrm{tr}\, A)\, \lambda + \det A,

where $\mathrm{tr}\, A = a_{11} + a_{22}$ is the trace (sum of diagonal entries). The quadratic formula gives the two eigenvalues directly:

\lambda_{\pm} \;=\; \frac{\mathrm{tr}\,A}{2} \pm \sqrt{\left(\frac{\mathrm{tr}\,A}{2}\right)^2 - \det A}.

This is exactly the formula that appears in the ODE phase-plane analysis with $A$ as the linearised flow matrix; the sign of the discriminant determines whether the eigenvalues are real distinct (overdamped), real repeated (critical), or complex conjugate (underdamped, spirals).

Worked example: a 2x2 matrix by hand

▶ Worked example: every step, no shortcuts

The problem. Find the eigenvalues and eigenvectors of

A \;=\; \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}.

Step 1 — Set up the characteristic equation. $A - \lambda I$ is

\begin{pmatrix} 3 - \lambda & 1 \\ 1 & 3 - \lambda \end{pmatrix},

and the determinant is

\det(A - \lambda I) \;=\; (3 - \lambda)^2 - 1 \cdot 1 \;=\; (3 - \lambda)^2 - 1.

Step 2 — Solve. Expand $(3 - \lambda)^2 - 1 = \lambda^2 - 6\lambda + 9 - 1 = \lambda^2 - 6\lambda + 8 = 0$ . Quadratic formula or factoring:

\lambda^2 - 6\lambda + 8 = (\lambda - 4)(\lambda - 2) = 0.

So $\lambda_1 = 4$ and $\lambda_2 = 2$ . Two distinct real eigenvalues.

(Cross-check: trace = $3 + 3 = 6 = \lambda_1 + \lambda_2$ . Determinant = $9 - 1 = 8 = \lambda_1 \lambda_2$ . ✓ The trace always equals the sum of eigenvalues; the determinant always equals their product.)

Step 3 — Find the eigenvector for $\lambda_1 = 4$ . Solve $(A - 4 I) \mathbf{v} = \mathbf{0}$ :

A - 4 I \;=\; \begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix}.

The two rows are negatives of each other (as expected — the matrix is singular). The single non-trivial equation is $-v_1 + v_2 = 0$ , i.e. $v_2 = v_1$ . Any non-zero $\mathbf{v} = (1, 1)$ works. Conventionally we normalise:

\mathbf{v}_1 \;=\; \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ 1 \end{pmatrix}.

Step 4 — Find the eigenvector for $\lambda_2 = 2$ . Solve $(A - 2 I) \mathbf{v} = \mathbf{0}$ :

A - 2 I \;=\; \begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}.

The non-trivial equation is $v_1 + v_2 = 0$ , i.e. $v_2 = -v_1$ . Take $\mathbf{v} = (1, -1)$ and normalise:

\mathbf{v}_2 \;=\; \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ -1 \end{pmatrix}.

Step 5 — Verify. Check that $A \mathbf{v}_1 = \lambda_1 \mathbf{v}_1$ :

\begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix} \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \frac{1}{\sqrt{2}} \begin{pmatrix} 4 \\ 4 \end{pmatrix} = 4 \cdot \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \lambda_1 \mathbf{v}_1. \;\checkmark

And $A \mathbf{v}_2 = \lambda_2 \mathbf{v}_2$ :

\begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix} \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ -1 \end{pmatrix} = \frac{1}{\sqrt{2}} \begin{pmatrix} 2 \\ -2 \end{pmatrix} = 2 \cdot \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ -1 \end{pmatrix} = \lambda_2 \mathbf{v}_2. \;\checkmark

Step 6 — Notice. The two eigenvectors are orthogonal: $\mathbf{v}_1 \cdot \mathbf{v}_2 = \tfrac12 (1 \cdot 1 + 1 \cdot (-1)) = 0$ . This is not a coincidence — the matrix $A$ here happens to be symmetric ( $A^T = A$ ), and symmetric matrices always have orthogonal eigenvectors. That fact is the spectral theorem for symmetric matrices, which we will meet in full generality in 4.6. It is what makes self-adjoint operators in PDEs so well-behaved.

When eigenvalues are complex

Not every matrix has real eigenvalues. The matrix

R \;=\; \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}

is a $90°$ rotation. Its characteristic polynomial is $\lambda^2 + 1 = 0$ , with roots $\lambda = \pm i$ . No real direction is preserved under a $90°$ rotation — every arrow turns ninety degrees. The eigenvalues are complex, and the corresponding eigenvectors live in $\mathbb{C}^2$ rather than $\mathbb{R}^2$ . The complex eigenvectors are useful: they let you express the rotation as componentwise complex multiplication in the right basis, recovering the connection to complex exponentials from Foundations 3.

In acoustics and elsewhere, complex eigenvalues typically signal oscillation — they appear in the underdamped regime of the damped oscillator, in the spiral fixed points of Foundations 5.4, and in any time-evolution problem with rotational character.

Power iteration: finding eigenvectors numerically

For large matrices the characteristic polynomial is impractical to solve directly. The standard numerical method instead is power iteration: pick an arbitrary unit vector $\mathbf{v}_0$ and repeatedly apply $A$ , renormalising at each step:

\mathbf{v}_{k+1} \;=\; \frac{A \mathbf{v}_k}{\|A \mathbf{v}_k\|}.

Generically, $\mathbf{v}_k$ converges to the eigenvector associated with the eigenvalue of largest absolute value (the dominant eigenvalue). The intuition: writing $\mathbf{v}_0 = \sum c_i \mathbf{v}_i$ in the eigenvector basis, applying $A$ many times gives $A^k \mathbf{v}_0 = \sum c_i \lambda_i^k \mathbf{v}_i$ . Whichever $\lambda_i$ is largest in absolute value, its term grows fastest, and after enough iterations the sum is dominated by that one mode.

Once the dominant eigenvector is known, the dominant eigenvalue follows from the Rayleigh quotient $\mathbf{v}^T A \mathbf{v} / \mathbf{v}^T \mathbf{v}$ , which approaches the dominant eigenvalue as $\mathbf{v}_k$ approaches its eigenvector.

preset:

start direction θ₀ = 50°

Each click of iterate applies A to the current unit vector and renormalises. Generically the result converges to the dominant eigenvector — drawn as the dashed gold line for reference when the eigenvalues are real. The Rayleigh quotient vᵀA v / vᵀv approaches the dominant eigenvalue at the same time. The complex eigenvalues preset shows what happens with no real eigenvector to attract toward: the vector spins around indefinitely. The shear (degenerate) preset has a single repeated eigenvalue and one eigenvector — convergence still happens but is slower.

Pick a matrix preset, set the starting direction $\theta_0$ , and step the iteration. Watch the vector swing toward the dominant eigenvector (the dashed gold line). The Rayleigh quotient on the side converges to the dominant eigenvalue in parallel. The complex eigenvalues preset shows what happens when no real eigenvector exists — the iteration rotates rather than converging.

Power iteration is the simplest of a family of iterative eigenvalue algorithms that scale to large matrices. The PageRank algorithm Google used to launch in 1998 is power iteration on a billion-by-billion link matrix; the largest eigenvalue’s eigenvector ranks web pages by importance. Modal analysis in audio, image-compression’s principal component analysis (PCA), and many machine-learning algorithms all reduce, at the bottom, to “find the dominant eigenvectors of this matrix.”

⏳ The history — From Cayley to Hilbert: a century building the spectral theorem

Matrix algebra as we know it was assembled by Arthur Cayley and James Joseph Sylvester in the 1850s in England. Cayley’s 1858 Memoir on the Theory of Matrices defined matrix addition, multiplication, and the characteristic polynomial — the equation $\det(A - \lambda I) = 0$ from this lesson. Sylvester coined the word “matrix” in 1850 and introduced “discriminant” and “minor” along with much of the modern vocabulary. The two were friends and lifelong correspondents; the era is sometimes called the Cayley–Sylvester period of algebra.

The eigenvalue–eigenvector machinery was fully understood for finite matrices by the 1880s. The leap to infinite dimensions — operators on function spaces, the natural home of PDEs and quantum mechanics — was made by David Hilbert in the early 1900s, in his work on integral equations. Hilbert’s six papers from 1904–1910 established what we now call Hilbert space, and the proof that self-adjoint operators on a Hilbert space have a complete orthonormal eigenbasis is the spectral theorem, the deepest result in the chain. The full machinery was reformulated and extended by Hilbert’s student John von Neumann in the 1930s, providing the mathematical foundation that Werner Heisenberg’s matrix mechanics and Erwin Schrödinger’s wave mechanics needed to be the same theory. Eigenvalues, in other words, ran the central arc of mathematical physics from 1850 to 1930.

What we use this for

A short tour of where eigenvalues appear in the bookshelf:

ODE phase-plane stability (Foundations 5.4) — the matrix $A$ that linearises a system near a fixed point has eigenvalues whose real parts determine stability and whose imaginary parts determine oscillation.
Mode catalogue of a bounded domain (Foundations 6.5) — the spatial differential operator (with boundary conditions) has eigenfunctions $X_n$ and eigenvalues $-k_n^2$ ; the time evolution decomposes mode by mode.
Helmholtz cavity modes (Foundations 6.7) — exactly an eigenvalue problem: $\nabla^2 \phi = -k^2 \phi$ with boundary conditions, eigenvalues $k_{mn}^2$ giving cavity frequencies $\omega_{mn} = c k_{mn}$ .
Time-independent Schrödinger equation (Foundations 6.8) — $\hat H \psi = E \psi$ is the eigenvalue problem; energy eigenstates are the eigenvectors of the Hamiltonian operator.
The principal modes of vibration of a coupled-oscillator system, like a chain of masses (treated in Sound 3.1) — solving the matrix system gives normal modes as eigenvectors and frequencies as $\sqrt{|\lambda|}$ .
Statistical room acoustics (Sound 7.8) — counting the density of cavity eigenvalues per unit frequency is what gives the Schroeder frequency.

Whenever you see “in the right basis the problem becomes diagonal,” that is eigenvalue analysis happening. The next lesson, 4.5, develops the inner product structure that makes the “right basis” of orthogonal eigenvectors usable for projection — the formal underpinning of Fourier expansion and modal decomposition.