4.5 Inner products and orthogonality

The vector-space axioms — addition and scalar multiplication — let you add and stretch arrows but say nothing about length or angle. Pythagoras’ theorem, the cosine rule, the very idea of “perpendicular” — none of these are part of the bare vector-space structure. To recover them, you need one extra piece of equipment: an inner product.

This lesson defines the inner product, develops the geometric notions it makes possible (length, angle, projection, orthogonality), and walks through the Gram–Schmidt procedure that turns any linearly-independent set of vectors into an orthonormal basis. The same machinery, lifted to function spaces, is what makes Fourier expansion and modal projection work — it is the algebraic structure underlying Foundations 6.5.

The dot product

In $\mathbb{R}^n$ the inner product is the familiar dot product:

This is just a single number — the inner product of two vectors is a scalar, not another vector. Three properties pin it down and generalise to other inner-product spaces:

• Symmetric. $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$.
• Bilinear. $(\alpha \mathbf{u} + \beta \mathbf{w}) \cdot \mathbf{v} = \alpha (\mathbf{u} \cdot \mathbf{v}) + \beta (\mathbf{w} \cdot \mathbf{v})$, and similarly in the second slot.
• Positive definite. $\mathbf{v} \cdot \mathbf{v} \geq 0$, with equality if and only if $\mathbf{v} = \mathbf{0}$.

Any operation on a vector space with these three properties is called an inner product, often written $\langle \mathbf{u}, \mathbf{v} \rangle$ in more abstract settings. The integral $\int_0^L f(x) g(x)\, dx$, for instance, satisfies all three properties (with functions $f, g$ as the “vectors”), and so it is an inner product on the space of integrable functions on $[0, L]$. That fact is what makes the Fourier-projection formula in Foundations 6.5 work mechanically the same way as the projection of a 2-D vector onto an axis.

Length and angle

The inner product gives you a norm (length):

In $\mathbb{R}^2$ this is just Pythagoras: $\|(x, y)\| = \sqrt{x^2 + y^2}$. The norm satisfies the triangle inequality $\|\mathbf{u} + \mathbf{v}\| \leq \|\mathbf{u}\| + \|\mathbf{v}\|$ — a consequence of the Cauchy–Schwarz inequality $|\mathbf{u} \cdot \mathbf{v}| \leq \|\mathbf{u}\|\, \|\mathbf{v}\|$, which is one of the most-used inequalities in mathematics.

The inner product also encodes the angle between two vectors:

Two vectors are orthogonal (perpendicular) if their inner product is zero. In $\mathbb{R}^n$ orthogonality is the algebraic condition $\sum_k u_k v_k = 0$; in function-space settings it becomes $\int f(x) g(x)\, dx = 0$. The integral version is what makes the Fourier sine series of Foundations 6.3 work:

— different-mode sines are orthogonal as elements of the function space $L^2[0, L]$.

Projection

If $\mathbf{u}$ is a unit vector (i.e. $\|\mathbf{u}\| = 1$), the projection of any vector $\mathbf{v}$ onto $\mathbf{u}$ is

The scalar $\mathbf{v} \cdot \mathbf{u}$ is the length of the projection — how much of $\mathbf{v}$ points along $\mathbf{u}$. The vector $(\mathbf{v} \cdot \mathbf{u})\, \mathbf{u}$ is the projection itself, a vector parallel to $\mathbf{u}$.

The residual $\mathbf{w} = \mathbf{v} - \text{proj}_\mathbf{u}(\mathbf{v})$ is the component of $\mathbf{v}$ perpendicular to $\mathbf{u}$. Two checks: $\mathbf{u} \cdot \mathbf{w} = \mathbf{u} \cdot \mathbf{v} - (\mathbf{v} \cdot \mathbf{u}) \cdot 1 = 0$ (so $\mathbf{w} \perp \mathbf{u}$ ✓), and $\mathbf{v} = \text{proj}_\mathbf{u}(\mathbf{v}) + \mathbf{w}$ (so the decomposition is exact ✓).

Projection is the bread-and-butter geometric operation of linear algebra. If $\{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$ is an orthonormal basis, then any vector decomposes as

That formula is the Fourier expansion of $\mathbf{v}$ in the basis $\{\mathbf{e}_k\}$. When the basis is the modes of a PDE rather than $\mathbb{R}^n$ basis vectors, the same formula extracts modal coefficients from initial data; that is the Fourier-projection step in Foundations 6.5.

Gram–Schmidt orthogonalisation

Given a set of linearly-independent vectors $\{\mathbf{a}_1, \mathbf{a}_2, \ldots, \mathbf{a}_k\}$, we want to produce an orthonormal set $\{\mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_k\}$ spanning the same space — orthogonal to each other and each of unit length. The Gram–Schmidt procedure does this iteratively, by subtracting projections.

The recipe for two input vectors $\mathbf{a}$ and $\mathbf{b}$:

1. $\mathbf{e}_1 = \mathbf{a} / \|\mathbf{a}\|$. (Normalise the first vector.)
2. $\mathbf{w} = \mathbf{b} - (\mathbf{b} \cdot \mathbf{e}_1)\, \mathbf{e}_1$. (Subtract from $\mathbf{b}$ its component along $\mathbf{e}_1$.)
3. $\mathbf{e}_2 = \mathbf{w} / \|\mathbf{w}\|$. (Normalise.)

For three or more inputs the procedure continues: subtract from each subsequent vector its components along all previously orthonormalised vectors, then normalise.

Drag the blue and red input vectors $\mathbf{a}$ and $\mathbf{b}$. Step through the three operations. The little right-angle marker in step 2 confirms that the green residual $\mathbf{w}$ is perpendicular to $\mathbf{e}_1$; in step 3 it gets normalised to $\mathbf{e}_2$. The output basis $\{\mathbf{e}_1, \mathbf{e}_2\}$ spans the same plane as the input but with orthogonal axes.

Worked example

Inner products on function spaces

The bridge to PDEs: the same recipe — projection, orthogonality, Gram–Schmidt — works on infinite-dimensional spaces of functions, provided the inner product is the integral

(possibly weighted; the Sturm–Liouville theorem describes which weight functions are appropriate for which differential operators). The functions $\sin(n \pi x / L)$ on $[0, L]$ are pairwise orthogonal under this inner product, with $\|sin(n \pi x / L)\|^2 = L/2$. Therefore the Fourier sine series

is exactly the orthonormal-basis expansion $\mathbf{v} = \sum_k (\mathbf{v} \cdot \mathbf{e}_k)\, \mathbf{e}_k$ from the finite-dimensional case, lifted to the function space. The arithmetic is identical; the only difference is that “sum” becomes “integral” in the inner product, and "$\sin(n\pi x/L)$" plays the role of $\mathbf{e}_n$. This is the deep reason separation of variables and Fourier expansion work: the natural eigenfunctions of self-adjoint differential operators are orthogonal under an appropriate inner product, and the spectral theorem in 4.6 guarantees their completeness as a basis.

What we use this for

Inner products are everywhere across the bookshelf:

• Fourier-projection for mode coefficients (Foundations 6.5) — the $A_n$ formula is an inner product divided by a norm squared.
• Orthogonality of mode shapes in Sound 7.6 — tube modes, Sound 7.7 — cavity modes, Sound 7.8 — room modes — distinct cavity modes are orthogonal under the appropriate volume integral.
• Vector projection in 3-D geometry — every plane-wave acoustic problem involves projecting a wavevector $\mathbf{k}$ onto direction vectors, computing inner products like $\mathbf{k} \cdot \mathbf{r}$ to find the phase at a position.
• Inner products of complex signals in Hearing 5.3 — phase-locking — vector strength is a circular-mean inner product of $e^{i \phi}$ unit vectors, generalising the dot product to complex unit phasors.
• Optimal control and matched filters — the matched filter that maximises signal-to-noise ratio for detecting a signal $s(t)$ in noise is the inner product of the received waveform with $s(t)$ itself, just as projection extracts the component of one vector along another.

The next lesson, 4.6, ties together everything in the chapter: the eigenvalues of 4.4, the orthogonality of this lesson, and the spectral theorem that guarantees a self-adjoint operator’s eigenvectors form a complete orthonormal basis. That theorem is what makes mode expansions for PDEs not just convenient but mathematically complete.