Spectral Theory

Take the unit circle of $\mathbb{R}^2$ and push it through a linear map. What comes out is an ellipse. The ellipse has two axes, at right angles to each other, and it is natural to ask whether those axes sit along directions that the map merely stretches — whether, in other words, the map’s eigenvectors are perpendicular. Usually they are not. Chapter 9 found eigenvectors and asked only that they be independent; a basis of eigenvectors can be as skewed as it likes, and for a matrix like $\begin{pmatrix} 1 & 1 \\ 0 & 2 \end{pmatrix}$ it is. Perpendicular eigenvectors are a strictly stronger demand, and it is the demand that matters in practice: coordinates in an orthonormal basis are read off by one inner product each, the change of basis is an isometry, and nothing is distorted along the way.

This chapter answers the question completely. There is an orthonormal basis of eigenvectors exactly when the operator commutes with its adjoint over $\mathbb{C}$ , and exactly when it equals its adjoint over $\mathbb{R}$ . Those two statements are the spectral theorems, and they are the most used theorems in applied linear algebra. Everything else here grows out of them: the triangular form that every complex matrix has in an orthonormal basis (Schur), the shape of a rotation in three dimensions, several commuting operators diagonalized at once, functions of an operator computed on its spectrum, the discrete Fourier transform as one instance of the whole picture, and finally a number that measures how far a matrix is from being normal. The field is $\mathbb{R}$ or $\mathbb{C}$ throughout, as in Chapter 11, and every statement says which one it needs. The difference between the two is not cosmetic: the Fundamental Theorem of Algebra hands $\mathbb{C}$ an eigenvalue for free, and over $\mathbb{R}$ each eigenvalue has to be earned.

What you need. From Chapter 6, the Fundamental Theorem of Algebra and the splitting it gives (Theorem 6.5.1, Corollary 6.5.5), together with Lagrange interpolation (Theorem 6.6.1). From Chapter 9, eigenvalues and eigenspaces, the characteristic polynomial and multiplicities, the existence of an eigenvalue over $\mathbb{C}$ (Definition 9.2.1, Theorem 9.2.1, Theorem 9.2.4), triangularization and what its diagonal records (Theorem 9.7.2, Corollary 9.7.3, Theorem 9.7.4, Theorem 9.3.3), the conjugate-pair behavior of real matrices (Theorem 9.2.5), and the two results on commuting operators (Theorem 9.9.3, Theorem 9.9.5). From Chapter 1, the complexification of a real vector space (Definition 1.9.2, Theorem 1.9.3). From Chapter 0, the trace with its cyclic invariance, the transpose, polynomials of a matrix and the roots of unity that §09’s Fourier matrix is built from (Theorem 0.9.6, Theorem 0.9.5, Theorem 0.9.9, Example (Roots of Unity)). From Chapter 2, the bookkeeping every proof here does silently: a linear map is free on a basis (Theorem 2.1.5), polynomials of an operator (Definition 2.4.2) and invertibility (Theorem 2.5.7). From Chapter 3, permutation matrices (Lemma 3.4.7) and the Invertible Matrix Theorem (Theorem 3.4.4). From Chapter 4, the matrix of a map and of a composition with the isomorphism onto matrices (Definition 4.1.1, Theorem 4.1.2, Theorem 4.1.4), and change of coordinates with similarity (Definition 4.2.1, Theorem 4.2.3, Theorem 4.2.4). From Chapter 7, determinants: the triangular and block triangular rules, multiplicativity, the invertibility criterion and the characteristic polynomial with its similarity invariance (Theorem 7.4.6, Theorem 7.8.4, Theorem 7.5.4, Theorem 7.5.2, Corollary 7.5.7, Definition 7.10.2, Theorem 7.10.2, Theorem 7.10.3). From Chapter 10, the Jordan form and the rotation-scaling block (Theorem 10.3.1, Definition 10.4.1). From Chapter 8, the Kronecker product and the vec operator (Definition 8.4.1, Theorem 8.4.1, Definition 8.4.2, Theorem 8.4.4). Above all, from Chapter 11: orthonormal bases and Gram–Schmidt (Theorem 11.2.2, Corollary 11.2.3, Theorem 11.2.1), orthogonal complements and projections (Theorem 11.3.2, Definition 11.3.2, Theorem 11.3.3), the adjoint with its arithmetic and its matrix (Definition 11.6.1, Theorem 11.6.2, Theorem 11.6.3, Theorem 11.6.5), the two classes of operators it singles out (Definition 11.6.2, Definition 11.6.3, Proposition 11.6.7), and isometries (Definition 11.7.1, Theorem 11.7.3, Definition 11.7.2, Definition 11.7.3).

Roadmap

Schur’s unitary triangularization. Every complex matrix is upper triangular in some orthonormal basis; the workhorse the rest of the chapter runs on, and the real block version alongside it.
Self-adjoint operators. Real eigenvalues over both fields, orthogonal eigenspaces, invariant complements, and the test $\langle T\mathbf{v}, \mathbf{v} \rangle = 0$ that forces $T = 0$ .
Normal operators. The weakest hypothesis that works over $\mathbb{C}$ : four equivalent descriptions, $\operatorname{ker} T = \operatorname{ker} T^{*}$ , and why a normal operator has no Jordan blocks.
The complex spectral theorem. Normal is exactly the same as having an orthonormal eigenbasis; Schur plus one entry-by-entry computation.
The real spectral theorem. Self-adjoint is exactly the same as having an orthonormal eigenbasis over $\mathbb{R}$ ; the principal axis theorem, and the rotation that shows why “normal” is not enough here.
The shape of an orthogonal matrix. Blocks $[1]$ , $[-1]$ and plane rotations; every element of $\operatorname{SO}(3)$ is a turn about an axis, and reflections generate.
Commuting families. One orthonormal basis that diagonalizes every member of a commuting family of normal operators at once.
Spectral resolutions and functions of a normal operator. Writing $T = \sum \lambda_i P_i$ with orthogonal projections, and defining $f(T)$ by applying $f$ to the spectrum.
Circulants and the discrete Fourier transform. One unitary matrix diagonalizes every circulant, and multiplication becomes convolution.
The Sylvester equation and block diagonalization. When $\mathbf{X} \mapsto \mathbf{A}\mathbf{X} - \mathbf{X}\mathbf{B}$ is invertible, and what that buys for block triangular matrices.
How far from normal? Schur’s inequality turns normality from a yes-or-no property into a measurement.

Named moves

Take one eigenvector, then pass to its orthogonal complement. As soon as the operator is normal or self-adjoint, that complement is invariant, so the induction on dimension runs inside the space instead of in a quotient. This replaces Chapter 9’s “extend a basis and pass to the quotient”.
Compute $\langle T\mathbf{v}, \mathbf{v} \rangle$ two ways. Once with the eigenvalue equation, once by moving $T$ across to $T^{*}$ . Comparing the two is almost the whole of the elementary theory.
Work with $\left\| T\mathbf{v} \right\|^2 = \langle T^{*}T\mathbf{v}, \mathbf{v} \rangle$ . A statement about lengths becomes a statement about the single operator $T^{*}T$ , and a difference of two such operators is tested by the zero test.
Triangular plus normal forces diagonal. Compare one diagonal entry of $\mathbf{A}^{*}\mathbf{A}$ with the same entry of $\mathbf{A}\mathbf{A}^{*}$ , and a row of off-diagonal entries has to vanish.
Split into real and imaginary parts. Write $T = A + iB$ with $A$ and $B$ self-adjoint; then $T$ is normal exactly when $A$ and $B$ commute, and questions about $T$ become questions about a commuting pair.
Diagonalize, then read the answer off the diagonal. Powers, square roots, exponentials and the whole functional calculus are one line once an orthonormal eigenbasis is in hand.