14.6 Definiteness and Second Derivatives

Chapter 13 studied positive definite matrices at length, and the previous section showed that the signature is what congruence remembers. Putting the two together costs almost nothing and buys two things: a vocabulary that covers the forms Chapter 13 had no name for, the indefinite and the semidefinite ones, and the classical application that made quadratic forms worth studying in the first place. Nothing from Chapter 13 is reproved here. The field is $\mathbb{R}$ throughout, since none of the five words below means anything without an order.

Throughout, $V$ is a finite-dimensional real vector space, $\beta$ is a symmetric bilinear form on $V$ (Definition 14.3.1), $q(\mathbf{v}) = \beta(\mathbf{v},\mathbf{v})$ is its quadratic form (Definition 14.3.2), and $\mathbf{A} = [\beta]_{\mathcal{B}}^{\mathcal{B}}$ is its matrix in a basis $\mathcal{B}$ (Definition 14.1.2), a real symmetric matrix.

14.6.1. Five words for the sign of a form

Chapter 13 named two of the possible behaviors of $q$ : never negative, and always positive away from $\mathbf{0}$ . There are three more, and the missing ones are not exotic. The form $q(x_1, x_2) = x_1^2 - x_2^2$ is positive at $\mathbf{e}_1$ , negative at $\mathbf{e}_2$ , and nothing in Chapter 13’s vocabulary describes it.

A form is definite when $q$ has one strict sign, semidefinite when it has one sign but is allowed to vanish, and indefinite when it has both signs.

Definition 14.6.1 (Definiteness of a Real Symmetric Form)

Let $\beta$ be a symmetric bilinear form on a real vector space $V$ , with quadratic form $q$ . Then $\beta$ is:

(a) positive definite if $q(\mathbf{v}) > 0$ for every non-zero $\mathbf{v} \in V$ ;
(b) positive semidefinite if $q(\mathbf{v}) \ge 0$ for every $\mathbf{v} \in V$ ;
(c) negative definite if $q(\mathbf{v}) < 0$ for every non-zero $\mathbf{v} \in V$ ;
(d) negative semidefinite if $q(\mathbf{v}) \le 0$ for every $\mathbf{v} \in V$ ;
(e) indefinite if there exist $\mathbf{u}, \mathbf{w} \in V$ with $q(\mathbf{u}) > 0$ and $q(\mathbf{w}) < 0$ .

The same five words apply to a real symmetric matrix $\mathbf{A}$ , through the form $\beta(\mathbf{x},\mathbf{y}) = \mathbf{x}^{\top}\mathbf{A}\mathbf{y}$ on $\mathbb{R}^n$ .

The small words carry the weight. In (a) and (c) the inequality is strict and the vector must be non-zero; in (b) and (d) it is weak and $\mathbf{v} = \mathbf{0}$ is allowed, where $q(\mathbf{0}) = 0$ makes the condition automatic. Definite implies semidefinite of the same sign. Clause (e) is not the negation of (b): $q(x_1,x_2) = -x_1^2$ is neither positive semidefinite nor indefinite. For matrices, (a) and (b) are Definition 13.1.1 restricted to $F = \mathbb{R}$ , so $\mathbf{A} \succ 0$ and $\mathbf{A} \succeq 0$ keep their meanings, and the negative versions are the positive ones for $-\mathbf{A}$ .

Four examples on $\mathbb{R}^2$ , all diagonal so that $q$ can be read off Eq. 14.4.1: $x_1^2 + x_2^2$ is positive definite; $x_1^2$ is positive semidefinite but not definite, since it vanishes at $\mathbf{e}_2 \ne \mathbf{0}$ ; $-x_1^2 - x_2^2$ is negative definite; $x_1^2 - x_2^2$ is indefinite. The zero form is the degenerate case, satisfying (b) and (d) and nothing else — which is why the five classes overlap rather than partition. Changing one sign in the third gives the non-example $-x_1^2 + x_2^2$ : it is no longer negative semidefinite, and the clause that fails is (d) at $\mathbf{v} = \mathbf{e}_2$ , where $q = 1 > 0$ .

Now the reading through the signature, which makes all five conditions into statements about three integers.

Theorem 14.6.1 (Definiteness Is the Signature)

Let $\beta$ be a symmetric bilinear form on a real vector space $V$ of dimension $n$ , with inertia $(n_+, n_-, n_0)$ (Definition 14.5.1). Then:

(a) $\beta$ is positive definite $\iff$ $(n_+, n_-, n_0) = (n, 0, 0)$ ;
(b) $\beta$ is positive semidefinite $\iff$ $n_- = 0$ ;
(c) $\beta$ is negative definite $\iff$ $(n_+, n_-, n_0) = (0, n, 0)$ ;
(d) $\beta$ is negative semidefinite $\iff$ $n_+ = 0$ ;
(e) $\beta$ is indefinite $\iff$ $n_+ > 0$ and $n_- > 0$ .

Idea. Fix an orthogonal basis. Then $q$ is $\sum_i d_i c_i^2$ with no cross terms, and each of the five conditions becomes a visible condition on the signs of the $d_i$ : plugging in a basis vector reads off one $d_i$ , and a general $\mathbf{v}$ mixes the $d_i$ with non-negative weights. Sylvester’s law is what lets us fix one basis and still be talking about $\beta$ .

Proof. By Theorem 14.4.2 there is an orthogonal basis $\mathcal{B} = (\mathbf{v}_1, \dots, \mathbf{v}_n)$ for $\beta$ ; put $d_i = q(\mathbf{v}_i)$ . By Theorem 14.5.1 the numbers of positive, negative and zero $d_i$ are $n_+$ , $n_-$ and $n_0$ , whichever orthogonal basis is used. For $\mathbf{v} = \sum_i c_i\mathbf{v}_i$ , Eq. 14.4.1 gives $q(\mathbf{v}) = \sum_{i=1}^{n} d_i c_i^2 .$

(b) $(\Rightarrow)$ If some $d_j < 0$ then $q(\mathbf{v}_j) = d_j < 0$ , so $\beta$ is not positive semidefinite. $(\Leftarrow)$ If no $d_i$ is negative then every term $d_ic_i^2$ is $\ge 0$ , so $q(\mathbf{v}) \ge 0$ .
(c) $(\Rightarrow)$ A positive definite $\beta$ is positive semidefinite, so $n_- = 0$ by (b); and if some $d_j = 0$ then $q(\mathbf{v}_j) = 0$ with $\mathbf{v}_j \ne \mathbf{0}$ , a contradiction. Hence $n_0 = 0$ and $n_+ = n$ . $(\Leftarrow)$ If every $d_i > 0$ and $\mathbf{v} \ne \mathbf{0}$ , some $c_j \ne 0$ , so $q(\mathbf{v}) \ge d_jc_j^2 > 0$ , the other terms being $\ge 0$ .
(d) and (d) follow by applying (a) and (b) to $-\beta$ , whose diagonal entries are $-d_i$ and whose inertia is therefore $(n_-, n_+, n_0)$ .
(e) $(\Rightarrow)$ Suppose $q(\mathbf{u}) > 0$ and $q(\mathbf{w}) < 0$ . Then $\beta$ is neither positive semidefinite nor negative semidefinite, so $n_- > 0$ by (b) and $n_+ > 0$ by (d). $(\Leftarrow)$ If $n_+ > 0$ and $n_- > 0$ , some $d_i > 0$ and some $d_j < 0$ , and the basis vectors $\mathbf{v}_i, \mathbf{v}_j$ are the required $\mathbf{u}, \mathbf{w}$ . This proves the theorem.

So the five words sort the $\tfrac12(n+1)(n+2)$ congruence classes of Corollary 14.5.2 into named families, and every one of them is decided by counting signs. Two of these five verdicts were already settled in Chapter 13 by other means, and it is worth putting the tests side by side.

Proposition 14.6.2 (Chapter 13’s Tests, in the Language of Forms)

Let $\beta$ be a symmetric bilinear form on an $n$ -dimensional real vector space, $\mathbf{A} = [\beta]_{\mathcal{B}}^{\mathcal{B}}$ its matrix in any basis, and $\mathbf{A}_k$ the top-left $k \times k$ corner of $\mathbf{A}$ . Then:

(a) $\beta$ is positive definite $\iff$ every eigenvalue of $\mathbf{A}$ is positive $\iff$ $\det \mathbf{A}_k > 0$ for $k = 1, \dots, n$ ;
(b) $\beta$ is negative definite $\iff$ every eigenvalue of $\mathbf{A}$ is negative $\iff$ $(-1)^k\det \mathbf{A}_k > 0$ for $k = 1, \dots, n$ .

Proof.

(a) By Definition 14.1.2 and Theorem 14.1.1, $q(\mathbf{v}) = \mathbf{x}^{\top}\mathbf{A}\mathbf{x}$ where $\mathbf{x} = [\mathbf{v}]_{\mathcal{B}}$ , and $\mathbf{v} \ne \mathbf{0}$ exactly when $\mathbf{x} \ne \mathbf{0}$ . So $\beta$ is positive definite exactly when $\mathbf{A} \succ 0$ , and the two stated tests are Theorem 13.1.5 (b) and (d), the second being Sylvester’s criterion.
(b) Apply (a) to $-\beta$ , whose matrix is $-\mathbf{A}$ . The eigenvalues of $-\mathbf{A}$ are the negatives of those of $\mathbf{A}$ , and $(-\mathbf{A})_k = -\mathbf{A}_k$ has $\det(-\mathbf{A}_k) = (-1)^k\det\mathbf{A}_k$ , since scaling each of the $k$ rows by $-1$ multiplies the determinant by $-1$ once per row (Theorem 7.5.1 (b)). This proves the proposition.

The eigenvalue test is the one to think with, the minor test the one to compute with; cheapest of all is the symmetric elimination of Section 4, which returns the whole signature rather than a verdict. Nothing is said about the semidefinite cases on purpose: Sylvester’s criterion has no weak version, as Chapter 13 showed with $\operatorname{diag}(0,-1)$ , whose leading minors are both $0$ .

Quick check. A real symmetric $\mathbf{A} \in M_3(\mathbb{R})$ has $\det \mathbf{A}_1 = -2$ , $\det \mathbf{A}_2 = 5$ , $\det \mathbf{A}_3 = -1$ . What is its definiteness?

Solution. Negative definite. The signs alternate starting with a negative one, so $(-1)^k\det\mathbf{A}_k > 0$ for $k = 1, 2, 3$ : indeed $(-1)(-2) = 2 > 0$ , $(+1)(5) = 5 > 0$ , $(-1)(-1) = 1 > 0$ . By Proposition 14.6.2 (b) the form is negative definite, and by Theorem 14.6.1 (c) its inertia is $(0, 3, 0)$ . Note that $\det \mathbf{A}_3 = \det\mathbf{A} < 0$ on its own would have been no evidence at all.

14.6.2. The second-derivative test

Here is the use. A differentiable function $f \colon \mathbb{R}^n \to \mathbb{R}$ is constant to first order at a critical point, so whether the point is a minimum, a maximum or neither is decided — usually — by the second-order behavior. That behavior is a quadratic form, and Theorem 14.6.1 decides it. The linear algebra is the whole content; what has to be imported is one theorem of analysis, and we state it rather than hide it.

Definition 14.6.2 (Hessian)

Let $f$ be a real-valued function on an open set $U \subseteq \mathbb{R}^n$ , twice continuously differentiable on $U$ , and let $\mathbf{a} \in U$ . The Hessian of $f$ at $\mathbf{a}$ is the matrix $\mathbf{H}_f(\mathbf{a}) \in M_n(\mathbb{R}), \qquad \bigl(\mathbf{H}_f(\mathbf{a})\bigr)_{ij} = \frac{\partial^2 f}{\partial x_i\,\partial x_j}(\mathbf{a}) .$ A point $\mathbf{a} \in U$ with $\nabla f(\mathbf{a}) = \mathbf{0}$ is a critical point of $f$ .

Two facts about $\mathbf{H}_f(\mathbf{a})$ are needed, and neither is linear algebra. They are exactly the analysis this book imports, so we set them out as assumptions rather than let them pass unnoticed. Let $f$ be twice continuously differentiable on an open $U \subseteq \mathbb{R}^n$ and let $\mathbf{a} \in U$ .

(A1) (Symmetry of the second derivatives.) The mixed partial derivatives satisfy $\partial^2f/\partial x_i\partial x_j = \partial^2f/\partial x_j\partial x_i$ on $U$ , so $\mathbf{H}_f(\mathbf{a})$ is a symmetric matrix.
(A2) (Taylor’s theorem with a second-order remainder.) For every $\mathbf{h}$ small enough that $\mathbf{a} + \mathbf{h} \in U$ , $f(\mathbf{a} + \mathbf{h}) = f(\mathbf{a}) + \nabla f(\mathbf{a})\cdot\mathbf{h} + \tfrac12\,\mathbf{h}^{\top}\mathbf{H}_f(\mathbf{a})\mathbf{h} + r(\mathbf{h}),$ where the remainder satisfies $r(\mathbf{h})/\left\| \mathbf{h} \right\|^2 \to 0$ as $\mathbf{h} \to \mathbf{0}$ .

These are two standard theorems of multivariable calculus. This book proves neither. They are quoted so that the theorem below has hypotheses to stand on, and Chapter 18 quotes them again for its second-derivative test of convexity, exactly as the theorem of Abel and Ruffini was quoted in Chapter 11. Everything after them is linear algebra.

By (A1) the Hessian is a real symmetric matrix, so it has an inertia, and by Theorem 14.6.1 that inertia decides the definiteness of the quadratic form $\mathbf{h} \mapsto \mathbf{h}^{\top}\mathbf{H}_f(\mathbf{a})\mathbf{h}$ . One small lemma converts definiteness into a quantitative bound, and it is the only place where a real number is needed rather than a sign.

Lemma 14.6.3 (A Positive Definite Form Is Bounded Below by a Multiple of the Norm Squared)

Let $\mathbf{A} \in M_n(\mathbb{R})$ be symmetric and positive definite, and let $\lambda$ be its smallest eigenvalue. Then $\lambda > 0$ and $\mathbf{x}^{\top}\mathbf{A}\mathbf{x} \ \ge\ \lambda\left\| \mathbf{x} \right\|^2 \qquad \text{for every } \mathbf{x} \in \mathbb{R}^n .$

Proof. By Corollary 12.5.2 there are an orthogonal $\mathbf{Q} \in \operatorname{O}(n)$ and a diagonal $\mathbf{D} = \operatorname{diag}(\lambda_1, \dots, \lambda_n)$ listing the eigenvalues of $\mathbf{A}$ with $\mathbf{A} = \mathbf{Q}\mathbf{D}\mathbf{Q}^{\top}$ . By Theorem 13.1.5 (b) every $\lambda_i > 0$ , so $\lambda = \min_i \lambda_i > 0$ . Put $\mathbf{y} = \mathbf{Q}^{\top}\mathbf{x}$ . Then $\mathbf{x}^{\top}\mathbf{A}\mathbf{x} = \mathbf{y}^{\top}\mathbf{D}\mathbf{y} = \sum_{i=1}^n \lambda_i y_i^2 \ \ge\ \lambda\sum_{i=1}^n y_i^2 = \lambda\left\| \mathbf{y} \right\|^2 ,$ using $\lambda_i \ge \lambda$ and $y_i^2 \ge 0$ termwise. Finally $\mathbf{Q}\mathbf{Q}^{\top} = \mathbf{I}$ , so $\left\| \mathbf{y} \right\|^2 = \mathbf{x}^{\top}\mathbf{Q}\mathbf{Q}^{\top}\mathbf{x} = \left\| \mathbf{x} \right\|^2$ . This proves the lemma.

Theorem 14.6.4 (The Second-Derivative Test)

Let $f$ be a real-valued function, twice continuously differentiable on an open set $U \subseteq \mathbb{R}^n$ , let $\mathbf{a} \in U$ be a critical point of $f$ , and let $(n_+, n_-, n_0)$ be the inertia of $\mathbf{H}_f(\mathbf{a})$ . Then:

(a) if $\mathbf{H}_f(\mathbf{a})$ is positive definite, that is $(n_+, n_-, n_0) = (n, 0, 0)$ , then $\mathbf{a}$ is a strict local minimum of $f$ ;
(b) if $\mathbf{H}_f(\mathbf{a})$ is negative definite, that is $(n_+, n_-, n_0) = (0, n, 0)$ , then $\mathbf{a}$ is a strict local maximum of $f$ ;
(c) if $\mathbf{H}_f(\mathbf{a})$ is indefinite, that is $n_+ > 0$ and $n_- > 0$ , then $\mathbf{a}$ is neither a local minimum nor a local maximum: every neighborhood of $\mathbf{a}$ contains points where $f > f(\mathbf{a})$ and points where $f < f(\mathbf{a})$ . Such an $\mathbf{a}$ is a saddle point.

If $n_0 > 0$ and $\mathbf{a}$ is not covered by (a)–(c), the test gives no information.

Idea. Taylor says $f(\mathbf{a}+\mathbf{h}) - f(\mathbf{a})$ is $\tfrac12 q(\mathbf{h})$ plus a remainder that is smaller than $\left\| \mathbf{h} \right\|^2$ near $\mathbf{0}$ , where $q(\mathbf{h}) = \mathbf{h}^{\top}\mathbf{H}_f(\mathbf{a})\mathbf{h}$ . For (a) the quadratic term is at least $\tfrac12\lambda\left\| \mathbf{h} \right\|^2$ by Lemma 14.6.3, which beats a remainder eventually smaller than $\tfrac14\lambda\left\| \mathbf{h} \right\|^2$ ; so the difference is positive. For (c) there is no bound in every direction, and none is needed: walk along a single line $\mathbf{h} = t\mathbf{u}$ where $q(\mathbf{u}) > 0$ , and the whole thing is $t^2$ times something tending to $\tfrac12q(\mathbf{u}) > 0$ . One direction up, one direction down, and no extremum.

Proof. Write $\mathbf{H} = \mathbf{H}_f(\mathbf{a})$ and $q(\mathbf{h}) = \mathbf{h}^{\top}\mathbf{H}\mathbf{h}$ . Since $\mathbf{a}$ is critical, $\nabla f(\mathbf{a}) = \mathbf{0}$ , so (A2) reads $f(\mathbf{a}+\mathbf{h}) - f(\mathbf{a}) = \tfrac12 q(\mathbf{h}) + r(\mathbf{h}), \qquad \frac{r(\mathbf{h})}{\left\| \mathbf{h} \right\|^2} \to 0 . \tag{$\ast$}$

(a) Let $\lambda > 0$ be the smallest eigenvalue of $\mathbf{H}$ , so that $q(\mathbf{h}) \ge \lambda\left\| \mathbf{h} \right\|^2$ by Lemma 14.6.3. Since $r(\mathbf{h})/\left\| \mathbf{h} \right\|^2 \to 0$ , there is $\delta > 0$ with $\mathbf{a} + \mathbf{h} \in U$ and $\lvert r(\mathbf{h})\rvert \le \tfrac14\lambda\left\| \mathbf{h} \right\|^2$ whenever $0 < \left\| \mathbf{h} \right\| < \delta$ . For such $\mathbf{h}$ , by $(\ast)$ , $f(\mathbf{a}+\mathbf{h}) - f(\mathbf{a}) \ \ge\ \tfrac12\lambda\left\| \mathbf{h} \right\|^2 - \tfrac14\lambda\left\| \mathbf{h} \right\|^2 = \tfrac14\lambda\left\| \mathbf{h} \right\|^2 > 0 .$ So $f(\mathbf{a} + \mathbf{h}) > f(\mathbf{a})$ for every $\mathbf{h} \ne \mathbf{0}$ with $\left\| \mathbf{h} \right\| < \delta$ , which is what “strict local minimum” says.
(b) The Hessian of $-f$ at $\mathbf{a}$ is $-\mathbf{H}$ , whose inertia is $(n, 0, 0)$ because that of $\mathbf{H}$ is $(0, n, 0)$ ; so $-\mathbf{H}$ is positive definite by Theorem 14.6.1 (a). Part (a) applied to $-f$ makes $\mathbf{a}$ a strict local minimum of $-f$ , that is a strict local maximum of $f$ .
(c) By Theorem 14.6.1 (e) there are $\mathbf{u}, \mathbf{w} \in \mathbb{R}^n$ with $q(\mathbf{u}) > 0$ and $q(\mathbf{w}) < 0$ ; both are non-zero, since $q(\mathbf{0}) = 0$ . Fix $t \ne 0$ small and put $\mathbf{h} = t\mathbf{u}$ . Then $q(t\mathbf{u}) = t^2q(\mathbf{u})$ , so $(\ast)$ gives $\frac{f(\mathbf{a} + t\mathbf{u}) - f(\mathbf{a})}{t^2} = \tfrac12 q(\mathbf{u}) + \frac{r(t\mathbf{u})}{t^2} .$ Now $r(t\mathbf{u})/t^2 = \left\| \mathbf{u} \right\|^2\cdot r(t\mathbf{u})/\left\| t\mathbf{u} \right\|^2 \to 0$ as $t \to 0$ , because $\left\| t\mathbf{u} \right\| \to 0$ and $\left\| \mathbf{u} \right\|^2$ is a constant. So for all small enough $t \ne 0$ the right-hand side is at least $\tfrac14q(\mathbf{u}) > 0$ , and hence $f(\mathbf{a} + t\mathbf{u}) > f(\mathbf{a})$ . The same computation with $\mathbf{w}$ in place of $\mathbf{u}$ gives $f(\mathbf{a} + t\mathbf{w}) < f(\mathbf{a})$ for all small enough $t \ne 0$ . Every neighborhood of $\mathbf{a}$ contains points of both kinds, so $\mathbf{a}$ is neither a local minimum nor a local maximum. This proves the theorem.

Cases (a)–(c) exhaust the non-degenerate ones: if $n_0 = 0$ then $n_+ + n_- = n$ , so either $n_+ = n$ , or $n_- = n$ , or both are positive. What is left undecided is exactly $n_0 > 0$ , and there the test is silent because it has to be.

Example (Two critical points of a cubic surface)

Find the critical points of $f(x,y) = x^3 - 3xy + y^3$ on $\mathbb{R}^2$ and classify each one.

Solution. The partial derivatives are $f_x = 3x^2 - 3y$ and $f_y = -3x + 3y^2$ . Setting both to zero gives $y = x^2$ and $x = y^2$ , hence $x = x^4$ , so $x(x^3 - 1) = 0$ and $x \in \{0, 1\}$ . The real critical points are $(0,0)$ and $(1,1)$ .

Differentiating again, $\mathbf{H}_f(x,y) = \begin{pmatrix} 6x & -3 \\ -3 & 6y\end{pmatrix} .$

At $(0,0)$ the Hessian is $\begin{psmallmatrix} 0 & -3 \\ -3 & 0\end{psmallmatrix}$ , with $\det = -9 < 0$ . A $2 \times 2$ symmetric matrix with negative determinant has eigenvalue product $-9$ , hence one eigenvalue of each sign, so its inertia is $(1,1,0)$ by Theorem 14.5.3 and the form is indefinite. By Theorem 14.6.4 (c), $(0,0)$ is a saddle point. Concretely $f(t,t) = 2t^3 - 3t^2 < 0$ and $f(t,-t) = 3t^2 > 0$ for small $t > 0$ .

At $(1,1)$ the Hessian is $\begin{psmallmatrix} 6 & -3 \\ -3 & 6\end{psmallmatrix}$ , whose leading principal minors are $6 > 0$ and $36 - 9 = 27 > 0$ . By Proposition 14.6.2 (a) it is positive definite, so $(1,1)$ is a strict local minimum, with value $f(1,1) = -1$ .

Example (One Hessian, three answers)

The three functions $g_1 = x^2 + y^4, \qquad g_2 = x^2 - y^4, \qquad g_3 = x^2$ all have $\nabla g_i(0,0) = \mathbf{0}$ and the same Hessian at the origin. What is that Hessian, and what happens at the origin in each case?

Solution. Every second derivative of $y^4$ vanishes at $y = 0$ , so all three have $\mathbf{H}_{g_i}(0,0) = \begin{pmatrix} 2 & 0 \\ 0 & 0\end{pmatrix},$ with inertia $(1, 0, 1)$ : positive semidefinite, not definite, not indefinite. The test is silent, and it must be, because the three answers differ. For $g_1$ the origin is a strict local minimum, since $x^2 + y^4 > 0$ away from $\mathbf{0}$ . For $g_2$ it is a saddle: $g_2(t,0) = t^2 > 0$ and $g_2(0,t) = -t^4 < 0$ for $t \ne 0$ . For $g_3$ it is a minimum but not a strict one, since $g_3$ vanishes along the whole $y$ -axis. The quadratic term decided nothing here; the quartic term decided everything.

Warning. The test reads the signature, not the determinant. In two variables the familiar rule is “ $\det \mathbf{H} > 0$ and $f_{xx} > 0$ means a minimum”, and the second half is not decoration: $\operatorname{diag}(-1,-1)$ has $\det = 1 > 0$ , and $-x^2-y^2$ has a strict local maximum at the origin. All $\det\mathbf{H} > 0$ says in two variables is that $n_0 = 0$ and $n_-$ is even — definite, of a sign the determinant cannot see. In three variables it says even less: $\operatorname{diag}(1,-1,-1)$ has $\det = 1 > 0$ and is indefinite.

14.6.3. Exercises

A. Check your understanding

Exercise (A1)

(a) Define what it means for a symmetric bilinear form on a real vector space to be indefinite.
(b) State the signature condition equivalent to negative semidefiniteness, and the one equivalent to positive definiteness.
(c) Determine whether the following statement is correct, and justify your answer: a symmetric form that is not positive semidefinite is indefinite.
(d) Name the two results of analysis that Theorem 14.6.4 assumes, and say which part of its proof uses each.
(e) A critical point has $n_0 = 2$ and $n_- = 0$ . What does Theorem 14.6.4 conclude?

Solution.

(a) $\beta$ is indefinite if there exist $\mathbf{u}$ and $\mathbf{w}$ in $V$ with $q(\mathbf{u}) > 0$ and $q(\mathbf{w}) < 0$ , where $q(\mathbf{v}) = \beta(\mathbf{v},\mathbf{v})$ .
(b) Negative semidefinite $\iff n_+ = 0$ ; positive definite $\iff (n_+, n_-, n_0) = (n, 0, 0)$ , equivalently $n_+ = n$ . Both are Theorem 14.6.1.
(c) Incorrect. The form $q(x_1,x_2) = -x_1^2$ has inertia $(0,1,1)$ ; it is not positive semidefinite, since $q(\mathbf{e}_1) = -1$ , and not indefinite, since $q$ is never positive. Failing (b) of Definition 14.6.1 means only $n_- > 0$ ; being indefinite needs $n_+ > 0$ as well.
(d) Symmetry of the mixed second partial derivatives, which makes the Hessian symmetric so that it has an inertia at all; and Taylor’s theorem with a second-order remainder, which is the identity $(\ast)$ that every part of the proof manipulates.
(e) Nothing. Here $n_+ = n - 2$ and $n_- = 0$ , so the Hessian is positive semidefinite but not definite and not indefinite, and none of (a), (b), (c) applies. Example (One Hessian, three answers) shows that all of “strict minimum”, “non-strict minimum” and “saddle” really do occur in this situation.

B. Practice

Exercise (B1: Classify by the signature)

Determine the definiteness of each of the following real symmetric matrices, and give the inertia of each. Justify your answers. $\mathbf{A}_1 = \begin{pmatrix} 2 & 1 \\ 1 & 2\end{pmatrix},\quad \mathbf{A}_2 = \begin{pmatrix} -1 & 2 \\ 2 & -5\end{pmatrix},\quad \mathbf{A}_3 = \begin{pmatrix} 1 & 2 \\ 2 & 1\end{pmatrix}.$

Solution. For $\mathbf{A}_1$ : $\det(\mathbf{A}_1)_1 = 2 > 0$ and $\det \mathbf{A}_1 = 4 - 1 = 3 > 0$ , so $\mathbf{A}_1$ is positive definite by Proposition 14.6.2 (a), with inertia $(2,0,0)$ .

For $\mathbf{A}_2$ : $(-1)^1\det(\mathbf{A}_2)_1 = 1 > 0$ and $(-1)^2\det\mathbf{A}_2 = 5 - 4 = 1 > 0$ , so $\mathbf{A}_2$ is negative definite by Proposition 14.6.2 (b), with inertia $(0,2,0)$ .

For $\mathbf{A}_3$ : $\det\mathbf{A}_3 = 1 - 4 = -3 < 0$ , so the two eigenvalues have product $-3$ and hence opposite signs. The inertia is $(1,1,0)$ by Theorem 14.5.3, and $\mathbf{A}_3$ is indefinite by Theorem 14.6.1 (e). Explicitly $q(1,1) = 6 > 0$ and $q(1,-1) = -2 < 0$ .

Exercise (B2: Critical points of a quartic)

Find all critical points of $f(x,y) = x^4 + y^4 - 4xy$ and classify each with Theorem 14.6.4.

Solution. $f_x = 4x^3 - 4y$ and $f_y = 4y^3 - 4x$ , so $y = x^3$ and $x = y^3 = x^9$ . Then $x(x^8 - 1) = 0$ , so $x \in \{0, 1, -1\}$ and the critical points are $(0,0)$ , $(1,1)$ , $(-1,-1)$ . The Hessian is $\mathbf{H}_f(x,y) = \begin{pmatrix} 12x^2 & -4 \\ -4 & 12y^2\end{pmatrix}.$ At $(0,0)$ it is $\begin{psmallmatrix} 0 & -4 \\ -4 & 0\end{psmallmatrix}$ , with determinant $-16 < 0$ , hence inertia $(1,1,0)$ and a saddle point. At $(\pm1,\pm1)$ it is $\begin{psmallmatrix} 12 & -4 \\ -4 & 12\end{psmallmatrix}$ , with leading minors $12 > 0$ and $144 - 16 = 128 > 0$ , hence positive definite: both are strict local minima, each of value $f = 1 + 1 - 4 = -2$ .

C. Going deeper

Exercise (C1: The signature measures positive subspaces)

Let $\beta$ be a symmetric bilinear form on a real vector space $V$ of dimension $n$ , with inertia $(n_+, n_-, n_0)$ . Call a subspace $U \subseteq V$ positive if $q(\mathbf{v}) > 0$ for every non-zero $\mathbf{v} \in U$ .

(a) Prove that $V$ has a positive subspace of dimension $n_+$ .
(b) Prove that every positive subspace has dimension at most $n_+$ . Hint: consider a subspace on which $q \le 0$ of the largest available dimension.
(c) Hence deduce that $n_+$ equals the largest dimension of a positive subspace, and that this gives a second proof that $n_+$ does not depend on the orthogonal basis.

Solution. Fix an orthogonal basis $\mathcal{B} = (\mathbf{v}_1, \dots, \mathbf{v}_n)$ for $\beta$ , ordered so that $d_i = q(\mathbf{v}_i) > 0$ for $i \le n_+$ , then negative, then zero.

(a) Put $U = \operatorname{span}(\mathbf{v}_1, \dots, \mathbf{v}_{n_+})$ . For a non-zero $\mathbf{v} = \sum_{i \le n_+} c_i\mathbf{v}_i$ , Eq. 14.4.1 gives $q(\mathbf{v}) = \sum_{i\le n_+} d_ic_i^2 > 0$ , since every $d_i > 0$ and some $c_i \ne 0$ . So $U$ is positive of dimension $n_+$ .
(b) Put $U' = \operatorname{span}(\mathbf{v}_{n_++1}, \dots, \mathbf{v}_n)$ , of dimension $n - n_+$ . For $\mathbf{v} \in U'$ , Eq. 14.4.1 gives $q(\mathbf{v}) = \sum_{i > n_+} d_ic_i^2 \le 0$ , every $d_i$ there being $\le 0$ . Let $W$ be any positive subspace. Then $W \cap U' = \{\mathbf{0}\}$ , since a non-zero vector in the intersection would have $q > 0$ and $q \le 0$ . By the dimension formula (Theorem 1.7.3), $\dim W + (n - n_+) = \dim(W + U') \le n ,$ so $\dim W \le n_+$ .
(c) By (a) and (b) the largest dimension of a positive subspace is exactly $n_+$ . That description mentions only $\beta$ and $V$ , no basis, so $n_+$ is an invariant of $\beta$ . Together with the invariance of $\operatorname{rank}\beta = n_+ + n_-$ under congruence (Theorem 14.2.3) this recovers Theorem 14.5.1.

Exercise (C2: A saddle the determinant hides)

Let $f(x,y,z) = x^2 - y^2 - z^2 + xyz$ .

(a) Verify that the origin is a critical point and compute $\mathbf{H}_f(\mathbf{0})$ and its determinant.
(b) Classify the origin, and explain why the sign of $\det \mathbf{H}_f(\mathbf{0})$ alone would have been useless.
(c) Prove that no twice continuously differentiable $g \colon \mathbb{R}^3 \to \mathbb{R}$ has a strict local maximum at a critical point $\mathbf{0}$ with $\det\mathbf{H}_g(\mathbf{0}) > 0$ , and give a $g \colon \mathbb{R}^2 \to \mathbb{R}$ that does. Which feature of the dimension is responsible?

Solution.

(a) $f_x = 2x + yz$ , $f_y = -2y + xz$ , $f_z = -2z + xy$ , all zero at $\mathbf{0}$ . The second derivatives at $\mathbf{0}$ are $f_{xx} = 2$ , $f_{yy} = f_{zz} = -2$ , and $f_{xy} = z$ , $f_{xz} = y$ , $f_{yz} = x$ , all zero at $\mathbf{0}$ . So $\mathbf{H}_f(\mathbf{0}) = \operatorname{diag}(2,-2,-2)$ , with $\det = 8 > 0$ .
(b) The inertia is $(1,2,0)$ , so the Hessian is indefinite and $\mathbf{0}$ is a saddle point by Theorem 14.6.4 (c). The determinant is the product of the eigenvalues (Corollary 12.1.5) and therefore records only the parity of $n_-$ ; here $\det > 0$ says $n_-$ is even, which is equally true of the negative definite $\operatorname{diag}(-2,-2,-2)$ and of the positive definite $\mathbf{I}_3$ . It cannot distinguish them.
(c) Suppose $\mathbf{0}$ is a local maximum of $g$ and write $q(\mathbf{h}) = \mathbf{h}^{\top}\mathbf{H}_g(\mathbf{0})\mathbf{h}$ . If $q(\mathbf{u}) > 0$ for some $\mathbf{u}$ , the computation in the proof of Theorem 14.6.4 (c) gives $g(t\mathbf{u}) > g(\mathbf{0})$ for all small $t \ne 0$ , contradicting the maximum. So $q(\mathbf{u}) \le 0$ for every $\mathbf{u}$ , that is $\mathbf{H}_g(\mathbf{0})$ is negative semidefinite and $n_+ = 0$ by Theorem 14.6.1 (d). If moreover $\det \mathbf{H}_g(\mathbf{0}) > 0$ then $0$ is not an eigenvalue, so $n_0 = 0$ and the inertia is $(0,3,0)$ . But then $\det\mathbf{H}_g(\mathbf{0}) = \lambda_1\lambda_2\lambda_3$ is a product of three negative numbers, hence negative — a contradiction. On $\mathbb{R}^2$ there is no obstruction: $g(x,y) = -x^2 - y^2$ has $\mathbf{H}_g(\mathbf{0}) = -2\mathbf{I}_2$ , determinant $4 > 0$ , and a strict local maximum at $\mathbf{0}$ . The responsible feature is the parity of the dimension: the determinant is the product of $n$ eigenvalues, so a negative definite Hessian has $\det$ of sign $(-1)^n$ .