Local Extrema and Saddle Points, multivariable and vector calculus lesson

Stand at the bottom of a bowl, on top of a hill, or in the middle of a mountain pass. In all three places the ground is level in every direction, so the tangent plane is horizontal. Yet only two of them are high or low points. A pass rises in one direction and falls in another. Telling these three situations apart is the job of the second derivative test, and spotting when it cannot decide is just as important.

Key terms

Local maxima and minima

Definition Let $f$ be defined on a set $D \subseteq \mathbb{R}^n$ and let $a$ be a point of $D$ . The function has a local maximum at $a$ if there is an $r > 0$ such that

\begin{gathered} f(x) \le f(a) \\ \text{for all } x \in D \text{ with } \lVert x - a \rVert < r. \end{gathered}

A local minimum is defined the same way with $f(x) \ge f(a)$ . The extremum is strict if the inequality is strict for every such $x \ne a$ . Local maxima and minima together are called local extrema.

In plain words A local maximum is a point that is at least as high as everything close to it. It is strict when it is the only point at that height nearby.

Example The function $f(x, y) = x^2 + y^2$ has a strict local minimum at $(0, 0)$ , since $f > 0$ everywhere else. The function $g(x, y) = x^2$ has a local minimum at every point $(0, y_0)$ , but none of them is strict, because $g = 0$ along the whole $y$ -axis.

Critical points

Definition Let $a$ be an interior point of the domain of $f$ . The point $a$ is a critical point of $f$ if $\nabla f(a) = 0$ or if at least one first partial derivative of $f$ does not exist at $a$ . If $f$ has a local extremum at an interior point $a$ and $f_{x_i}(a)$ exists, then $f_{x_i}(a) = 0$ . Hence every local extremum at an interior point occurs at a critical point. The converse is false.

In plain words At an interior high or low point, either the ground is level or the function has a corner or spike. So you look for both kinds of candidates, smooth ones and singular ones.

Example For $f(x, y) = x^3 - 3x + y^2$ , the partials $f_x = 3x^2 - 3$ and $f_y = 2y$ exist everywhere and vanish together exactly at $(1, 0)$ and $(-1, 0)$ . Those are the only critical points. For $k(x, y) = \sqrt{x^2 + y^2}$ , the partials exist away from the origin and never vanish together there, but $k_x(0, 0)$ does not exist, since $k(h, 0) = \lvert h \rvert$ . So the origin is a singular critical point, and it is a strict local minimum.

Saddle points

Definition A critical point $a$ of $f$ is a saddle point if every open ball centered at $a$ contains points $x$ in the domain with $f(x) > f(a)$ and points with $f(x) < f(a)$ . Equivalently, a saddle point is a critical point that is neither a local maximum nor a local minimum.

In plain words Near a saddle, the function goes up in some directions and down in others, no matter how closely you look. A horizontal tangent plane does not make a point a peak or a pit.

Example For $f(x, y) = x^2 - y^2$ the gradient $\langle 2x, -2y \rangle$ vanishes only at the origin. Along the $x$ -axis $f(t, 0) = t^2 > 0$ , and along the $y$ -axis $f(0, t) = -t^2 < 0$ for $t \ne 0$ . So both signs appear in every disk around the origin, and it is a saddle point.

Hessian definiteness

Definition A symmetric $n \times n$ matrix $A$ is positive definite if $h^{T} A h > 0$ for every $h \ne 0$ , negative definite if $h^{T} A h < 0$ for every $h \ne 0$ , and indefinite if $h^{T} A h$ takes both positive and negative values. If $h^{T} A h \ge 0$ for all $h$ it is positive semidefinite, and if $h^{T} A h \le 0$ for all $h$ it is negative semidefinite. Equivalently, these are the cases where all eigenvalues are positive, all are negative, some are positive and some negative, all are nonnegative, and all are nonpositive. For a $2 \times 2$ symmetric matrix

A = \begin{bmatrix} a & b \\ b & c \end{bmatrix}, \qquad \det A = ac - b^2,

the three main cases read

\begin{aligned} &\text{positive definite} \\ &\quad \iff a > 0 \text{ and } \det A > 0, \\[0.4em] &\text{negative definite} \\ &\quad \iff a < 0 \text{ and } \det A > 0, \\[0.4em] &\text{indefinite} \iff \det A < 0. \end{aligned}

In plain words Definiteness tells you the sign of the quadratic part of the Taylor model. Positive definite means the quadratic part curves up in every direction, like a bowl.

Example Compare

A = \begin{bmatrix} 2 & 1 \\ 1 & 3 \end{bmatrix}, \qquad B = \begin{bmatrix} 1 & 2 \\ 2 & 1 \end{bmatrix}.

The matrix $A$ has $a = 2 > 0$ and determinant $6 - 1 = 5 > 0$ , so it is positive definite. The matrix $B$ has determinant $-3$ , so it is indefinite. Indeed at $h = \langle 1, -1 \rangle$ its form gives $1 - 4 + 1 = -2$ , while at $h = \langle 1, 1 \rangle$ it gives $6$ .

The second derivative test

Definition Let $f$ be $C^2$ on an open set containing $a$ , and let $\nabla f(a) = 0$ . If $Hf(a)$ is positive definite, $f$ has a strict local minimum at $a$ . If $Hf(a)$ is negative definite, $f$ has a strict local maximum at $a$ . If $Hf(a)$ is indefinite, $a$ is a saddle point. For two variables put

D = f_{xx}(a) f_{yy}(a) - f_{xy}(a)^2.

Then $D > 0$ with $f_{xx}(a) > 0$ gives a strict local minimum, $D > 0$ with $f_{xx}(a) < 0$ gives a strict local maximum, and $D < 0$ gives a saddle point. If $D = 0$ the test gives no information.

In plain words At a critical point the gradient term of the Taylor model is zero, so along each line $a + th$ on which $h^{T} Hf(a) h \ne 0$ , the difference $f(a + th) - f(a)$ has the sign of that number for all small $t \ne 0$ . Definiteness makes this sign the same in every direction at once. The determinant $D$ is a quick way to read off which case you are in.

Example Classify the critical points of $f(x, y) = x^3 + y^3 - 3xy$ . The equations $3x^2 - 3y = 0$ and $3y^2 - 3x = 0$ give $y = x^2$ and $x^4 = x$ , so the critical points are $(0, 0)$ and $(1, 1)$ . The Hessian is

Hf = \begin{bmatrix} 6x & -3 \\ -3 & 6y \end{bmatrix}.

Point	$f_{xx}$	$D$	Type
$(0, 0)$	$0$	$-9$	saddle point
$(1, 1)$	$6$	$27$	strict local minimum, $f = -1$

Degenerate and inconclusive cases

Definition Let $f$ be $C^2$ on an open set containing $a$ . A critical point $a$ with $\nabla f(a) = 0$ is degenerate when $\det Hf(a) = 0$ , which for two variables means $D = 0$ . For two variables the Hessian is then semidefinite and the test above gives no conclusion. In three or more variables a degenerate Hessian can still be indefinite, for example with eigenvalues $1$ , $-1$ and $0$ , and then the point is a saddle by the test above, because $f$ has a strict minimum along one line and a strict maximum along another. A degenerate Hessian that is semidefinite gives no conclusion. In the inconclusive case the point may be a strict or non-strict extremum or a saddle, and it must be classified by other means, such as the sign of $f(x) - f(a)$ near $a$ , restrictions of $f$ to lines or curves through $a$ , or factoring. The test also does not apply to singular critical points, where $f$ is not differentiable.

In plain words When the quadratic part is flat in some direction, it is too weak to decide, and higher order terms take over. Look at the function itself.

Example At the origin, $x^4 + y^4$ , $-(x^4 + y^4)$ and $x^4 - y^4$ all have zero Hessian, so $D = 0$ . Directly, the first has a strict minimum, the second a strict maximum, and the third a saddle, since it is positive on the $x$ -axis and negative on the $y$ -axis. A less obvious case is $f(x, y) = x^2 + y^3$ , whose Hessian at the origin is

\begin{bmatrix} 2 & 0 \\ 0 & 0 \end{bmatrix},

so $D = 0$ . Along the $y$ -axis $f(0, t) = t^3$ takes both signs, so the origin is a saddle point.

Check yourself

Find and classify the critical points of $f(x, y) = x^2 + xy + y^2 - 3x$ .

Show the answer

The equations $2x + y - 3 = 0$ and $x + 2y = 0$ give the single point $(2, -1)$ . There $f_{xx} = 2$ , $f_{yy} = 2$ , $f_{xy} = 1$ , so $D = 3 > 0$ with $f_{xx} > 0$ . It is a strict local minimum with value $f(2, -1) = -3$ .

The second derivative test fails for $f(x, y) = x^4 + y^2$ at the origin. Why, and what kind of point is it?

Show the answer

The Hessian at the origin is

\begin{bmatrix} 0 & 0 \\ 0 & 2 \end{bmatrix},

so $D = 0$ . Directly, $f(x, y) > 0 = f(0, 0)$ for every $(x, y) \ne (0, 0)$ , so the origin is a strict local minimum.

Find all critical points of $f(x, y) = x^2 - 2xy + y^2$ and say whether the extrema are strict.

Show the answer

The gradient $\langle 2(x - y), -2(x - y) \rangle$ vanishes exactly on the line $y = x$ , and there $D = 4 - 4 = 0$ . Since $f = (x - y)^2 \ge 0$ with equality on that line, every point of the line is a local minimum. None is strict, because nearby points on the line have the same value $0$ .

Local Extrema and Saddle Points

Key terms

Local maxima and minima

Critical points

Saddle points

Hessian definiteness

The second derivative test

Degenerate and inconclusive cases

Check yourself

How this lesson connects