Directional Derivatives and the Gradient, multivariable and vector calculus lesson

Standing on a hillside, you can walk north, east, or any direction in between, and the slope you feel depends on which way you face. Partial derivatives give the slope only due east and due north. The directional derivative gives it for every heading, and for a differentiable function a single vector, the gradient, holds all of those slopes at once.

Key terms

Unit direction

Definition A direction in $\mathbb{R}^n$ is given by a unit vector $\mathbf{u}$ , one with $\lVert \mathbf{u} \rVert = 1$ . Any nonzero vector $\mathbf{v}$ determines the unit direction

\mathbf{u} = \frac{\mathbf{v}}{\lVert \mathbf{v} \rVert}.

The zero vector gives no direction.

In plain words A direction should say which way to go, not how far. Dividing by the length removes the “how far” part.

Example The vector $\mathbf{v} = \langle 3, 4 \rangle$ has length $5$ , so its direction is $\mathbf{u} = \left\langle \frac{3}{5}, \frac{4}{5} \right\rangle$ . Using $\mathbf{v}$ itself in the formulas below would multiply every rate by $5$ .

Directional derivative

Definition Let $f$ be real valued on an open set $U \subseteq \mathbb{R}^n$ , let $\mathbf{a}$ be in $U$ , and let $\mathbf{u}$ be a unit vector. The directional derivative of $f$ at $\mathbf{a}$ in the direction $\mathbf{u}$ is the one-variable limit

D_{\mathbf{u}} f(\mathbf{a}) = \lim_{t \to 0} \frac{f(\mathbf{a} + t\mathbf{u}) - f(\mathbf{a})}{t},

when it exists. The partial derivatives are the special cases $\mathbf{u} = \mathbf{e}_i$ , the standard basis vectors. Replacing $\mathbf{u}$ by $-\mathbf{u}$ changes the sign of the limit.

In plain words Walk through $\mathbf{a}$ along a straight line at unit speed and measure how fast $f$ changes as you pass. On the graph this is the slope of the curve cut out by a vertical plane through $\mathbf{a}$ in the direction $\mathbf{u}$ .

Example For $f(x, y) = x^2 y$ at $(1, 2)$ with $\mathbf{u} = \left\langle \frac{3}{5}, \frac{4}{5} \right\rangle$ , expanding gives

\begin{aligned} &f\big(1 + \tfrac{3}{5}t, \ 2 + \tfrac{4}{5}t\big) \\[0.4em] &\quad = 2 + 3.2t + 1.68t^2 + 0.288t^3. \end{aligned}

Subtracting $f(1, 2) = 2$ , dividing by $t \ne 0$ and letting $t \to 0$ gives $D_{\mathbf{u}} f(1, 2) = 3.2$ .

Gradient formula for differentiable functions

Definition If all first partial derivatives of $f$ exist at $\mathbf{a}$ , the gradient of $f$ at $\mathbf{a}$ is the vector

\nabla f(\mathbf{a}) = \left\langle \frac{\partial f}{\partial x_1}(\mathbf{a}), \dots, \frac{\partial f}{\partial x_n}(\mathbf{a}) \right\rangle.

If $f$ is differentiable at $\mathbf{a}$ , then the directional derivative exists for every unit vector $\mathbf{u}$ and

D_{\mathbf{u}} f(\mathbf{a}) = \nabla f(\mathbf{a}) \cdot \mathbf{u}.

This follows from the chain rule applied to $t \mapsto f(\mathbf{a} + t\mathbf{u})$ . Without differentiability the formula can fail even when every directional derivative exists.

In plain words For a differentiable function you never need the limit again. Compute the partial derivatives once, and every directional rate is a dot product.

Example The polynomial $f(x, y, z) = xy + z^2$ is differentiable everywhere. At $(1, 2, -1)$ its gradient is $\langle y, x, 2z \rangle = \langle 2, 1, -2 \rangle$ . Toward $\mathbf{v} = \langle 2, -1, 2 \rangle$ , of length $3$ , the unit direction is $\frac{1}{3}\langle 2, -1, 2 \rangle$ , and

D_{\mathbf{u}} f = \frac{4 - 1 - 4}{3} = -\frac{1}{3}.

In the first example the same formula gives $\langle 4, 1 \rangle \cdot \left\langle \frac{3}{5}, \frac{4}{5} \right\rangle = 3.2$ , matching the limit.

Steepest ascent

Definition Let $f$ be differentiable at $\mathbf{a}$ with $\nabla f(\mathbf{a}) \ne \mathbf{0}$ . If $\theta$ is the angle between $\mathbf{u}$ and $\nabla f(\mathbf{a})$ , then

D_{\mathbf{u}} f(\mathbf{a}) = \lVert \nabla f(\mathbf{a}) \rVert \cos\theta.

So the largest directional derivative is $\lVert \nabla f(\mathbf{a}) \rVert$ , reached only for $\mathbf{u} = \nabla f(\mathbf{a}) / \lVert \nabla f(\mathbf{a}) \rVert$ , the smallest is $-\lVert \nabla f(\mathbf{a}) \rVert$ , reached only in the opposite direction, and the rate is $0$ exactly for directions perpendicular to the gradient.

In plain words The gradient points in the input space toward the steepest climb, and its length is how steep that climb is. Walking against it is the steepest descent.

Example On the hill $h(x, y) = 100 - x^2 - 2y^2$ at $(3, 1)$ the gradient is $\langle -2x, -4y \rangle = \langle -6, -4 \rangle$ , of length $\sqrt{52} = 2\sqrt{13} \approx 7.21$ . The steepest way up is the direction

\mathbf{u} = \frac{\langle -3, -2 \rangle}{\sqrt{13}},

with slope about $7.21$ . Moving along $\pm\langle 2, -3 \rangle / \sqrt{13}$ keeps the height unchanged to first order.

Gradient as a level-set normal

Definition Let $f$ be real valued on an open set $U \subseteq \mathbb{R}^n$ , let $c$ be a constant, and let $\mathbf{a}$ lie on the level set $f(\mathbf{x}) = c$ . Suppose $f$ is differentiable at $\mathbf{a}$ . If $\mathbf{r}(t)$ , defined on an open interval around $0$ , is a curve in the level set, differentiable at $t = 0$ with $\mathbf{r}(0) = \mathbf{a}$ , then differentiating $f(\mathbf{r}(t)) = c$ gives

\nabla f(\mathbf{a}) \cdot \mathbf{r}'(0) = 0.

If moreover $f$ is $C^1$ on $U$ and $\nabla f(\mathbf{a}) \ne \mathbf{0}$ , then by the implicit function theorem the level set near $\mathbf{a}$ is a smooth curve ( $n = 2$ ) or surface ( $n = 3$ ), namely the graph of a $C^1$ function of $n - 1$ of the variables. Then $\nabla f(\mathbf{a})$ is a normal vector to it, and its tangent line or tangent plane at $\mathbf{a}$ is $\nabla f(\mathbf{a}) \cdot (\mathbf{x} - \mathbf{a}) = 0$ .

In plain words Moving along a level set keeps $f$ constant, so the rate of change is zero, and that forces the gradient to be perpendicular to the level set. Wherever the gradient is not zero, the steepest way uphill is at right angles to the contour lines.

Example The ellipsoid $x^2 + 2y^2 + 3z^2 = 6$ is a level set of a polynomial and contains $(1, 1, 1)$ . The gradient there is $\langle 2x, 4y, 6z \rangle = \langle 2, 4, 6 \rangle$ , which is not zero. The tangent plane is $\langle 2, 4, 6 \rangle \cdot \langle x - 1, y - 1, z - 1 \rangle = 0$ . Dividing by $2$ gives

(x - 1) + 2(y - 1) + 3(z - 1) = 0,

which simplifies to $x + 2y + 3z = 6$ .

Gradient zero

Definition A point $\mathbf{a}$ where $f$ is differentiable and $\nabla f(\mathbf{a}) = \mathbf{0}$ is a critical point. There every directional derivative is $0$ , there is no direction of steepest ascent, and the level set through $\mathbf{a}$ need not be a smooth curve or surface, so the gradient gives no normal vector there.

In plain words A zero gradient means the surface is flat to first order in every direction. It could be a peak, a pit, or a saddle, and the contour through it can look strange.

Example The function $f(x, y) = x^2 - y^2$ has $\nabla f = \langle 2x, -2y \rangle$ , which is $\mathbf{0}$ at the origin. The level set $f = 0$ is the pair of lines $y = x$ and $y = -x$ , which cross at the origin, so no single normal line exists there. For $g(x, y) = x^2 + y^2$ the level set $g = 0$ is only the point $(0, 0)$ .

Directional derivatives alone are not enough

Definition A function can have a directional derivative at $\mathbf{a}$ in every unit direction without being differentiable at $\mathbf{a}$ , and even without being continuous there. In that case the formula $D_{\mathbf{u}} f(\mathbf{a}) = \nabla f(\mathbf{a}) \cdot \mathbf{u}$ may fail. Differentiability must be checked separately, for example through partial derivatives that are continuous on an open set containing $\mathbf{a}$ .

In plain words Directional derivatives test only straight lines through the point. A function can behave nicely on every line and still jump along a curve.

Example Let $f(x, y) = \frac{x^2 y}{x^4 + y^2}$ for $(x, y) \ne (0, 0)$ and $f(0, 0) = 0$ . For a unit vector $\mathbf{u} = \langle a, b \rangle$ and $t \ne 0$ ,

\frac{f(ta, tb)}{t} = \frac{a^2 b}{t^2 a^4 + b^2},

which tends to $\frac{a^2}{b}$ if $b \ne 0$ and is $0$ if $b = 0$ . So every directional derivative exists. For $\mathbf{u} = \left\langle \frac{3}{5}, \frac{4}{5} \right\rangle$ it is $\frac{9}{20}$ , while $\nabla f(0, 0) = \langle 0, 0 \rangle$ would predict $0$ . Along the parabola $y = x^2$ with $x \ne 0$ the function equals $\frac{1}{2}$ , so it is not continuous at the origin and not differentiable there.

Check yourself

Find the rate of change of $f(x, y) = x e^y$ at $(2, 0)$ in the direction of $\mathbf{v} = \langle 1, -1 \rangle$ .

Show the answer

The function has continuous partial derivatives, so it is differentiable. The gradient is $\langle e^y, x e^y \rangle = \langle 1, 2 \rangle$ at $(2, 0)$ , and the unit direction is $\langle 1, -1 \rangle / \sqrt{2}$ . The rate is

D_{\mathbf{u}} f(2, 0) = \frac{1 - 2}{\sqrt{2}} = -\frac{\sqrt{2}}{2}.

In which unit direction does $f(x, y) = x^2 + xy$ decrease fastest at $(1, 2)$ , and at what rate?

Show the answer

The gradient is $\langle 2x + y, x \rangle = \langle 4, 1 \rangle$ , of length $\sqrt{17}$ . The fastest decrease is in the direction $-\langle 4, 1 \rangle / \sqrt{17}$ , at the rate $-\sqrt{17} \approx -4.12$ .

Find the tangent plane to the surface $xyz = 6$ at $(1, 2, 3)$ .

Show the answer

With $F(x, y, z) = xyz$ the gradient is $\langle yz, xz, xy \rangle = \langle 6, 3, 2 \rangle$ at $(1, 2, 3)$ , which is not zero. The tangent plane is $6(x - 1) + 3(y - 2) + 2(z - 3) = 0$ , or $6x + 3y + 2z = 18$ .

Directional Derivatives and the Gradient

Key terms

Unit direction

Directional derivative

Gradient formula for differentiable functions

Steepest ascent

Gradient as a level-set normal

Gradient zero

Directional derivatives alone are not enough

Check yourself

How this lesson connects