Extrema and optimization

The last chapter ended with the box problem. Equal little squares of side $x$ are cut from the corners of a $30 \times 30$ cm sheet of cardboard and the edges are folded up. The cut can be anything from zero to fifteen centimetres, and each gives its own box. Which of the infinitely many boxes holds the most?

Picture an inventors’ contest. The jury sets five tasks, and in each one you have to find the best option among infinitely many: the roomiest box, the fastest route, the cheapest can. You can’t try everything, and guessing is risky. The winner will have one tool, the derivative, and one idea going back to Pierre de Fermat: at the top of a hill the tangent is horizontal. Each task will add a new rule to this idea, and by the end of the contest we’ll have assembled a complete rulebook for finding the best.

Task one: the box

The volume of the box is the area of the base times the height. The base is a square of side $30 - 2x$, the height is $x$, so

$$V(x) = x\,(30 - 2x)^2, \qquad 0 < x < 15.$$

Let’s compute for whole-number cuts: $V(1) = 784$, $V(2) = 1352$, $V(3) = 1728$, $V(4) = 1936$, $V(5) = 2000$, $V(6) = 1944$, $V(7) = 1792$ cubic centimetres. It looks as if the best cut is about five centimetres. But perhaps at $x = 4.9$ or $5.03$ the volume is even larger? The table won’t tell us, but the derivative will.

Near the top the graph becomes flat, and a small error in the cut hardly changes the volume: at $x = 4.5$ we get $1984.5$ cm³, at $x = 5.5$, $1985.5$ cm³, only one per cent less than the best, although the cut was off by ten per cent. This observation is much older than calculus. In 1615 Johannes Kepler published the New Stereometry of Wine Barrels: he was working out which barrel, for a given length of the measuring rod merchants inserted through the bung-hole, holds the most wine, and he noticed that near the largest volume small changes in the dimensions barely change the capacity. Around 1636 Pierre de Fermat turned this idea into a method. Today it’s written like this.

The trick is to look at the signs of the secants’ slopes to the left and right of the top, and squeeze the derivative between them.

Let there be a maximum at $c$ (for a minimum the argument is the same, with opposite signs). So near $c$ all the values are no larger than $f(c)$: the graph doesn’t rise above the dashed line $y = f(c)$. Take a point $x$ to the left of $c$, $x = c + h$ with $h < 0$. The slope of the secant is $\frac{f(c + h) - f(c)}{h}$: in the numerator $f(c + h) - f(c) \le 0$, in the denominator $h < 0$. Negative or zero divided by negative is non-negative, so the slope is $\ge 0$. A limit of non-negative numbers can’t be negative, so the left limit, that is, $f'(c)$, is at least zero. On the right, with $h > 0$, the numerator is still $\le 0$ and the denominator positive: the slope is $\le 0$, and the limit from the right gives $f'(c) \le 0$. The function is differentiable at $c$, so the limits from the left and right exist and equal the same number $f'(c)$. This number is both $\ge 0$ and $\le 0$, that is, $f'(c) = 0$: the tangent at the top is horizontal.

Let’s find the critical points of the volume. Expand the brackets: $V(x) = 4x^3 - 120x^2 + 900x$, so $V'(x) = 12x^2 - 240x + 900 = 12(x - 5)(x - 15)$. On the interval $(0, 15)$ the derivative vanishes only at $x = 5$. So if the volume has a largest value, it’s reached at $x = 5$, and it’s $2000$ cm³.

A small “if” remains. Fermat’s theorem says where a maximum can be, but doesn’t promise there is one. The function $\frac1x$ on the interval $(0, 1)$ has no largest value: it grows without bound. The function $x$ on the same interval has none either, although it’s bounded: you can get arbitrarily close to one but never reach it. Existence is guaranteed by a theorem that needs a closed interval with its ends and continuity.

For the box, add the ends: $V(0) = V(15) = 0$ (a flat sheet, and a sheet with nothing left of it). The function $V$ is continuous on $[0, 15]$, so by the extreme value theorem there is a largest value. At the ends the volume is zero, so the largest value is reached inside, and there, by Fermat’s theorem, the derivative is zero. The only candidate is $x = 5$. Answer: cut out squares of side $5$ cm, giving a $20 \times 20 \times 5$ box of volume $2000$ cm³. For a sheet of side $a$ the same calculation gives $x = \frac a6$.

The derivative of a function at a point $c$ is zero. What can be said for certain?

There’s a maximum or a minimum at $c$ The tangent at $c$ is horizontal, but there may be no extremum All the extrema of a function are at points where the derivative is zero The function changes direction at $c$

Fermat’s theorem is a necessary condition: at an extremum the derivative is zero (if it exists), but an extremum doesn’t follow from $f'(c) = 0$. The cubic $y = x^3$ briefly becomes horizontal at zero and keeps on rising. And extrema occur where there’s no derivative too, as with $|x|$ at zero. So the list of candidates includes all the critical points and the ends of the interval.

Task two: the speed camera

On a country highway there are two cameras ten kilometres apart. Each records a car’s number plate and the time. A car covered the stretch in six minutes, that is, at an average speed of $10 : \frac{6}{60} = 100$ km/h, where the limit is $90$. A fine arrives. The driver is indignant: “I was watching the speedometer, it never showed more than ninety!” The jury asks the inventors to settle the dispute. Intuition is on the camera’s side, but proving it is no five-minute job.

Let’s start with a special case that’s simpler: the car set off and came back to the same point, or, on the graph, the function’s values at the ends of the interval are equal. Then somewhere it must have stopped or turned round.

The idea: the largest or smallest value must turn up inside the interval, and there Fermat’s theorem works.

The function is continuous on $[a, b]$, differentiable inside, and its values at the ends are equal. By the extreme value theorem, the interval $[a, b]$ has a point of largest value and a point of smallest value. If both lie at the ends, the largest and smallest values both equal $f(a) = f(b)$. Then all the values are squeezed between two equal numbers, the function is constant, and its derivative is zero at every point of the interval. Otherwise at least one of these points, call it $c$, lies inside, in $(a, b)$. The largest (or smallest) value on the interval is certainly a local extremum, and the function is differentiable at $c$. By Fermat’s theorem $f'(c) = 0$.

Now the general case. The average speed over the interval $[a, b]$ is the slope of the chord joining the ends of the distance graph. The claim about the camera reads: somewhere on the interval the tangent is parallel to the chord.

The trick: subtract the chord from the function. The picture “tilts” so that the chord becomes horizontal, and we’re in the setting of Rolle’s theorem.

Draw the chord through the ends of the graph, $A(a, f(a))$ and $B(b, f(b))$. Its slope is $k = \frac{f(b) - f(a)}{b - a}$, and its equation $y = f(a) + k(x - a)$. Consider the difference between the graph and the chord: $g(x) = f(x) - f(a) - k(x - a)$. In the picture these are the lengths of the vertical segments between them. The function $g$ is continuous on $[a, b]$ and differentiable inside, as a difference of such functions, and $g'(x) = f'(x) - k$. At the ends $g(a) = 0$ and $g(b) = f(b) - f(a) - k(b - a) = 0$: the chord passes through the ends of the graph. Subtracting $k(x - a)$ means shifting each point of the graph vertically, “tilting” the picture so that the chord lies horizontal. By Rolle’s theorem there’s a point $c \in (a, b)$ where $g'(c) = 0$. Restore the tilt: $g'(c) = f'(c) - k = 0$, that is, $f'(c) = k = \frac{f(b) - f(a)}{b - a}$. The tangent at $c$ is parallel to the chord, and $f(b) - f(a) = f'(c)(b - a)$. The increment of the function over the interval: how far the car went between the cameras. The instantaneous speed at some moment $c$ between $a$ and $b$. Where exactly that moment is, the theorem doesn’t say. The length of the interval: how long the trip took. Example: for $f(x) = x^2$ on $[1, 3]$ the left side is $9 - 1 = 8$, the right $2c \cdot 2 = 4c$, and $c = 2$. For the cameras: $10$ km in $0.1$ hours, so at some moment the speedometer showed exactly $100$ km/h. The driver is wrong.

Lagrange published this theorem in 1797, in the same Theory of Analytic Functions where the prime appeared. Its main strength isn’t in cameras but in its consequences: it turns information about the derivative at each point into information about the function itself over a whole interval. The first consequence is something we only saw on graphs in the last chapter.

The last statement looks obvious: if the speed is zero all the time, the car is standing still. But it can’t be proved without Lagrange’s theorem, and it’s needed in analysis all the time. For example, it says that two functions with equal derivatives differ by a constant: the derivative of their difference is zero. This idea will become the main one in the chapter on the integral.

The jury’s report: the shape of the graph

One answer isn’t enough for the jury; it wants to know why it’s the best. Let’s collect the rules by which derivatives describe the shape of a graph: where it rises, where it turns and which way it bends.

Let’s check on the cubic $f(x) = x^3 - 3x$. The derivative $f'(x) = 3x^2 - 3 = 3(x - 1)(x + 1)$ is positive for $x < -1$, negative between $-1$ and $1$, and positive again for $x > 1$. So there’s a maximum at $-1$, $f(-1) = 2$, and a minimum at $1$, $f(1) = -2$. The largest value on a closed interval is found like this: find the critical points inside, compute the values there and at the ends, and choose the largest. The rulebook is short, but it works for any continuous function with finitely many critical points: by the extreme value theorem the largest value exists, and by Fermat’s theorem it’s either at an end or at a critical point.

Find the largest value of $f(x) = x^3 - 3x$ on the interval $[0, 2]$.

The critical points: $f'(x) = 3(x - 1)(x + 1) = 0$ at $x = \pm 1$, and only $x = 1$ is in the interval. The values: $f(0) = 0$, $f(1) = -2$, $f(2) = 8 - 6 = 2$. The largest is $2$, at the right end. The critical point $x = 1$ turned out to be a minimum, so looking for the answer only among the zeros of the derivative would have been a mistake.

The derivative of the derivative is called the second derivative and written $f''(x)$. If the first derivative is velocity, the second is acceleration: how fast the slope is changing. For $x^3 - 3x$ the second derivative is $6x$.

For $x^3 - 3x$ the second derivative $6x$ is negative at $-1$ (a maximum, a cap), positive at $1$ (a minimum, a cup), and changes sign at zero: that’s a point of inflection, where the curve turns from a cap into a cup. The trainer has three levels: critical points, points of maximum and minimum, and the largest and smallest values on an interval. If you need the full picture of a graph with asymptotes and convexity, the solver will help.

Task three: the lifeguard

A lifeguard stands on the sand $18$ metres from the water’s edge. In the sea, $16$ metres out and $46$ metres further along the beach, someone is drowning. Run in a straight line? Then there’s too much swimming, and a person swims several times slower than they run. Run along the sand to the point directly opposite the swimmer? Then there’s too much running. The best route is somewhere in between, and it bends at the water’s edge.

In 2003 the American mathematician Timothy Pennings published a paper in the College Mathematics Journal with a question for a title: “Do Dogs Know Calculus?” His Welsh corgi Elvis, on the shore of Lake Michigan, didn’t chase a ball thrown into the water in a straight line: he ran along the sand and leapt into the water at some point. Pennings measured the speeds (about $6.4$ m/s on sand and about $0.9$ m/s swimming), calculated the optimal point of entry, and compared it with where Elvis actually jumped. The agreement was astonishingly good. The dog, of course, didn’t compute any derivatives, but he did solve the problem.

Let’s do the calculation. Let the lifeguard $L$ be at distance $a$ from the shore, the swimmer $S$ at distance $b$, with $d$ between them along the shore. If you enter the water at the point $P$, a distance $x$ from the point of the shore nearest the lifeguard, then by Pythagoras’ theorem the run takes $\frac{\sqrt{a^2 + x^2}}{v_1}$ and the swim $\frac{\sqrt{b^2 + (d - x)^2}}{v_2}$, where $v_1$ and $v_2$ are the speeds on land and in water. We need the smallest value of the sum

$$T(x) = \frac{\sqrt{a^2 + x^2}}{v_1} + \frac{\sqrt{b^2 + (d - x)^2}}{v_2}.$$

The idea: the derivative of each leg of the route with respect to the position of $P$ turns out to be the sine of an angle, a ratio of sides in a right triangle.

The time of the route is $T(x) = \frac{LP}{v_1} + \frac{PS}{v_2}$. Drop the perpendiculars $LF$ and $SG$ to the shore: $LF = a$, $FP = x$, $SG = b$, $PG = d - x$. In the right triangle $LFP$ the angle at the vertex $L$ equals the angle $\alpha$ between the path and the perpendicular to the shore at $P$: they’re alternate angles for the parallel lines $LF$ and the perpendicular at $P$. So $\sin\alpha = \frac{FP}{LP} = \frac{x}{\sqrt{a^2 + x^2}}$. By the chain rule $\left(\sqrt{a^2 + x^2}\right)' = \frac{2x}{2\sqrt{a^2 + x^2}} = \sin\alpha$. In the same way in triangle $SGP$ we have $\sin\beta = \frac{d - x}{\sqrt{b^2 + (d - x)^2}}$, and the derivative of the second leg is $\left(\sqrt{b^2 + (d - x)^2}\right)' = \frac{-2(d - x)}{2\sqrt{b^2 + (d - x)^2}} = -\sin\beta$. The minus sign makes sense: moving $P$ to the right shortens the swim. So $T'(x) = \frac{\sin\alpha}{v_1} - \frac{\sin\beta}{v_2}$. As $P$ moves to the right, the leg $FP$ grows and $\sin\alpha$ grows; the leg $PG$ shrinks and $\sin\beta$ shrinks. So $T'$ is increasing on the whole interval. At the left end $x = 0$ we have $\sin\alpha = 0$ and $T'(0) = -\frac{\sin\beta}{v_2} < 0$; at the right end $x = d$ we have $\sin\beta = 0$ and $T'(d) = \frac{\sin\alpha}{v_1} > 0$. An increasing continuous derivative passes from minus to plus exactly once, at some point $x_0$. To the left of $x_0$ the time decreases, to the right it increases: by the sufficient condition this is the only minimum, and there $T'(x_0) = 0$, that is, $\frac{\sin\alpha}{v_1} = \frac{\sin\beta}{v_2}$. The angle between the path on land and the perpendicular to the shore (for light, the angle of incidence). The speed in the first medium: running on sand, or the speed of light in air. The angle between the path in water and the perpendicular (for light, the angle of refraction). The speed in the second medium. Where it’s slower, the angle is smaller: the path turns more steeply towards the perpendicular. Example: light travels at about $300{,}000$ km/s in air and about $225{,}000$ km/s in water. A ray striking water at $\alpha = 45°$ travels in the water at an angle $\beta$ with $\sin\beta = \frac{225}{300}\sin 45° \approx 0.53$, that is, $\beta \approx 32°$. That’s why a spoon in a glass looks broken.

Task four: the can

A canning factory wants a $330$ ml can, $V = 330$ cm³, using as little tin as possible. A can is a cylinder with base radius $r$ and height $h$. The tin goes into two discs (the base and the lid) and the side wall, whose unrolled shape is a $2\pi r \times h$ rectangle:

$$S = 2\pi r^2 + 2\pi r h.$$

There are two variables here, but they’re linked: the volume $\pi r^2 h = V$ is fixed, so $h = \frac{V}{\pi r^2}$. Substitute and we get a function of one variable, $S(r) = 2\pi r^2 + \frac{2V}{r}$ for $r > 0$. Its derivative $S'(r) = 4\pi r - \frac{2V}{r^2}$ vanishes when $r^3 = \frac{V}{2\pi}$. To the left of this point $S' < 0$, to the right $S' > 0$: it’s a minimum, and the smallest value on the whole ray $r > 0$, because to the left the function decreases and to the right it increases. The most interesting thing is the height at the optimum:

$$h = \frac{V}{\pi r^2} = \frac{2\pi r^3}{\pi r^2} = 2r.$$

The cheapest can is the one whose height equals its diameter: in profile it looks like a square. For $330$ ml we get $r \approx 3.74$ cm and $h \approx 7.5$ cm, and such a can takes about $264$ cm² of tin.

Look in the fridge: an ordinary $330$ ml can is about $6.6$ cm across and $11.5$ cm tall, noticeably taller than the “square” one. The engineers haven’t made a mistake; their problem is just different. The base and lid of an aluminium can are two to three times thicker than the thin wall: the lid has to hold the pressure of a fizzy drink and carry the ring pull. If a square centimetre of the ends costs $k$ times as much as the wall, the same calculation gives $h = 2kr$: the dearer the lids, the more a tall, narrow can pays off. And a can has to be comfortable to hold in the hand. The derivative honestly solves the problem it’s given, so setting the problem right is half the invention.

An open-topped tank with a square base and a volume of $32$ dm³ is to be made from as little metal as possible. What should the side of the base be (in decimetres)?

Let the side of the base be $x$ and the height $y$; then $x^2 y = 32$ and $y = \frac{32}{x^2}$. The metal: the base $x^2$ and four walls $4xy$, in all $S(x) = x^2 + \frac{128}{x}$. The derivative $S'(x) = 2x - \frac{128}{x^2}$ is zero at $x^3 = 64$, that is, $x = 4$; to the left it’s negative, to the right positive, so it’s a minimum. The tank is $4 \times 4 \times 2$ dm: without a lid it comes out half as tall as it is wide.

Task five: a root without the key

The last task is an invention for a calculator. It has four operations and no square-root key. How do you find $\sqrt2$, that is, the root of the equation $x^2 - 2 = 0$? The idea comes from the magnifying glass of the last chapter: near any point the graph almost coincides with its tangent. Finding where a curve crosses the axis is hard; finding where a line crosses it is easy. Replace the curve with its tangent, find where the tangent crosses the axis, and repeat from the new point.

The tangent to the graph $y = f(x)$ at the point $x_n$ is the line $y = f(x_n) + f'(x_n)(x - x_n)$. It crosses the $x$-axis where $y = 0$, that is, at $x = x_n - \frac{f(x_n)}{f'(x_n)}$. That’s the next approximation.

The current approximation to the root. The residual: how far the function’s value is from zero. The smaller it is, the smaller the step. The slope of the tangent. If it’s close to zero, the tangent is almost horizontal and leads far away; if it’s zero, the step is impossible. Example: for $f(x) = x^2 - 2$ we get $x_{n+1} = x_n - \frac{x_n^2 - 2}{2x_n} = \frac12\left(x_n + \frac{2}{x_n}\right)$. From $x_0 = 1$: $x_1 = 1.5$, $x_2 = 1.41666\ldots$, $x_3 = 1.4142157\ldots$, $x_4 = 1.41421356237469\ldots$: twelve correct digits in four steps, with nothing but additions and divisions.

This is the very Babylonian method, or Heron’s method, from the chapter on the square root of two: there we got it from rectangles and proved that the error is roughly squared at each step. Now it’s clear where it comes from, and how to build the same kind of recipe for any equation. For $\sqrt[3]{2}$ take $f(x) = x^3 - 2$ and get $x_{n+1} = \frac{2x_n^3 + 2}{3x_n^2}$. For the equation $\cos x = x$, which no formula solves, take $f(x) = \cos x - x$, and in four steps from $x_0 = 1$ you find the root $0.7390851\ldots$ Quadratic convergence is a general property of the method.

If the error is already less than $\frac{m}{M} \cdot 10^{-k}$, the next one will be less than $\frac{m}{M} \cdot 10^{-2k}$: the number of correct digits roughly doubles. Near a root Newton’s method is one of the fastest known ways of solving equations, and calculators and computers use it or its relatives.

The theorem’s conditions aren’t a formality. Start the method for $x^3 - 2x + 2 = 0$ at zero: the tangent leads to one, and from one back to zero, and so on for ever. For $\arctan x = 0$ from the point $x_0 = 1.5$ the tangents lead further and further away with each step: the theorem promises success only near the root, and how near isn’t always clear in advance. In practice Newton’s method is often combined with the reliable but slow bisection from the chapter on limits.

A patent dispute

Every inventors’ contest has its patent disputes, and calculus is no exception. In 1696 the first textbook of the differential calculus came out in Paris, Analysis of the Infinitely Small for the Understanding of Curved Lines. Its author was given as the Marquis Guillaume de l’Hôpital. The marquis was an able mathematician, and in the preface he thanked Leibniz and the Bernoulli brothers. After his death Johann Bernoulli claimed that most of the book was his own work, and he wasn’t much believed. Correspondence published in the twentieth century showed that Bernoulli was telling the truth: under an agreement of 1694 he sent the marquis his discoveries in return for an annual salary, and the marquis could do with them as he pleased. The most famous of them has gone down in history under l’Hôpital’s name. It helps compute limits of the form “zero over zero”.

For example, $\lim\limits_{x \to 1} \frac{\ln x}{x - 1} = \frac{1/1}{1} = 1$, and $\lim\limits_{x \to 2} \frac{x^3 - 8}{x - 2} = \frac{3 \cdot 4}{1} = 12$. There’s also a stronger version: if $f$ and $g$ tend to zero and the limit of the ratio of the derivatives $\frac{f'(x)}{g'(x)}$ exists, then $\frac{f(x)}{g(x)}$ tends to the same number. It’s proved through a generalisation of Lagrange’s theorem, Cauchy’s mean value theorem, and it works even when $f'(a)$ and $g'(a)$ don’t exist or are zero. That’s covered in the chapter on rigorous analysis.

Can the limit $\lim\limits_{x \to 0} \frac{x + 1}{x + 2}$ be found by l’Hôpital’s rule?

Yes, it’s $1$ No: this isn’t a “zero over zero” form, the limit is simply $\frac12$ No, and the limit doesn’t exist

At zero the numerator is $1$ and the denominator $2$, the function is continuous, and the limit is $\frac12$. L’Hôpital’s rule applies only to indeterminate forms. A similar trap is to “prove” by l’Hôpital that $\frac{\sin x}{x} \to 1$: that needs the derivative of the sine at zero, which in the last chapter we derived precisely from this limit. The argument goes round in a circle.

Where next

All the problems in this chapter went one way: we know a function, we find its rate of change, and from the rate, where the function is largest. Life often poses the reverse problem. A dashcam recorded the speedometer readings for the whole trip: at every moment we know the speed, but the odometer is broken. How far did the car go? If the speed were constant, a multiplication would do. But the needle was moving all the time. Lagrange’s theorem has already hinted that a function is determined by its derivative up to a constant; what remains is to learn how to recover it. That’s the job of the chapter on the integral.