Rigorous analysis

The last chapter ended with a confession. We compared infinities, counted fractions and proved that a segment has more points than there are natural numbers, but we never said what a single point, a single real number, is. For two hundred years analysis got by perfectly well without an answer. Newton, Leibniz and Euler relied on a visual certainty: a number is a point on a line, a function is a curve that can be drawn, and whatever is true of the picture is true of the numbers.

In the second half of the nineteenth century this certainty collapsed. One after another, functions appeared that behaved in ways that “can’t happen” and refuted statements that had been considered proved for decades. Henri Poincaré greeted them without enthusiasm:

Logic sometimes makes monsters. For half a century we have seen a mass of bizarre functions which appear to be forced to resemble as little as possible honest functions which serve some purpose.

A few lines further on Poincaré calls the whole collection a teratological museum, a museum of freaks. We’ll choose an older genre. Medieval books about monsters were called bestiaries: for each beast they tell you where it lives, why it’s dangerous and how to overcome it. This chapter is built the same way. Each monster in the bestiary of analysis forced some notion to be made rigorous: the real number, continuity, convergence. Along the way we’ll pay the debts the course has been running up since the chapter on limits: theorems we called obvious or promised to prove later will get full proofs here.

Jagged all the way down

Take a cosine wave, add to it a cosine wave three times as frequent and half as high, then another three times more frequent and half as high again, and so on without end:

The height of the $n$th wave. The number $a$ lies between zero and one, so the waves get lower and lower. The series converges at every point: its terms are no bigger in absolute value than the terms of the geometric series $a^n$ (the comparison test, chapter 30). The frequency of the $n$th wave: $b$ is an integer greater than one, and each next wave is $b$ times as frequent as the one before. Example: with $a = \frac12$, $b = 3$, all the cosines at zero equal one, and $W(0) = 1 + \frac12 + \frac14 + \ldots = 2$. At the point $\frac13$ the first cosine is $\cos\frac\pi3 = \frac12$, and the others are $\cos(3^{n-1}\pi) = -1$, because $3^{n-1}$ is odd. We get $W\bigl(\frac13\bigr) = \frac12 - \bigl(\frac12 + \frac14 + \ldots\bigr) = -\frac12$.

Each term is a smooth wave with a tangent at every point. Any finite sum of such waves is smooth too. But Weierstrass proved that the infinite sum, for odd $b$ and $ab > 1 + \frac{3\pi}{2}$, has no derivative at any point, though the function itself is continuous everywhere. In 1916 Godfrey Hardy weakened the condition to $ab \ge 1$, so the pair $a = \frac12$, $b = 3$ from the example will do too.

Why was this a blow? Intuition says a continuous curve can break only here and there, like a polygonal line, and is smooth between the breaks. In 1806 André-Marie Ampère published a “proof” that a continuous function has a derivative everywhere except at isolated points, and many nineteenth-century textbooks repeated the conclusion. In chapter 26 we saw how a smooth graph straightens into its tangent under strong magnification. Let’s see what the Weierstrass function does under the magnifying glass.

However far you zoom, the picture never settles down: every window has as many jags as the whole graph. The key thing shows in the row of slopes. For a smooth function the slope of the secant almost stops changing and approaches the derivative. For the monster it jumps about and grows in absolute value, and it has no limit at all. And the derivative is exactly the limit of the slopes of secants.

Not everyone accepted such functions as lawful citizens of mathematics. Charles Hermite wrote to Thomas Stieltjes:

I turn away with fright and horror from this lamentable plague of continuous functions which have no derivatives.

There was only one way to argue with the Weierstrass function: find a mistake in the proof. There was no mistake. The mistake was in the picture on which the earlier reasoning relied: continuity defined through limits (chapter 25) turned out to be much broader than “a curve without breaks and with tangents”. The lesson of the first monster is to trust definitions, not drawings. We’ll tame it at the end of the chapter: we’ll prove that it’s continuous, and prove that a close relative, whose calculations are shorter, is nowhere differentiable.

Each term of a series is a smooth function, and the series converges at every point. What follows for the sum?

The sum is smooth The sum is at least continuous Without further conditions, neither

The smoothness of the terms doesn’t pass to the sum of an infinite series, and even continuity doesn’t always pass. What has to be added to make it pass we’ll find out in the section on Cauchy’s mistake: it’s uniform convergence.

A function without a graph

It can’t be drawn: any tiny segment contains both fractions and irrational numbers, so the “graph” is two seemingly solid bands at heights $0$ and $1$, and at no point can you say where it goes next. Dirichlet himself spoke of two arbitrary constants $c$ and $d$; one and zero are a later convention. For the eighteenth century this wasn’t a function at all: Euler understood a function as an expression made up of the variable and numbers. Dirichlet insisted on something else: a function is any rule assigning a number to each $x$, and it doesn’t matter whether the rule has a formula or a graph. That’s how it’s defined today (chapter 9).

The Dirichlet function is discontinuous everywhere because every point has equal numbers of neighbours of both kinds: infinitely many. What if the jumps at fractions are made smaller the larger the denominator?

A kernel has popped up above each fraction: above a half to a height of $\frac12$, above the thirds to $\frac13$, and the smaller the fraction’s pieces, the lower. The function is periodic with period $1$: shifting by an integer doesn’t change the denominator. So it’s enough to study it on the segment $[0, 1]$.

The trick is that only finitely many kernels rise above any level, and an irrational number can always be fenced off from finitely many points.

Mark the points $\bigl(\frac pq,\ \frac1q\bigr)$ over the segment $[0, 1]$: the smaller the denominator, the higher the point. All the irrational points lie on the axis, where $T = 0$. Let $x$ be irrational, and let the Sceptic name a tolerance $\varepsilon$. Only fractions with denominators $q \le \frac1\varepsilon$ rise above the level $\varepsilon$, and on the segment $[0, 1]$ there are finitely many such fractions: for each $q$ at most $q + 1$. The number $x$ coincides with none of them, since it’s irrational. The distances from $x$ to finitely many points are positive, and the smallest of them is positive too. Call it $\delta$. The window $(x - \delta,\ x + \delta)$ contains no tall kernel: irrational points give $0$, and fractions give values below $\varepsilon$. So $|T(y) - T(x)| = T(y) < \varepsilon$, and the game is won: $T$ is continuous at $x$. If $x = \frac pq$, then $T(x) = \frac1q > 0$, while any window around $x$ contains irrational points where $T = 0$. For the tolerance $\varepsilon = \frac1q$ there’s no window: at every rational point there’s a discontinuity.

It’s an amusing monster: it’s continuous where it “isn’t there” (on the axis) and breaks at every kernel. The reverse can’t happen: a theorem proved by Vito Volterra in 1881 implies that there’s no function continuous at every rational point and discontinuous at every irrational one. And popcorn does have an integral.

Holes between the fractions

We saw the holes in the rational line in chapter 6, and now we can see how costly they are for analysis. If there were only as many numbers as there are fractions, three main theorems would collapse. The sequence $1,\ 1.4,\ 1.41,\ 1.414,\ \ldots$ increases and never exceeds $2$, but it has no limit among the fractions. The function $x^2 - 2$ on the fractions of the segment $[1, 2]$ is continuous and changes sign, yet it’s zero nowhere. The function $-(x^2 - 2)^2$ on the same fractions is continuous and bounded but has no largest value: it never takes the value zero, yet comes as close to zero as you like.

So the “no holes” property has to be stated explicitly, and for that we have to say what a real number is without appealing to a picture. The definition from chapter 6, a number as an infinite decimal, is awkward for this: it’s unclear how to add and multiply infinite expansions if carries come “from infinity”. Richard Dedekind, by his own account, found a way out on 24 November 1858, while preparing lectures at the Zurich Polytechnic: he was uncomfortable proving the basic theorems of analysis by appealing to geometric intuition. He published his solution only in 1872, in the booklet Continuity and Irrational Numbers.

Dedekind’s idea is this. Cut the line somewhere: all the points to the left of the cut are the lower class, to the right the upper class. The continuity of the line means that a cut always falls at a point. For fractions that’s not so: cut them by the rule “$r < 0$ or $r^2 < 2$”, with everything to the left of $\sqrt2$ below and everything to the right above. There’s no point at the place of the cut. Dedekind proposed not to look for a point for such a cut but to declare the cut itself a number.

Each fraction $r$ gives a cut $\{x \in \mathbb Q : x < r\}$; its upper class has a smallest element, $r$ itself. That’s how the fractions take their places in the new world. The cut $\sqrt2 = \{r \in \mathbb Q : r < 0 \text{ or } r^2 < 2\}$ is different: there’s no largest fraction in the lower class and no smallest in the upper. Check for yourself: try to name the largest fraction whose square is less than two.

The machine uses one formula: $r \mapsto \frac{2r + 2}{r + 2}$. If $r > 0$ and $r^2 < 2$, the new fraction is larger: the difference is $\frac{2 - r^2}{r + 2} > 0$. And its square is still less than two: $\left(\frac{2r + 2}{r + 2}\right)^2 - 2 = \frac{2(r^2 - 2)}{(r + 2)^2} < 0$. For a fraction from the upper class the same calculation gives a smaller fraction whose square is still greater than two. Starting from one, the machine produces $\frac43$, $\frac75$, $\frac{24}{17}$, $\frac{41}{29}$, …, among them the convergents of $\sqrt2$ familiar from chapter 6.

Cuts are compared by inclusion: $A \le B$ if $A \subset B$. Any two cuts are comparable. If $A$ isn’t contained in $B$, take a fraction $a \in A$ outside $B$. Every fraction $b \in B$ is less than $a$: otherwise $a \le b$, and $a$ would lie in $B$ along with $b$. And since $b < a$, $b \in A$. So $B \subset A$. The sum is defined elementwise: $A + B = \{a + b : a \in A,\ b \in B\}$. Multiplication takes more fuss, as the signs have to be sorted out. Checking all the rules of arithmetic and inequalities for cuts is long and dull but honest; it’s done in full, for example, in the appendix to the first chapter of Walter Rudin’s Principles of Mathematical Analysis. The main property of cuts, though, takes three lines to check.

A cut is a ray of fractions extending to the left. A union of rays is again a ray, and none of them goes beyond it.

Draw the cuts $A_i$ as rays of fractions. In the picture their ends creep up on $\sqrt2$ from below, like the cuts $\{x < 1\}$, $\{x < 1.4\}$, $\{x < 1.41\}$, … They all lie inside the cut $\Gamma$: that’s the upper bound. Take their union: $U$ is all the fractions that belong to at least one $A_i$. $U$ is a cut. It isn’t empty, because it contains each $A_i$, and it isn’t all of $\mathbb Q$, because it lies inside $\Gamma$. If $u \in U$, then $u$ lies in some $A_i$; together with $u$ that $A_i$ contains all smaller fractions and some fraction larger than $u$ (there’s no largest in $A_i$). All of them fall into $U$, so $U$ contains everything smaller than $u$ and has no largest element. $U$ is an upper bound of the family: each $A_i \subset U$, that is, $A_i \le U$. And the smallest one: if a cut $\Gamma'$ is greater than or equal to all the $A_i$, then all $A_i \subset \Gamma'$, and so their union $U \subset \Gamma'$. In the picture the union of the rays reached exactly $\sqrt2$: the hole there was nothing to plug with among the fractions.

In the same year, 1872, Georg Cantor, and three years earlier Charles Méray, proposed another path: a number is a sequence of fractions that “converges in itself”, though it may have no limit among the fractions (below such sequences are called Cauchy sequences). Two such sequences are taken to be the same number if their difference tends to zero: that’s an equivalence relation from chapter 51, and a number is its class. Weierstrass too presented his own construction in his lectures. All the paths lead to the same result.

The completeness axiom

The scaffolding can come down. From now on we won’t need to remember what the real numbers are made of: the rules of arithmetic, the rules of inequalities and one property that distinguishes $\mathbb R$ from $\mathbb Q$ will be enough. Let’s state it without any cuts.

For cuts this isn’t an axiom but the theorem we’ve just proved: the union is the least upper bound. So by accepting the axiom we risk nothing: a model in which it holds has been built from fractions. For the fractions themselves it’s false: the set $\{r \in \mathbb Q : r^2 < 2\}$ has as many upper bounds among the fractions as you like, but no least one.

$s$ is an upper bound: no element of the set is above it. Nothing smaller will do: however far you lower the bar to $s - \varepsilon$, some element ends up above it. That’s what “least” means. Example: $A = \bigl\{1 - \frac1n : n \in \mathbb N\bigr\}$. All the elements are less than $1$, and for $\varepsilon = 0.01$ there’s the element $1 - \frac1{101} \approx 0.9901 > 0.99$; in general any $n > \frac1\varepsilon$ will do. So $\sup A = 1$, though $A$ has no largest element.

When a set has a largest element, that’s its least upper bound: $\sup [0, 1] = 1$. But the least upper bound is useful precisely when there’s no largest element: $\sup\,[0, 1) = 1$, and $1 \notin [0, 1)$. The first thing the axiom gives is a property we accepted without proof in chapter 25.

A set $A$ is bounded above, and $s = \sup A$. Which of these is always true?

$s$ belongs to $A$ Every number less than $s$ is less than some element of $A$ Every neighbourhood of $s$ contains infinitely many elements of $A$

That’s the second half of the definition: if $t < s$, then $t = s - \varepsilon$ with $\varepsilon = s - t > 0$, and some element is greater than $t$. The rest depends on the set.

Finding least upper bounds is a skill that will come in handy throughout what follows. The recipe is always the same: guess a candidate, check that it’s an upper bound, and show that elements come arbitrarily close to it.

Four faces of completeness

The axiom speaks about sets, while analysis speaks about sequences. The four theorems below translate it into the language of sequences, and each has its own role. The course has already used two of them as axioms: the nested intervals principle in chapter 25, and the theorem on monotone sequences in chapter 30. Now they’ll become theorems.

There’s no need to search for the limit: the completeness axiom suggests it.

The terms $a_1 \le a_2 \le a_3 \le \ldots$ lie below some number, and by the completeness axiom the set of terms has a least upper bound $s$. The Sceptic names a tolerance $\varepsilon$. The number $s - \varepsilon$ is less than $s$ and so is no longer an upper bound: some term $a_N$ is greater than $s - \varepsilon$. The sequence doesn’t decrease, so all the terms after $a_N$ are also greater than $s - \varepsilon$. And no term rises above $s$. So for $n > N$ all the $a_n$ lie in the band $(s - \varepsilon,\ s]$, and $|a_n - s| < \varepsilon$. For every tolerance an index has been found: $a_n \to s$.

For a non-increasing sequence bounded below it’s the same with the greatest lower bound. The theorem pays the debt of chapter 12: the powers $2^{1.4} \approx 2.6390$, $2^{1.41} \approx 2.6574$, $2^{1.414} \approx 2.6647$, … grow with the exponents and don’t exceed $2^{1.5} \approx 2.8284$, so they have a limit. That limit, $2.66514\ldots$, is what’s called $2^{\sqrt2}$. The infinite decimals of chapter 30 get their values for the same reason.

Now the proof of the intermediate value theorem from chapter 25, which rested on this principle, is fully rigorous. The third face of completeness is about sequences that don’t tend anywhere.

We’ll keep halving the segment the terms live in, each time keeping a half with infinitely many of them. The picture shows the sequence $a_n = \sin n$: it has no limit, and its terms turn up infinitely often in every little piece of the segment $[-1, 1]$, so either half will do; we stick with the one containing $0.3$.

The sequence is bounded, so all its terms lie in some segment $I_0$. Halve $I_0$. There are infinitely many indices $n$, so at least one half contains infinitely many terms $a_n$. Keep that one and call it $I_1$. Repeat with $I_1$, then with $I_2$, and so on. We get nested segments $I_0 \supset I_1 \supset I_2 \supset \ldots$, each half as long as the one before and with infinitely many terms. Choose a term in each: $a_{n_1} \in I_1$, then $a_{n_2} \in I_2$ with index $n_2 > n_1$ (there is one, since $I_2$ has infinitely many terms), then $a_{n_3} \in I_3$ with $n_3 > n_2$, and so on. By the nested intervals principle all the $I_k$ have a common point $x$. Both $a_{n_k}$ and $x$ lie in $I_k$, so $|a_{n_k} - x|$ is at most the length of $I_k$, which is the length of $I_0$ divided by $2^k$ and tends to zero. The subsequence converges to $x$.

The last face of completeness lets you learn that a limit exists without knowing what it is.

Cauchy’s criterion is the most portable form of completeness: it involves neither order nor least upper bounds, only distances. That’s why it’s taken as the definition of completeness where “greater” and “less” don’t exist: for complex numbers, vectors, functions (chapter 55). Among the fractions the sequence $1,\ 1.4,\ 1.41,\ \ldots$ is a Cauchy sequence but doesn’t converge: the same hole again.

Neighbouring terms of a sequence get closer: $a_{n+1} - a_n \to 0$. Does this mean the sequence converges?

Yes, that’s what a Cauchy sequence is No

Take the partial sums of the harmonic series $H_n = 1 + \frac12 + \ldots + \frac1n$ (chapter 30). Neighbours differ by $\frac1{n + 1} \to 0$, but $H_{2n} - H_n \ge n \cdot \frac1{2n} = \frac12$, and the sequence goes off to infinity. The steps get smaller, but the journey never ends.

Can it be done the other way round, with nested intervals as the axiom?

If an ordered field satisfies the Archimedean property and the nested intervals principle, it satisfies the completeness axiom too.

Let $A$ be non-empty and bounded above. Take an element $a_0 \in A$ and an upper bound $b_0$, and keep halving the segment $[a_0, b_0]$. If the midpoint is an upper bound of $A$, keep the left half; if not, the right half. This preserves two properties: the right end $b_k$ is an upper bound of $A$, and the segment $[a_k, b_k]$ contains an element of $A$ (in the first case the old one, in the second an element to the right of the midpoint). The lengths $\frac{b_0 - a_0}{2^k}$ tend to zero by the Archimedean property, and the segments have a unique common point $s$. If some $a \in A$ were greater than $s$, then once $b_k - a_k < a - s$ the right end $b_k \le s + (b_k - a_k) < a$ wouldn’t be an upper bound. And for any $\varepsilon > 0$, once $b_k - a_k < \varepsilon$, an element of $[a_k, b_k]$ is greater than $b_k - \varepsilon \ge s - \varepsilon$. So $s = \sup A$.

That’s why different textbooks state completeness differently, and they’re all right: nested intervals (with Archimedes), monotone sequences, Cauchy sequences (with Archimedes) and least upper bounds are one and the same property. Chapter 25 lost nothing by taking nested intervals as an axiom.

Paying the debts

Rigorous proofs were put off “until chapter 53” more than once in the course. Here’s the ledger of debts:

Let’s start with the first row. In chapter 25 the intermediate value theorem was proved by halving, and that proof is now complete. Here’s a second one, in the spirit of this chapter: the root will be pointed out by a least upper bound.

Take the rightmost place up to which the function is still sometimes negative: the least upper bound of the set where $f < 0$.

The set $S = \{x \in [a, b] : f(x) < 0\}$ isn’t empty (it contains $a$) and is bounded above by $b$. Let $c = \sup S$; then $a \le c \le b$. Could $f(c) < 0$? Then $c \ne b$, and by continuity $f$ is negative on a whole window around $c$ (play the Sceptic with tolerance $|f(c)|$). The points of the window to the right of $c$ lie in $S$, but $c$ was an upper bound. Impossible. Could $f(c) > 0$? Then $c \ne a$, and $f$ is positive on a window $(c - \delta,\ c]$. There are no points of $S$ in it, and still fewer to the right of $c$. So $c - \delta$ is also an upper bound of $S$, smaller than $c$, contrary to $c$ being the least one. Impossible. What’s left is $f(c) = 0$. And $c$ lies inside the segment, because $f$ isn’t zero at the ends.

The root found this way is special: to the right of it the function is never negative again. The next debt requires the Bolzano–Weierstrass theorem and one simple observation: a continuous function maps convergent sequences to convergent ones. If $x_k \to c$ and $f$ is continuous at $c$, then for a tolerance $\varepsilon$ continuity gives a window $\delta$, the terms $x_k$ fall into it from some index on, and from then on $|f(x_k) - f(c)| < \varepsilon$. We’ll also need the fact that the limit of terms from a segment $[a, b]$ lies in the same segment: if $x_k \to c > b$, then for the tolerance $c - b$ the terms would end up to the right of $b$.

We’ll apply the Bolzano–Weierstrass theorem twice: first to prove that the values are bounded, then to find a point where their least upper bound is attained.

Suppose the values of $f$ aren’t bounded above. Then for each $n$ there’s an $x_n \in [a, b]$ with $f(x_n) > n$. The sequence $x_n$ is confined to the segment, and it has a subsequence $x_{n_k} \to c$, with $c \in [a, b]$. By continuity $f(x_{n_k}) \to f(c)$, that is, to a number. But $f(x_{n_k}) > n_k \ge k$, and the values grow without bound. A contradiction: $f$ is bounded above. So the set of values has a least upper bound $M$. For each $n$ the number $M - \frac1n$ is no longer an upper bound: there’s a $y_n$ with $M - \frac1n < f(y_n) \le M$. Again choose a subsequence $y_{n_k} \to d \in [a, b]$. By continuity $f(y_{n_k}) \to f(d)$, and by the squeeze theorem (chapter 25) $f(y_{n_k}) \to M$. A sequence has only one limit, so $f(d) = M$: the largest value is attained at the point $d$. The smallest value of $f$ is the largest value of $-f$ with a minus sign.

Both conditions are needed, and the examples from chapter 27 show what happens without them: $\frac1x$ on $(0, 1]$ has unbounded values, and $x$ on $[0, 1)$ has no largest value. You can see it in the proof: where there’s no closed segment, the limit $c$ can drop out of the domain. And chapter 38 needed the theorem for functions on a sphere.

The next debt is to roots and logarithms. Their continuity follows from a general fact about inverse functions.

The function maps a window on the $x$-axis to a window on the $y$-axis, and all that remains is to take a smaller window.

By the intermediate value theorem $f$ takes each value $y \in [f(a), f(b)]$, and since it’s strictly increasing, exactly once. So the inverse function $g$ is defined on the whole segment $[f(a), f(b)]$ and is increasing too. Take $y_0 = f(x_0)$ and a tolerance $\varepsilon$ for $g$: we want $g(y)$ to land in the window $(x_0 - \varepsilon,\ x_0 + \varepsilon)$. Suppose it fits in $[a, b]$; near the ends of the segment everything is the same with half a window. The function $f$ maps the ends of this window to points $f(x_0 - \varepsilon) < y_0 < f(x_0 + \varepsilon)$. Take $\delta$ to be the smaller of the distances from $y_0$ to them. If $|y - y_0| < \delta$, then $f(x_0 - \varepsilon) < y < f(x_0 + \varepsilon)$, and since $g$ is increasing, $x_0 - \varepsilon < g(y) < x_0 + \varepsilon$. A window has been found: $g$ is continuous at $y_0$.

The function $x^n$ is continuous and strictly increasing on any segment $[0, R]$, so the root $\sqrt[n]{y}$ is continuous on $[0, R^n]$, and hence on the whole ray $[0, +\infty)$. In the same way $\arcsin$ (the inverse of sine on $\bigl[-\frac\pi2, \frac\pi2\bigr]$) and $\arctan$ are continuous, and the logarithm is the inverse of the continuous exponential (chapter 29).

The debt of chapter 27 is l’Hôpital’s rule for the cases where derivatives at the point itself don’t help. We need a mean value theorem for two functions at once.

Look at the pair of functions as a curve in the plane: then the left side of the equation is the slope of a chord, the right side the slope of a tangent, and the theorem says that somewhere they’re parallel.

Draw the curve $t \mapsto \bigl(g(t),\ f(t)\bigr)$ for $t \in [a, b]$. The chord between its ends has slope $\frac{f(b) - f(a)}{g(b) - g(a)}$; the denominator isn’t zero, otherwise by Rolle’s theorem (chapter 27) there’d be $g' = 0$ somewhere. The tangent at the point $t$ points along the vector $\bigl(g'(t),\ f'(t)\bigr)$ and has slope $\frac{f'(t)}{g'(t)}$. Consider $h(t) = \bigl(f(b) - f(a)\bigr)\bigl(g(t) - g(a)\bigr) - \bigl(g(b) - g(a)\bigr)\bigl(f(t) - f(a)\bigr)$. Up to a constant factor, this is the signed distance from a point of the curve to the line of the chord, and at the ends $h(a) = h(b) = 0$. The function $h$ is continuous on the segment and differentiable inside, and by Rolle’s theorem there’s a $\xi$ with $h'(\xi) = 0$. For example, where the curve is furthest from the chord. $h'(\xi) = \bigl(f(b) - f(a)\bigr)g'(\xi) - \bigl(g(b) - g(a)\bigr)f'(\xi) = 0$. Dividing by $g'(\xi)\bigl(g(b) - g(a)\bigr) \ne 0$, we get the required equation: the tangent at $\xi$ is parallel to the chord.

The simple case in chapter 27 required $g'(a) \ne 0$. Now we can do this: $\lim\limits_{x \to 0}\frac{x - \sin x}{x^3}$. The ratio of derivatives $\frac{1 - \cos x}{3x^2}$ is “zero over zero” again, so apply the rule to it: $\frac{\sin x}{6x} \to \frac16$. So $\frac{1 - \cos x}{3x^2} \to \frac16$ too, and $\frac{x - \sin x}{x^3} \to \frac16$. A numerical check: at $x = 0.1$ the fraction equals $0.16658\ldots$

One δ for everyone

In the game with the Sceptic from chapter 25 the window $\delta$ is chosen after $\varepsilon$ and after the point $x$, so it can depend on both. For $x^2$ at a point $x_0$ with tolerance $\varepsilon$ a window of about $\frac{\varepsilon}{2x_0}$ will do: the further right the point, the steeper the graph and the narrower the window. For $\sin\frac1x$ right next to zero the windows shrink to almost nothing. For many purposes that’s bad. To draw up a table of values with guaranteed accuracy, draw a graph or compute an integral, you need one grid step that works at all points at once.

The window $\delta$ is chosen before the points are known: there’s one for the whole set. Any two points, wherever they are. In ordinary continuity the point $x$ is fixed first, and only then is a $\delta$ sought for it: the quantifiers come in a different order (chapter 25). Close points mean close values, equally throughout the domain. Example: $x^2$ on $[0, 2]$. For $x, y$ in the segment $|x^2 - y^2| = |x + y|\,|x - y| \le 4|x - y|$, and for tolerance $\varepsilon$ the window $\delta = \frac\varepsilon4$ works for all points at once: at $\varepsilon = 0.01$ that’s $\delta = 0.0025$. On the whole line it can’t be done: the points $n$ and $n + \frac1n$ are arbitrarily close, but their squares differ by $2 + \frac1{n^2} > 2$.

The graph of $\sqrt x$ on $[0, 1]$ is vertical at zero, yet the function is uniformly continuous: $|\sqrt x - \sqrt y| \le \sqrt{|x - y|}$, and $\delta = \varepsilon^2$ will do. It’s not a matter of steepness but of whether the domain has its ends. That’s what the main theorem of this section says. Many textbooks call it the Heine–Cantor theorem, others simply Cantor’s theorem: Eduard Heine published it in 1872, and Dirichlet had proved it in lectures as early as 1854.

If there were no single $\delta$, there would be pairs of points arbitrarily close to each other but with distant values. The Bolzano–Weierstrass theorem will herd them to one point, and continuity at that point won’t allow it.

Suppose there’s no single $\delta$. By the rule for negating quantifiers (chapter 51) this means: there’s a tolerance $\varepsilon > 0$ for which no $\delta$ works. For $\delta = \frac1n$ there are points $x_n, y_n$ with $|x_n - y_n| < \frac1n$ but $|f(x_n) - f(y_n)| \ge \varepsilon$. By the Bolzano–Weierstrass theorem there’s a subsequence $x_{n_k} \to c$, with $c \in [a, b]$. Then $y_{n_k} \to c$ too: $|y_{n_k} - c| \le |y_{n_k} - x_{n_k}| + |x_{n_k} - c|$, and both terms tend to zero. The function is continuous at $c$: for the tolerance $\frac\varepsilon2$ there’s a window around $c$ where the values differ from $f(c)$ by less than $\frac\varepsilon2$. For large $k$ both $x_{n_k}$ and $y_{n_k}$ fall into it. Then $|f(x_{n_k}) - f(y_{n_k})| \le |f(x_{n_k}) - f(c)| + |f(c) - f(y_{n_k})| < \varepsilon$, contrary to the choice of the pairs. A contradiction: one $\delta$ for the whole segment exists.

The proof breaks down exactly where the theorem does. For $\sin\frac1x$ on $(0, 1]$ the pairs of points are herded towards zero, but zero isn’t in the domain, and continuity there can’t be used. Now we can pay the debt of chapter 28.

One $\delta$ for the whole segment lets us make all the columns of the gap between the lower and upper sums low at once.

The Sceptic names $\varepsilon$. By the Heine–Cantor theorem there’s a $\delta$ such that on any piece shorter than $\delta$ the values of $f$ differ from one another by less than $\varepsilon$. Take any partition into pieces shorter than $\delta$. On each piece, by the extreme value theorem, there’s a smallest value $m_i$ and a largest $M_i$, and $M_i - m_i < \varepsilon$. The lower sum $L = \sum m_i\,\Delta x_i$ and the upper sum $U = \sum M_i\,\Delta x_i$ squeeze every Riemann sum of this partition. The gap $U - L$ is columns along the graph of width $\Delta x_i$ and height less than $\varepsilon$. Together their area is less than $\varepsilon\,(b - a)$. Any lower sum is at most any upper sum: two partitions have a common refinement, and on refining the lower sum doesn’t decrease and the upper sum doesn’t increase (chapter 28). So the lower sums have a least upper bound $I$, and $L \le I \le U$ for any partition. The Riemann sum and $I$ lie in the same interval $[L, U]$ of length less than $\varepsilon\,(b - a)$ and differ by less than that amount, for every partition finer than $\delta$ and every choice of points. So $f$ is integrable and $\int_a^b f(x)\,dx = I$.

For continuous functions on a rectangle from chapter 32 the argument is the same: the Heine–Cantor theorem in $\mathbb R^2$ is proved in the same way, with the Bolzano–Weierstrass theorem for points of the plane, and cells take the place of pieces of a segment.

Cauchy’s mistake

In his Cours d’analyse of 1821 Augustin Cauchy proved: if a series of continuous functions converges at every point, its sum is continuous. Five years later Abel, in a paper on the binomial series, cautiously remarked that the theorem seemed to have exceptions, and gave the series $\sin x - \frac12 \sin 2x + \frac13 \sin 3x - \ldots$: it converges for all $x$, its sum on the interval $(-\pi, \pi)$ is $\frac x2$, and at the point $\pi$ it’s zero. The sum of continuous sines jumps. A simpler example is to hand: the functions $x^n$ on $[0, 1]$ are continuous, but their limit is $0$ for $x < 1$ and $1$ at $x = 1$.

The index is chosen before the point: one for the whole set. For pointwise convergence it’s different for each $x$. All the points of the domain at once. The graph of $f_n$ lies entirely inside a tube of width $2\varepsilon$ around the graph of $f$. Equivalently: $\sup_{x \in D}|f_n(x) - f(x)| \to 0$. Example: $x^n$ on $[0, 0.9]$ tends to zero uniformly: the largest deviation is $0.9^n$, and for $\varepsilon = 0.01$ the index $N = 43$ will do, since $0.9^{44} \approx 0.0097$. On $[0, 1)$ the convergence to zero is pointwise but not uniform: for any $n$ there’s an $x < 1$ with $x^n > \frac12$.

Split the tolerance into three parts: two go on the distance from $f$ to $f_N$, one on the continuity of $f_N$ itself.

Take a point $x_0 \in D$ and a tolerance $\varepsilon$. By uniform convergence there’s an index $N$ for which the whole graph of $f$ lies inside a tube around $f_N$ of width $\frac\varepsilon3$ on each side: $|f(x) - f_N(x)| < \frac\varepsilon3$ for all $x$. The function $f_N$ is continuous at $x_0$: there’s a window $\delta$ in which $|f_N(x) - f_N(x_0)| < \frac\varepsilon3$. In this window the way from $f(x)$ to $f(x_0)$ is made up of three short ones: $|f(x) - f(x_0)| \le |f(x) - f_N(x)| + |f_N(x) - f_N(x_0)| + |f_N(x_0) - f(x_0)| < \frac\varepsilon3 + \frac\varepsilon3 + \frac\varepsilon3 = \varepsilon$. So $f$ is continuous at $x_0$.

This shows why $x^n$ on $[0, 1]$ can’t converge uniformly: the limit is discontinuous. In Cauchy’s proof the index $N$ was tacitly chosen once for all points, that is, uniform convergence was used, though it wasn’t in his hypothesis. Historians still argue about what exactly Cauchy meant; his “infinitely small” quantities can be read in different ways. But the notion that fixes everything was singled out in the 1840s by Weierstrass in an unpublished manuscript and, independently, by Philipp Seidel and George Stokes, and in Weierstrass’s lectures it became standard. Uniform convergence is conveniently checked with one test.

The first debt to the monster is paid: the terms of the Weierstrass series are at most $a^n$ in absolute value, the series $\sum a^n$ converges, and the function $W$ is the uniform limit of continuous partial sums, so it’s continuous. The steps in chapters 31 and 33, where the limit of successive approximations or of a Fourier series turned out to be a continuous function, are justified the same way. What remains is term-by-term differentiation, which we allowed ourselves without proof in chapter 30.

The functions $f_n$ are differentiable and converge to $f$ uniformly. Must $f$ be differentiable?

Yes: uniform convergence preserves everything good No

Take $f_n(x) = \sqrt{x^2 + \frac1n}$. They’re smooth, and $0 \le f_n(x) - |x| = \frac{1/n}{\sqrt{x^2 + 1/n} + |x|} \le \frac{1}{\sqrt n}$ for all $x$, so $f_n \to |x|$ uniformly. But $|x|$ has a corner at zero. The derivatives $f_n'$ converge to the sign of $x$, and not uniformly: hence the corner.

Taming

Back to the first monster. We’ve already proved its continuity with the Weierstrass M-test. Proving that $W$ has no derivative at any point requires long estimates of cosines. Let’s do the same for a relative that has saw-teeth instead of cosines. It was devised by the Japanese mathematician Teiji Takagi in 1903.

A saw: $s(y)$ is the distance from $y$ to the nearest integer. Its teeth are line segments with slopes $\pm1$ and height $\frac12$; the teeth of $s(2^n x)$ are $2^n$ times narrower. Each next saw is half as high. On the graph of $\frac{s(2^n x)}{2^n}$ the teeth have width $\frac1{2^{n+1}}$ and slopes $\pm1$. Example: $T\bigl(\frac12\bigr) = \frac12 + 0 + 0 + \ldots = \frac12$, since $s$ of an integer is zero. At the point $\frac13$ the numbers $\frac{2^n}{3}$ are exactly $\frac13$ from the nearest integer, and $T\bigl(\frac13\bigr) = \frac13\bigl(1 + \frac12 + \frac14 + \ldots\bigr) = \frac23$. That’s the largest value of the function.

The terms of the series are at most $\frac{1}{2^{n+1}}$, so by the Weierstrass M-test $T$ is continuous. Its graph looks like an old-fashioned blancmange pudding, and English-language books call it the blancmange function. We’ll need one lemma about chords.

Look at the chords over the dyadic segments containing the point $x$. Their slopes are integers, and neighbouring ones differ by exactly one, so they can’t approach a single number.

Take a point $x$ and for each $n$ the dyadic segment $[u_n, v_n] = \bigl[\frac{k}{2^n},\ \frac{k + 1}{2^n}\bigr]$ of length $\frac1{2^n}$ that contains it. The segments are nested and shrink to $x$. The terms with indices $m \ge n$ are zero at the ends of this segment: the numbers $2^m u_n$ and $2^m v_n$ are integers, and the saw vanishes at integers. They contribute nothing to the difference $T(v_n) - T(u_n)$. Each term with index $m < n$ is linear on the segment $[u_n, v_n]$ with slope $+1$ or $-1$: a tooth of width $\frac1{2^{m+1}}$ is also a dyadic segment, and our segment, being no longer, lies entirely on one of its slopes. So the slope of the chord $D_n = \frac{T(v_n) - T(u_n)}{v_n - u_n}$ is a sum of $n$ terms each equal to $\pm1$: an integer of the same parity as $n$. Passing to $n + 1$, the old slopes don’t change and exactly one new term is added: $D_{n+1} = D_n \pm 1$. A sequence whose neighbouring terms differ by one doesn’t converge. But if there were a derivative at the point $x$, the slopes of the chords squeezing $x$ would tend to it by the lemma. There’s no derivative, and so at every point.

The final blow to intuition was dealt in 1931 by Stefan Banach and Stefan Mazurkiewicz. They proved independently that, in a precise sense, “almost all” continuous functions are nowhere differentiable, and smooth ones are a rare exception. The monsters turned out to be the majority, and Poincaré’s honest functions a nature reserve. Their descendants live in physics too: the path of a Brownian particle has, with probability one, no derivative anywhere (Paley, Wiener and Zygmund, 1933), and in the chapter on fractals, where jaggedness at every scale isn’t a defect but the object of study.

Where next

Thomae’s function is integrable, with integral zero, while the Dirichlet function isn’t Riemann integrable at all: every piece of any partition contains both fractions and irrational numbers, and the Riemann sums jump between $0$ and $b - a$. Yet the difference between them doesn’t seem deep. Both equal zero at all irrational points, and the fractions, as we found in chapter 52, are “few”: their set is countable, while the set of points of a segment isn’t. If the set of fractions on a segment weighs nothing, the integral of the Dirichlet function ought to be zero. What is the “length” of the set of rational points of the segment $[0, 1]$? And of the irrational ones, or of the Cantor set? To answer, we’ll have to define length afresh, and on the way we’ll meet pieces that can’t be measured at all, a ball from which two identical balls are assembled, and the final answer to the question about the circle and the square. That’s chapter 54.