Series

The last chapter ended with a question: can any function be written as an “infinite polynomial”, a sum $c_0 + c_1x + c_2x^2 + \dots$ with no last term? For the number $e$ we’ve already met such a sum: $e = 1 + 1 + \frac12 + \frac16 + \frac1{24} + \dots$ Before expanding functions, we’ll have to deal with a simpler question: what does it even mean to add up infinitely many numbers?

Infinite sums are no joke. Add up $1 - 1 + 1 - 1 + \dots$ and you get $0$, $1$ or $\frac12$, depending on how you count. Add up $1 + \frac12 + \frac13 + \dots$: the terms melt away, and yet the sum is infinite. Rearrange the terms of $1 - \frac12 + \frac13 - \dots$ and the sum becomes different. There are enough such oddities for a whole museum. Let’s set one up: we’ll walk through the rooms of a cabinet of curiosities of infinite sums and, for each exhibit, find out how it works. The old name for such a cabinet, Kunstkamera, is no accident here: in the eighteenth century the Academy of Sciences, where Leonhard Euler worked, met in the building of the Kunstkamera in St Petersburg, and Euler is the hero of the fourth room.

Room one: a sum that doesn’t exist

Exhibit no. 1 · Grandi’s series · Italy, 1703

The label reads: $1 - 1 + 1 - 1 + 1 - \dots$ How much is that? If you group the terms in pairs, you get $(1 - 1) + (1 - 1) + \dots = 0 + 0 + \dots = 0$. If you set the first one aside and start the pairs from the second term, you get $1 - (1 - 1) - (1 - 1) - \dots = 1$. And if you call the sum $S$ and notice that after the first one comes the same sum with the opposite sign, then $S = 1 - S$, so $S = \frac12$.

The answer $\frac12$ was defended by the Italian monk and mathematician Guido Grandi, and even Leibniz agreed with him. Three arguments, three answers, and not one arithmetical mistake in any of them. The mistake lies deeper: all three treat the infinite sum as if it already existed and obeyed the usual laws. With finite sums you can do that, but for an infinite one we first have to agree on what it means.

The agreement is this: add one term at a time and see what the running totals are closing in on.

The general term of the series: the rule that gives the $n$th term. A limit in the sense of the chapter on limits: for every $\varepsilon > 0$ there’s an index from which on the partial sums differ from $S$ by less than $\varepsilon$. The partial sum $S_N$ is an ordinary finite sum, and you can do everything with it that you usually do. Example: the series $\frac12 + \frac14 + \frac18 + \dots$ has partial sums $1 - \frac1{2^N}$, which tend to $1$; that’s its sum. Grandi’s series has partial sums $1, 0, 1, 0, \dots$, which jump about and approach nothing. It diverges and has no sum: not $0$, not $1$, not $\frac12$.

Now we can see where all three arguments stumbled. The pairs $(1 - 1)$ pick out only the even partial sums, $S_2, S_4, \dots$, and those are zero; the second grouping picks out the odd ones, equal to one. For a convergent series that’s harmless: all the partial sums head for one limit, and so does any selection of them. And the equation $S = 1 - S$ silently assumes that $S$ exists. There’s also a simple sign that Grandi’s series is doomed.

In Grandi’s series the terms are $\pm1$ and don’t tend to zero, so it diverges. The converse is false: the terms can tend to zero while the series diverges. The most famous example is waiting in the next room.

This room also holds an exhibit we see every day: an infinite decimal. The notation $0.d_1d_2d_3\dots$ means the series $\frac{d_1}{10} + \frac{d_2}{100} + \dots$, and the chapter on fractions promised to explain why such notation always has a value. The partial sums grow and don’t reach one: even for all nines they’re less than $\frac{9}{10} + \frac{9}{100} + \dots = 1$ (chapter 13). Where can a growing sequence that has hit a ceiling go? The answer is a property on which the whole of calculus rests.

For fractions this is false. The sequence $1$, $1.4$, $1.41$, $1.414$, … grows and never exceeds $2$, but it closes in on $\sqrt2$, and there’s no such fraction (chapter 6): the rational line has holes. The completeness axiom says that the real line has none. We’ll accept it as a basic property of $\mathbb R$; in chapter 53 the real numbers will be built from fractions, and completeness will become a theorem.

So every infinite decimal has a value. The equation $0.(9) = 1$, that is, $0.999\ldots = 1$, from chapter 13 is a special case where the sum was found explicitly.

Room two: the terms melt, the sum grows

Exhibit no. 2 · The harmonic series · Paris, around 1350

The series $1 + \frac12 + \frac13 + \frac14 + \dots$ is called the harmonic series: a string vibrating in halves, thirds, quarters sounds overtones with wavelengths $\frac12, \frac13, \frac14$ of the fundamental. The partial sums $H_n = 1 + \frac12 + \dots + \frac1n$ are called harmonic numbers. The necessary condition holds, and the sums grow ever more slowly: $H_{10} \approx 2.93$, $H_{100} \approx 5.19$, $H_{1000} \approx 7.49$. It looks as if they’re about to hit a ceiling. They won’t.

Nicole Oresme’s trick: gather the terms into groups, each of which is at least a half.

Draw the terms as bars of width $1$: bar number $n$ has height $\frac1n$, and the area of the first $n$ bars is $H_n$. Split the bars into groups: $\frac12$; then $\frac13$ and $\frac14$; then four bars from $\frac15$ to $\frac18$; then eight from $\frac19$ to $\frac1{16}$; each group twice as long as the one before. Group number $k$ ends with the bar $\frac1{2^k}$ and contains $2^{k-1}$ bars. In each group the lowest bar is the last, $\frac1{2^k}$. Cut all the bars of the group down to its height: this can only make the sum smaller, and it becomes $2^{k-1}\cdot\frac1{2^k} = \p2{\frac12}$. So each group adds up to at least a half. Up to index $2^k$ there’s room for the first bar and $k$ whole groups, so $H_{2^k} \ge 1 + \p3{\frac k2}$. For example, by term number $2^{20} = 1{,}048{,}576$ the sum is certainly greater than $11$. The partial sums grow and overtake any number: to exceed $M$, it’s enough to reach index $2^k$ with $k > 2M$. So the harmonic series diverges, although its terms tend to zero.

The sum grows, but how slowly! To get past $10$ takes $12{,}367$ terms. The rate of growth is given away by the area under the hyperbola $y = \frac1x$: from $1$ to $n$ it equals $\ln n$ (chapter 29).

The idea: compare the bars $\frac1k$ with the area under the curve $y = \frac1x$, once putting them on top of the curve and once hiding them under it.

Stand the bar $\frac1k$ on the interval $[k, k+1]$. On this interval the curve is below $\frac1k$, so the bars numbered $1$ to $n$ cover the whole area under the curve from $1$ to $n + 1$ and a bit more: $\p1{H_n > \ln(n + 1)}$. Shift the bars one unit to the left: the bar $\frac1k$ for $k \ge 2$ now stands on the interval $[k - 1, k]$, where the curve is above $\frac1k$. These bars fit entirely under the curve from $1$ to $n$, so $H_n - 1 \le \ln n$, that is, $\p2{H_n \le 1 + \ln n}$. Equality holds only at $n = 1$, when there’s nothing to shift. Back to the first picture. Corners stick out above the curve, curved triangles; their total area is $\p3{D_n = H_n - \ln(n + 1)}$. As $n$ grows, new corners are added, so $D_n$ grows. Slide all the corners left into the first strip, $[1, 2]$. Corner number $k$ lies between the heights $\frac1{k+1}$ and $\frac1k$, so the shifted corners don’t overlap and fit in a $1 \times 1$ square. And they don’t fill the square completely: already the first corner leaves empty a piece of its rectangle of area $\ln 2 - \frac12 \approx 0.19$. So $\p4{D_n < 1.5 - \ln 2 \approx 0.81}$ for every $n$. The sequence $D_n$ grows and is bounded, and by the completeness axiom it has a limit $\gamma$. It’s at least $D_1 = 1 - \ln 2 > 0$ and at most $0.81$. Finally, $H_n - \ln n = D_n + \ln\left(1 + \frac1n\right)$, and the second term tends to zero, so $H_n - \ln n \to \gamma$ as well. The natural logarithm, the main part. It grows without limit, but slowly: to add one, the number of terms has to be multiplied by $e \approx 2.718$. The Euler–Mascheroni constant, $\gamma = 0.5772156649\dots$ Euler found it in the 1730s, but to this day nobody even knows whether it’s rational. A correction that disappears as $n$ grows; in fact it’s close to $\frac1{2n}$. Example: $H_{1000} = 7.48547\dots$, while $\ln 1000 + \gamma = 6.90776 + 0.57722 = 7.48498$; the difference is $0.0005 = \frac1{2 \cdot 1000}$. The sum first exceeds $20$ at term number $272{,}400{,}600$. Adding one term a second, that would take more than eight and a half years.

Room three: a stack of books over the abyss

Exhibit no. 3 · A stack of books · Try this at home

Stack identical books at the edge of a table so that each one sticks out beyond the one below. How far past the edge can the top book go? Intuition says no further than the length of one book. Try it yourself.

Four are enough: the top book sticks out beyond the edge by $\frac{25}{24}$ of its length. And with more books the overhang can be as large as you like.

We’ll build the stack from the top down and keep track of the centres of mass. The stack stands as long as the centre of mass of every top part of it is over the book that part rests on, and the centre of mass of the whole stack is over the table. Let the table edge be the point $0$, $r_k$ the right edge of the $k$th book from the top, and $c_k$ the centre of mass of the top $k$ books.

The top book is uniform, and its centre of mass is in the middle: $r_1 - c_1 = \p1{\frac12}$. Slide book number $k$ under the top $k - 1$ books. They stay put only if their centre of mass is over it: $c_{k-1} \le r_k$. The centre of mass of $k$ books is a weighted average: $c_k = \frac{(k - 1)c_{k-1} + (r_k - \frac12)}{k}$, so $c_{k-1} - c_k = \frac{c_{k-1} - r_k + \frac12}{k} \le \p2{\frac{1}{2k}}$. Each new book underneath shifts the overall centre of mass to the left by at most $\frac1{2k}$, and by exactly that much if its edge stands exactly under the centre of mass of the books above it. Add up the shifts: $r_1 - c_n = \frac12 + (c_1 - c_2) + \dots + (c_{n-1} - c_n) \le \frac12\left(1 + \frac12 + \dots + \frac1n\right) = \p3{\frac12 H_n}$. The table requires $c_n \le 0$, so the overhang is $r_1 \le \frac12 H_n$. Equality is reached when each book has its edge exactly under the centre of mass of the books above it, and the table edge is under the centre of mass of the whole stack. That’s how the “best stack” is built. Since $H_n \to \infty$, the overhang can be arbitrarily large.

Arbitrarily large, but at what cost! An overhang of two book lengths needs $31$ books, three needs $227$, and ten needs those same $272{,}400{,}600$ books that the harmonic series needs to reach twenty. With books $3$ cm thick, such a stack would rise eight thousand kilometres.

How many books are needed for the top one to stick out beyond the table edge by two of its lengths?

$4$ $8$ $31$ $227$

We need $\frac12 H_n \ge 2$, that is, $H_n \ge 4$. The harmonic numbers first pass four at $n = 31$: $H_{30} \approx 3.995$, $H_{31} \approx 4.027$.

The restoration workshop: convergence tests

The museum has a workshop where new arrivals are checked: does the series converge or not? A formula for the partial sums is rarely available, so the workshop’s tools compare an unfamiliar series with a familiar one.

The idea: you can’t build a taller tower from smaller blocks than from bigger ones.

Draw the terms of both series as bars: each bar $\p1{a_n}$ fits inside the bar $b_n$. Stack the bars into two towers, one storey at a time. The heights of the towers are the partial sums $\p2{A_n = a_1 + \dots + a_n}$ and $B_n = b_1 + \dots + b_n$. The terms are non-negative, so both towers only grow, and at every step $A_n \le B_n$. If $\sum b_n$ converges to $B$, the tower $B_n$ is never taller than $B$: a non-decreasing sequence never exceeds its limit. So $A_n \le B_n \le \p3{B}$: the tower $A$ grows and hits a ceiling, and by the completeness axiom its heights converge to a number no larger than $B$. The second half is the same read backwards: if tower $A$ grows without limit, then tower $B$, which is no lower, grows without limit too.

For example, the series $\sum \frac1{n^2}$ converges. For $n \ge 2$ we have $\frac1{n^2} < \frac{1}{n(n - 1)} = \frac{1}{n - 1} - \frac1n$, and the series of right-hand sides has partial sums that collapse like a telescope: $\left(1 - \frac12\right) + \left(\frac12 - \frac13\right) + \dots + \left(\frac1{n-1} - \frac1n\right) = 1 - \frac1n < 1$. So $\sum \frac1{n^2} < 1 + 1 = 2$. Exactly which number it is is the mystery of the next room.

The second tool compares a series with a geometric progression, and the progression itself is chosen to fit the series.

Jean le Rond d’Alembert published this test in 1768. It’s good for series with factorials and powers. For $\sum \frac{n}{2^n}$ the ratio $\frac{n + 1}{2n} \to \frac12$, and the series converges. For $\sum \frac{100^n}{n!}$ the ratio $\frac{100}{n + 1} \to 0$: sooner or later the factorial overtakes any power, and the series converges, while $\sum\frac{n!}{100^n}$ diverges for the same reason.

Find the sum of the series $\frac12 + \frac24 + \frac38 + \frac4{16} + \dots = \sum\limits_{n=1}^{\infty}\frac{n}{2^n}$.

The series converges by the ratio test; call its sum $S$. Subtract from $S$ the series $\frac S2 = \frac14 + \frac28 + \frac3{16} + \dots$, shifted by one place: $S - \frac S2 = \frac12 + \frac14 + \frac18 + \dots = 1$. Convergent series can be subtracted term by term: the partial sums of the difference are the differences of the partial sums. So $\frac S2 = 1$ and $S = 2$.

For the series $\sum\frac1{n^2}$ the ratio test gives $\frac{a_{n+1}}{a_n} = \frac{n^2}{(n+1)^2} \to 1$. What follows from this?

The series converges The series diverges Nothing: another test is needed

At $q = 1$ the test is silent. The convergence of $\sum\frac1{n^2}$ was proved by the comparison test, and for the harmonic series, which diverges, the ratio also tends to $1$.

Room four: π without circles

Exhibit no. 4 · The Basel problem · St Petersburg, 1734

The series $1 + \frac14 + \frac19 + \dots$ converges, and its sum is less than $2$. But which number is it? The question was asked in 1650 by Pietro Mengoli and made famous by Jacob Bernoulli of Basel, who admitted that he couldn’t crack it. Adding directly hardly helps: the tail after the $N$th term is roughly $\frac1N$, and each new correct digit costs ten times as much work.

The problem was solved in 1734 by Leonhard Euler, who wasn’t yet thirty. He was working at the St Petersburg Academy of Sciences and presented the solution to it in December 1735. The answer astonished everyone.

The squares of the natural numbers. No circles, no angles: pure arithmetic. The number $\pi$ has come from the zeros of the sine: $\sin x = 0$ at $x = \pm\pi, \pm2\pi, \dots$ The six is the $3!$ from the series for the sine, $\sin x = x - \frac{x^3}{6} + \dots$ Example: $\frac{\pi^2}{6} = 1.6449340668\dots$, while a thousand terms give $1.6439345667$, less by $0.0009995 \approx \frac{1}{1000}$, just as the estimate of the tail promises.

Euler’s reasoning was bold. A polynomial that equals $1$ at zero and has roots $r_1, \dots, r_k$ factors into the product $\left(1 - \frac{x}{r_1}\right)\cdots\left(1 - \frac{x}{r_k}\right)$ (chapter 14). The function $\frac{\sin x}{x}$ tends to $1$ at zero and vanishes at the points $\pm\pi, \pm2\pi, \dots$ Euler supposed that it factors in the same way, only with infinitely many factors; each pair of roots $\pm k\pi$ gives the factor $\left(1 - \frac{x}{k\pi}\right)\left(1 + \frac{x}{k\pi}\right) = 1 - \frac{x^2}{k^2\pi^2}$:

$$\frac{\sin x}{x} = \left(1 - \frac{x^2}{\pi^2}\right)\left(1 - \frac{x^2}{4\pi^2}\right)\left(1 - \frac{x^2}{9\pi^2}\right)\cdots$$

If you multiply out the brackets, the $x^2$ term is collected from all the factors: $-\left(\frac1{\pi^2} + \frac1{4\pi^2} + \frac1{9\pi^2} + \dots\right)x^2$. And by the series for the sine, which we’ll derive in room six, $\frac{\sin x}{x} = 1 - \frac{x^2}{6} + \frac{x^4}{120} - \dots$ Equating the coefficients of $x^2$, Euler got $\sum\frac1{n^2\pi^2} = \frac16$.

The boldness was no joke: an infinite product isn’t a polynomial, and having the same roots guarantees nothing yet. The function $e^x\frac{\sin x}{x}$ has the same roots but factors differently. Euler checked the answer numerically, and his factorisation of the sine was only justified rigorously in the nineteenth century. Here’s a proof that needs only trigonometry, complex numbers and Vieta’s formulas; it’s attributed to Augustin-Louis Cauchy.

The idea: squeeze $\frac1{x^2}$ between two functions whose sums at specially chosen points can be computed exactly.

For $0 < x < \frac\pi2$ we have $\sin x < x < \tan x$: in the unit circle the triangle inside a sector is smaller than the sector, and the sector is smaller than the triangle drawn around it (chapter on limits). All three numbers are positive. Take reciprocals and square: $\cot^2 x < \frac1{x^2} < \frac1{\sin^2 x} = 1 + \cot^2 x$. In the picture $\frac1{x^2}$ is squeezed between two curves. Choose a natural number $m$ and the points $x_k = \frac{k\pi}{2m + 1}$ for $k = 1, \dots, m$; they all lie between $0$ and $\frac\pi2$. Add up the inequalities at these points. The middle sum is $\frac{(2m + 1)^2}{\pi^2}\left(1 + \frac14 + \dots + \frac1{m^2}\right) = \frac{(2m+1)^2}{\pi^2}S_m$, where $S_m$ is a partial sum of our series. Writing $A_m = \cot^2 x_1 + \dots + \cot^2 x_m$, we get $\p2{A_m < \frac{(2m + 1)^2}{\pi^2}S_m < A_m + m}$. Let’s find $A_m$ exactly. By de Moivre’s formula (chapter 15) $\cos nx + i\sin nx = (\cos x + i\sin x)^n$. Take $n = 2m + 1$, expand the right side by the binomial theorem (chapter 45) and compare imaginary parts: $\sin nx = \binom n1\cos^{n-1}x\,\sin x - \binom n3\cos^{n-3}x\,\sin^3x + \dots$ Divide by $\sin^n x$: $\frac{\sin nx}{\sin^n x} = \binom n1 t^m - \binom n3 t^{m-1} + \binom n5 t^{m-2} - \dots$, where $t = \cot^2 x$. On the right is a polynomial of degree $m$ in $t$. At the points $x_k$ the left side vanishes: $\sin nx_k = \sin k\pi = 0$. So the numbers $\cot^2 x_1, \dots, \cot^2 x_m$ are roots of this polynomial. They’re distinct, because the cotangent is decreasing and positive on $\left(0, \frac\pi2\right)$, and a polynomial of degree $m$ has at most $m$ roots, so these are all its roots. By Vieta’s formulas (chapter 14) their sum is $A_m = \binom n3\Big/\binom n1 = \frac{2m(2m - 1)}{6} = \p4{\frac{m(2m - 1)}{3}}$. Substitute $A_m$ into the inequality of step 2 and multiply by $\frac{\pi^2}{(2m + 1)^2}$: $\frac{\pi^2}{6}\cdot\frac{2m(2m - 1)}{(2m + 1)^2} < S_m < \frac{\pi^2}{6}\cdot\frac{2m(2m + 2)}{(2m + 1)^2}$. Both fractions next to $\frac{\pi^2}{6}$ tend to $1$: numerator and denominator are $4m^2$ plus terms of lower degree. The partial sums are squeezed between quantities that tend to $\frac{\pi^2}{6}$, so they tend there too.

Room five: a sum that depends on the order

Exhibit no. 5 · The alternating harmonic series · Göttingen, 1854

Flip the sign of every other term in the harmonic series: $1 - \frac12 + \frac13 - \frac14 + \dots$ The partial sums zigzag: $1$, $0.5$, $0.833$, $0.583$, $0.783$, …; the steps get ever shorter, and the zigzag closes in on the number $0.6931\dots$ Such series are guaranteed to converge by a simple test.

The idea: the partial sums jump right and left, each jump no longer than the one before, so they close in on the sum ever more tightly.

On the number line the partial sums zigzag: a step right by $a_1$, left by $a_2$, right by $a_3$… Each step is no longer than the one before. The even sums don’t decrease: $S_{2k+2} = S_{2k} + (a_{2k+1} - a_{2k+2}) \ge S_{2k}$, since the bracket is non-negative. The odd ones, for the same reason, don’t increase: $S_{2k+1} = S_{2k-1} - (a_{2k} - a_{2k+1}) \le S_{2k-1}$. Every even sum $S_{2k}$ is at most every odd one $S_{2j+1}$: taking $l \ge k$ and $l \ge j$, we get $S_{2k} \le S_{2l} \le S_{2l} + a_{2l+1} = S_{2l+1} \le S_{2j+1}$. In particular, all the even sums are at most $S_1$: they grow and are bounded, and by the completeness axiom they tend to some $S$. The odd sums $S_{2k+1} = S_{2k} + a_{2k+1}$ tend to $S + 0 = S$. The growing even sums are at most their limit $S$, and the decreasing odd ones at least. So $S$ lies between any two neighbouring partial sums $S_n$ and $S_{n+1}$, and the distance between them is $a_{n+1}$. Hence $|S - S_n| \le a_{n+1}$.

The test doesn’t say what the sum is. It’s found by a trick that also works for the number $\pi$.

Now the exhibit itself. Take the same terms, but after each positive one put two negative ones:

$$1 - \frac12 - \frac14 + \frac13 - \frac16 - \frac18 + \frac15 - \frac1{10} - \frac1{12} + \dots$$

Not a single term is lost. Group them in threes: $\left(1 - \frac12\right) - \frac14 + \left(\frac13 - \frac16\right) - \frac18 + \dots = \frac12 - \frac14 + \frac16 - \dots$, exactly half the original series: each triple $\frac1{2j-1} - \frac1{4j-2} - \frac1{4j}$ equals $\frac12\left(\frac1{2j-1} - \frac1{2j}\right)$. After $3k$ terms the partial sum of the new series is half the $2k$th partial sum of the old one, and the sums in between differ from it by vanishingly small terms. The new series converges to $\frac12\ln 2$. The same numbers in a different order, and the sum is halved. The trouble is that the series converges only because the pluses and minuses cancel.

The series $1 - \frac12 + \frac13 - \dots$ converges conditionally: its series of absolute values is the harmonic series. And for such series the order of the terms decides everything.

The idea: the supplies of positive and of negative terms are infinite, and the terms themselves get smaller. So you can close in on any target by overshooting it with ever smaller swings.

Split the terms into the positive ones $p_1, p_2, \dots$ and the negative ones $-q_1, -q_2, \dots$ (in their original order). Both series $\sum p_k$ and $\sum q_k$ diverge. If both converged, the original series would converge absolutely. If only one converged, the partial sums of the original series, differences of sums of $p$’s and $q$’s, would run off to infinity. And $p_k \to 0$ and $q_k \to 0$ by the necessary condition. The greedy recipe. Take positive terms in order until the sum exceeds $L$: this will happen, because the supply of positives is infinite. Then take negative terms until the sum drops below $L$. Then positive ones again, and so on. Each stage uses up at least one term, so sooner or later every term is used, and exactly once. At the moment the sum overshoots $L$, it’s no further from $L$ than the last term taken, and until the next overshoot it stays on the same side and no further from $L$. The terms tend to zero, so the size of the swings around $L$ does too. The new series converges to $L$. To make the series diverge, keep moving the target: first $1$, then $2$, then $3$… Then the partial sums run off to infinity.

Bernhard Riemann proved this theorem in his 1854 paper on trigonometric series, published after his death. With absolutely convergent series the trick doesn’t work, as Dirichlet noticed back in 1837.

The terms of the series $1 - \frac14 + \frac19 - \frac1{16} + \dots$ have been rearranged. Can the sum change?

Yes, by Riemann’s theorem No: the series converges absolutely Yes, if infinitely many terms are rearranged

The series of absolute values $\sum\frac1{n^2}$ converges, and by Dirichlet’s theorem every rearrangement gives the same sum, $\frac{\pi^2}{12}$. You can see this in Riemann’s machine: the positive terms add up to only $\frac{\pi^2}{8} \approx 1.23$, and they can’t reach the target $2$.

Room six: the polynomial double

Exhibit no. 6 · The Taylor series · London, 1715

Back to the question we started with. Suppose a function is written as an “infinite polynomial” centred at $a$: $f(x) = c_0 + c_1(x - a) + c_2(x - a)^2 + \dots$ At $x = a$ all the terms but the first vanish, and $c_0 = f(a)$. Differentiate: $f'(x) = c_1 + 2c_2(x - a) + \dots$, so $c_1 = f'(a)$. After $k$ differentiations, what’s left of $c_k(x - a)^k$ is $k!\,c_k$, so $c_k = \frac{f^{(k)}(a)}{k!}$: the coefficients are dictated by the derivatives at a single point. Here we’ve allowed ourselves to differentiate term by term without proof (inside the interval of convergence it’s legitimate, chapter 53), but the definition below doesn’t rely on it.

The derivative of order $k$ at the centre $a$. All the series knows about the function is its value, slope, bend and so on at a single point. The factorial cancels the factor $k!$ that appears when $(x - a)^k$ is differentiated $k$ times. Powers of the distance from the centre. Near $a$ they’re tiny, so the first terms decide almost everything. Example: all the derivatives of $e^x$ equal $e^x$, and at zero they equal one, so $e^x = \sum\frac{x^k}{k!}$. At $x = 1$ the terms with $k \le 10$ give $2.71828180$, while $e = 2.71828183\dots$: an error of about $3 \cdot 10^{-8}$.

The derivatives of the sine go round in a circle: $\sin$, $\cos$, $-\sin$, $-\cos$, $\sin$, …, which at zero gives $0, 1, 0, -1, 0, \dots$ So only odd powers with alternating signs are left in the series for the sine. Here are the room’s main exhibits; next to each is where the series converges to the function.

The fifth row is the binomial theorem for any exponent, promised in chapter 45. For natural $\alpha$ the coefficients with index greater than $\alpha$ become zero, and the series breaks off, turning into the ordinary binomial expansion. For $\alpha = \frac12$ it never breaks off: $\sqrt{1 + x} = 1 + \frac x2 - \frac{x^2}{8} + \frac{x^3}{16} - \dots$ Newton found this series in 1665, and the general method was described by Brook Taylor in a book of 1715.

A Taylor polynomial is an approximation, and we’d like to know its price: how wrong it is.

The estimate can be almost exact. For $\sin 1$ the polynomial $x - \frac{x^3}{6} + \frac{x^5}{120}$ gives $0.841667$, while in fact $\sin 1 = 0.841471$. There’s no $x^6$ term in the series for the sine, so this polynomial is also $T_6$, and the estimate with $n = 6$ and $M = 1$ promises an error of at most $\frac{1}{7!} = \frac1{5040} \approx 0.000198$. The real error is $0.000196$.

Find the coefficient of $x^6$ in the Taylor series of $e^{-x^2}$ at zero.

Substitute $u = -x^2$ into the series for the exponential: $e^{u} = 1 + u + \frac{u^2}{2} + \frac{u^3}{6} + \dots$ The power $x^6$ comes only from the term $\frac{u^3}{6} = \frac{-x^6}{6}$. The coefficient is $-\frac16$. By the definition we’d have to differentiate $e^{-x^2}$ six times, find $f^{(6)}(0) = -120$, and then divide by $6! = 720$: the same answer.

Room seven: a boundary you can’t see

Exhibit no. 7 · The function $\frac{1}{1 + x^2}$ · an unsolved riddle

The series $1 + x + x^2 + \dots = \frac{1}{1 - x}$ holds only for $|x| < 1$, and that’s understandable: at $x = 1$ the function runs off to infinity. For $\ln(1 + x)$ the trouble is at $x = -1$, and the series converges only up to there. But $\frac1{1 + x^2}$ is a smooth function on the whole line, never more than one. Its series $1 - x^2 + x^4 - \dots$ (geometric, with $-x^2$ in place of $x$) also converges only for $|x| < 1$. At $x = 2$ the function is a calm $\frac15$, while the series $1 - 4 + 16 - \dots$ flies to pieces.

Every power series has a radius of convergence (it may be zero or infinity), but in general this needs the Cauchy–Hadamard formula. For the series we’ll meet, the ratio test is enough.

The coefficient of $(x - a)^k$. The next coefficient. If the coefficients decrease quickly, the radius is large; if slowly, it’s small. Examples: for $e^x$ the ratio is $\frac{(k + 1)!}{k!} = k + 1 \to \infty$, and the radius is infinite. For $\ln(1 + x)$ the coefficients are $\pm\frac1k$, the ratio $\frac{k + 1}{k} \to 1$, and $R = 1$. If powers are missing from the series, as in $1 - x^2 + x^4 - \dots$, the ratio test is applied directly to the terms: the ratio of neighbours is $x^2$, and the series converges for $x^2 < 1$.

On the boundary $|x - a| = R$ the theorem promises nothing, and things vary there. The series for $\ln(1 + x)$ at $x = 1$ becomes the alternating harmonic series and converges to $\ln 2$, while at $x = -1$ it becomes the harmonic series with a minus sign and diverges.

For which $x$ does the series $x - \frac{x^2}{2} + \frac{x^3}{3} - \frac{x^4}{4} + \dots$ converge?

For all $x$ For $-1 < x \le 1$ For $|x| < 1$ For $x \ge 0$

The radius of convergence is $1$. On the boundary: at $x = 1$ we get $1 - \frac12 + \frac13 - \dots = \ln 2$, and the series converges by the alternating series test; at $x = -1$ we get the series $-\left(1 + \frac12 + \frac13 + \dots\right)$, which diverges.

Back to the riddle. Why does the series for $\frac1{1+x^2}$ trip up at $x = \pm1$, where nothing happens to the function? Take the centre not at zero but at a point $a$. Then the radius of convergence turns out to be $\sqrt{1 + a^2}$: about $1.41$ at $a = 1$, about $3.16$ at $a = 3$. It looks like the distance from the point $a$ to something standing one unit off the line. There’s no such point on the number line. Where it’s hiding will be revealed in chapter 34.

The last room: π drop by drop

Exhibit no. 8 · A series for π · India, around 1400; Europe, 1670s

The series for the arctangent, $x - \frac{x^3}{3} + \frac{x^5}{5} - \dots$, has radius of convergence $1$, and on the boundary itself, at $x = 1$, it still converges; in room five we proved that it converges to $\frac\pi4$:

$$\frac\pi4 = 1 - \frac13 + \frac15 - \frac17 + \dots$$

In Europe the series for the arctangent was found in 1671 by James Gregory, and this special case by Leibniz in 1673–1674. In India it was known two and a half centuries earlier: the series is attributed to Madhava of Sangamagrama, the founder of the Kerala school.

The formula is beautiful, but computing with it is torture. By the alternating series test the error after $n$ terms is at most the next term, so after multiplying by $4$ it’s at most $\frac{4}{2n + 1}$, and in fact almost exactly $\frac1n$. A thousand terms give $3.14059265383979\dots$, with $\pi = 3.14159265358979\dots$ Surprisingly, just one digit is wrong, and the next six agree again. The reason is that the error differs from $\frac1{1000}$ by less than a billionth. Why this happens was explained in 1989 by Jonathan and Peter Borwein together with Karl Dilcher: the corrections to the error are expressed through the so-called Euler numbers.

What speeds up the computation is the arctangent series with an argument less than one: then the terms decrease like a geometric progression. Madhava, judging by the records of his followers, took $\arctan\frac1{\sqrt3} = \frac\pi6$ and got $\pi = \sqrt{12}\left(1 - \frac{1}{3 \cdot 3} + \frac{1}{5 \cdot 3^2} - \frac{1}{7 \cdot 3^3} + \dots\right)$: twenty terms of this series give ten correct decimal places. And in 1706 John Machin found an identity with which he computed a hundred digits.

Formulas of this kind remained the record-breakers’ main tool for more than two hundred years, right up to the first computer calculations of $\pi$ in 1949. And the 1706 book by William Jones in which Machin’s hundred digits appeared is famous for something else too: in it the ratio of a circle’s circumference to its diameter was first denoted by the letter $\pi$.

Before leaving the museum, consolidate the exhibits in the trainer.

Where next

We’ve been through the cabinet of curiosities, and the question from the last chapter has its answer: many functions really can be written as “infinite polynomials”, but only within their radius of convergence and only if the remainder tends to zero. In the Taylor laboratory you could see that the approximation $\sin x \approx x$ works excellently for small $x$. It’ll come in handy straight away.

The laws of nature rarely say how much of something there will be. They say how fast it changes. A cup of coffee cools the faster the hotter it is. A pendulum accelerates the harder the further it’s displaced. An epidemic grows the faster the more people are ill and the more there are who haven’t been ill yet. In each case the unknown function stands both on the right-hand side and under the derivative sign, and integrating the rate, as in chapter 28, doesn’t work. How do you find the process itself from the law by which it changes? Newton, incidentally, solved such problems with series: he substituted an infinite polynomial and found the coefficients one after another. That’s where the chapter on differential equations begins.