← The Fourth Dimension
Contents
  1. 01 What a dimension is
  2. 02 Flatland
  3. 03 Projections
  4. 04 Rotation
  5. 05 Cross-sections
  6. 06 Regular polytopes
  7. 07 The hypersphere
  8. 08 Many dimensions
  9. 09 Where dimensions live
  10. 10 History
  11. 11 How the showcase is drawn
RU

Chapter 08 · 17 min

The strange geometry of many dimensions

In a thousand dimensions almost all of a ball’s volume sits in a thin peel, random directions are almost perpendicular, and all points are nearly equally far apart. This is where statistics and machine learning actually live.

So far we have climbed one step at a time, from three dimensions to four. Now let us jump far ahead: $n = 10$, $100$, $1000$. There is nothing exotic about it. Any table with a thousand columns already describes points of a thousand-dimensional space, and statistics and neural networks work in exactly such spaces. But the geometry there is arranged so that intuition grown in three dimensions gets almost everything wrong. The good news is that the formulas keep working, and every “miracle” in this chapter takes a few lines to prove.

An orange that is all peel

Start with a simple question. Take an orange of radius 5 cm with peel 5 mm thick. What fraction of its volume is peel? The peel is the layer between radii $4.5$ and $5$ cm, and the volume of a ball is proportional to the cube of its radius. The flesh is $(4.5/5)^3 = 0.9^3 = 0.729$ of the whole, so the peel takes $27.1\%$. Now imagine the same orange in $n$ dimensions.

Theorem

The fraction of the volume of an $n$-dimensional ball of radius $R$ that lies within $\varepsilon R$ of its surface is $$1 - (1-\varepsilon)^n \;\ge\; 1 - e^{-\varepsilon n}.$$ For any fixed $\varepsilon > 0$ it tends to one as $n$ grows.

Stretch a shape in $n$-dimensional space by a factor $k$: every coordinate is multiplied by $k$, and the volume by $k^n$ — that is what happens to each of the little cubes the shape can be built from. Hence $V_n(r) = r^n\,V_n(1)$. The layer at the surface is the ball of radius $R$ minus the ball of radius $(1-\varepsilon)R$, so its share is $1 - (1-\varepsilon)^n$. The inequality follows from $1 - \varepsilon \le e^{-\varepsilon}$: the curve $e^{-x}$ is convex and lies above its tangent line $1 - x$ at $0$. ∎

Here is what that looks like in numbers:

dimensionpeel 1/10 of the radiuspeel 1/100 of the radius
327.1%3.0%
1065.1%9.6%
3095.8%26.0%
10099.997%63.4%
1000$100\% - 1.7\cdot 10^{-44}\%$99.996%

A hundred-dimensional orange is practically all peel, and a thousand-dimensional one keeps nearly all of its volume even in a layer one hundredth of the radius thick. For a point chosen uniformly at random in the ball, this means it almost certainly sits right at the surface. Its average distance from the centre is $n/(n+1)$ of the radius — $0.999$ for $n = 1000$.

Recall also that the ball itself “slims down” in high dimensions. The volume of the unit ball grows up to $n = 5$ (where it is about $5.26$) and then tends to zero, as the chapter on the hypersphere explains. The ball inscribed in the unit cube fills about $0.25\%$ of the cube in ten dimensions, and less than $10^{-69}$ of it in a hundred.

Try it

Set the peel thickness to $0.01$ and drag the dimension to the right: around $n = 69$ half of the volume has moved into a layer one percent of the radius thick. The dots are genuine random points of the $n$-dimensional ball. Each is drawn at its own distance from the centre and its own height $x_1$, so the picture is flat but honest.

…and all of it at the equator

Now a stronger oddity. Draw an “equator” through the centre of the ball — the hyperplane $x_1 = 0$ — and ask what fraction of the volume lies in the thin band $|x_1| < \delta$ around it.

Theorem

If $x$ is chosen uniformly in the unit $n$-dimensional ball, then $\mathbb{E}[x_1^2] = \dfrac{1}{n+2}$, and therefore $$P\bigl(|x_1| \ge \delta\bigr) \le \frac{1}{(n+2)\,\delta^2}.$$

First find $\mathbb{E}|x|^2$. The part of the ball inside radius $r$ is $r^n$ of the whole (the previous theorem), so $|x|$ has density $n r^{n-1}$ and $\mathbb{E}|x|^2 = \int_0^1 r^2 \cdot n r^{n-1}\,dr = \frac{n}{n+2}$. The ball does not change when the axes are permuted, so all the $\mathbb{E}[x_i^2]$ are equal, and they add up to $\mathbb{E}|x|^2$. Hence $\mathbb{E}[x_1^2] = \frac{1}{n+2}$. The inequality is Markov’s inequality for the non-negative quantity $x_1^2$: $P(x_1^2 \ge \delta^2) \le \mathbb{E}[x_1^2]/\delta^2$. ∎

The bound is crude; the true tail falls off like $e^{-n\delta^2/2}$. In a thousand-dimensional ball the band $|x_1| < 0.1$ holds $99.85\%$ of the volume. But the equator can be drawn perpendicular to any direction. So almost the entire ball lies at its surface and, at the same time, near every one of its equators.

There is no contradiction. A typical point of the ball has length close to 1, but each of its coordinates is about $\pm 1/\sqrt{n}$: the length is spread over a thousand coordinates, and none of them is ever large. What ends up near the equator is not some special set of points but almost all of them. This is called concentration of measure. Paul Lévy described it on the sphere in the 1920s, and in the 1970s Vitali Milman turned it into a working tool of high-dimensional geometry.

Try it

Switch the widget above to “Equator”. For $n = 3$ the band $|x_1| < 0.1$ holds 15% of the volume, for $n = 100$ already 69%, for $n = 1000$ nearly all of it. The dots crowd towards the rim and towards the horizontal line at once.

The spiky cube

The cube shows the same thing even more vividly. The unit cube $[0,1]^n$ has a diagonal of length $\sqrt{n}$: $3.16$ in ten dimensions, 10 in a hundred. The angle between the diagonal and an edge comes from $\cos\varphi = 1/\sqrt{n}$: $54.7°$ in 3D but $84.3°$ in 100D, so the diagonal is almost perpendicular to every edge at once. From the centre it is $1/2$ to a face and $\sqrt{n}/2$ to a corner, and there are $2^n$ corners. A typical point of the cube, meanwhile, is about $\sqrt{n/12}$ from the centre — $2.9$ in 100D, far outside the inscribed ball of radius $1/2$ and far from the corners. A high-dimensional cube looks less like a box than like a hedgehog, with narrow spikes running out to the corners.

The most striking illustration is the problem of the balls in the corners. Take the cube $[-2, 2]^n$ and put a unit ball in each corner, centred at $(\pm 1, \pm 1, \dots, \pm 1)$. There are $2^n$ of them, and each touches $n$ faces of the cube and $n$ neighbouring balls. In the middle, put the ball that touches them all.

Theorem

The central ball has radius $\sqrt{n} - 1$. For $n \le 8$ it lies strictly inside the cube, for $n = 9$ it touches the faces, and for $n \ge 10$ it pokes out of the cube, although it is still “squeezed” between the corner balls.

The centre of the corner ball $(1, 1, \dots, 1)$ is at distance $\sqrt{1 + 1 + \dots + 1} = \sqrt{n}$ from the origin. The central ball touches it when the radii add up to the distance between the centres: $r + 1 = \sqrt{n}$, so $r = \sqrt{n} - 1$. By symmetry the same holds for all $2^n$ corner balls. The points of the cube’s boundary closest to the centre are the centres of the faces, such as $(2, 0, \dots, 0)$, at distance 2. The central ball stays inside the cube as long as $\sqrt{n} - 1 \le 2$, that is $n \le 9$, with equality at $n = 9$. At $n = 10$ the radius is $\sqrt{10} - 1 \approx 2.16 > 2$. ∎

There are other curiosities along the way. In four dimensions $r = \sqrt{4} - 1 = 1$: the central ball is exactly as big as the corner ones. And if you compare volumes, a computation shows that from $n = 1206$ on, the central ball — the one that was supposed to huddle in the gap between the corner balls — has more volume than the whole cube $[-2,2]^n$.

How do you draw this for $n > 3$? You can honestly cut the figure with a plane. The “Slice” mode shows the plane through the centre that contains the $x_1$ axis and the diagonal direction $(0, 1, \dots, 1)$. In it the cube is a $4 \times 4\sqrt{n-1}$ rectangle, stretched along the diagonal. Exactly four corner balls have their centres in this plane (those with $x_2 = \dots = x_n$) and cut it in unit discs; all the others miss it, the nearest being $2\sqrt{(n-2)/(n-1)} > 1$ away. The central ball cuts a disc of radius $\sqrt{n} - 1$, and for $n \ge 10$ you can see it bulge out through the long sides of the rectangle.

Try it

At $n = 3$ turn the cube: the pink ball in the middle barely shows between the eight blue ones. Then switch to “Slice” and drag the dimension: at $n = 4$ all the discs are equal, at $n = 9$ the pink one touches the sides, at $n = 10$ it breaks out.

Everything is nearly perpendicular

In three dimensions, randomly chosen directions make all sorts of angles. In high dimensions, almost any two random directions are almost perpendicular.

Theorem

Let $u$ and $v$ be independent random unit vectors in $\mathbb{R}^n$, uniformly distributed on the sphere, and let $\theta$ be the angle between them. Then $$\mathbb{E}[\cos\theta] = 0,\quad \mathbb{E}[\cos^2\theta] = \frac{1}{n}.$$ So $\cos\theta$ typically deviates from zero by about $1/\sqrt{n}$, and the angle is close to $90°$ with a spread of about $1/\sqrt{n}$ radians, or $57.3°/\sqrt{n}$.

The distribution of $v$ does not change under rotations, so we may rotate everything until $u$ becomes $e_1 = (1, 0, \dots, 0)$. Then $\cos\theta = \langle u, v\rangle = v_1$. Replacing $v$ by $-v$ does not change the distribution, so $\mathbb{E}[v_1] = 0$. Next, $v_1^2 + v_2^2 + \dots + v_n^2 = 1$ always, and by symmetry all the $\mathbb{E}[v_i^2]$ are equal, so each of them is $1/n$. Finally, for small $\cos\theta$ the angle is $\theta \approx \pi/2 - \cos\theta$, so the spread of the angle is about the spread of the cosine. ∎

For $n = 1000$ that is $90° \pm 1.8°$. As with the equator, the tails are Gaussian: $P(|\cos\theta| \ge \varepsilon) \le 2e^{-n\varepsilon^2/2}$. This has a surprising consequence. $\mathbb{R}^n$ has at most $n$ exactly perpendicular directions, but you can pick exponentially many (in $n$) nearly perpendicular ones, with $|\cos\theta| < \varepsilon$ — just choose them at random.

Distances behave the same way. Take two random points $x, y$ of the unit cube. For each coordinate $\mathbb{E}(x_i - y_i)^2 = \operatorname{Var}x_i + \operatorname{Var}y_i$, and a variable spread uniformly over $[0, 1]$ has variance $1/12$. So $\mathbb{E}(x_i - y_i)^2 = 1/6$ and $$\mathbb{E}\,|x - y|^2 = \frac{n}{6}.$$ Carry the calculation on to fourth moments and you find that $|x-y|^2$ has variance $7n/180$, and the distance itself is $\sqrt{n/6}$ give or take about $0.24$ — whatever $n$ is. In a thousand-dimensional cube two random points are almost always about $12.9 \pm 0.24$ apart: all points are nearly equally far from one another.

On the left are angles between random vectors. The dashed line is the exact density, proportional to $\sin^{n-2}\theta$: flat for $n = 2$, a sine arch for $n = 3$, then an ever narrower peak. On the right are distances divided by $\sqrt{n}$, peaking at $1/\sqrt{6} \approx 0.408$. Below them is the ratio $(\max - \min)/\min$ over the distances from one random point to two hundred others. In the plane it runs to tens: the nearest neighbour is much closer than the farthest. In a thousand dimensions it is about $0.1$: the nearest and the farthest differ by ten percent.

The curse of dimensionality

Richard Bellman coined the phrase “curse of dimensionality” in his book Dynamic Programming (1957). He meant the most direct consequence: a grid with 10 points along each axis has $10^n$ points. For $n = 10$ that is ten billion; for $n = 100$ it is more than the number of atoms in the observable universe (usually estimated at around $10^{80}$). Trying every option, tabulating a function, filling the space with examples — none of this is possible in high dimensions.

The second consequence is subtler: the very notion of a “neighbourhood” disappears. Suppose the data are spread uniformly over the unit cube, and we want a small cube around a point that catches 1% of the data. Its edge must be $0.01^{1/n}$: $0.1$ in the plane, already $0.63$ in 10D, $0.955$ in 100D. A “local” neighbourhood covers almost the whole range of every coordinate. And since all distances are nearly equal, the “nearest neighbour” loses its meaning too. Beyer, Goldstein, Ramakrishnan and Shaft proved this in 1999: if the relative spread of the distances tends to zero, then $(\max - \min)/\min \to 0$.

Why does machine learning work at all, then? Because real data are not spread uniformly through space. Photos of faces, texts and sounds occupy a minuscule part of their space and usually cluster near sets of far lower dimension. Everything above is about “typical” points, and real data are atypical — that is exactly what makes them valuable.

The Johnson–Lindenstrauss lemma

Concentration has a useful side too. Since the length of a random projection hardly fluctuates, a cloud of points can be squeezed into a space of much lower dimension with almost no distortion of distances. William B. Johnson and Joram Lindenstrauss proved this in 1984.

Theorem

Let $0 < \varepsilon < 1$ and let $N$ points in $\mathbb{R}^n$ be given. If $$k \ge \frac{4\ln N}{\varepsilon^2/2 - \varepsilon^3/3},$$ then there is a linear map $f\colon \mathbb{R}^n \to \mathbb{R}^k$ such that for any two of the points $u, v$ the distance $d = |u - v|$ and the distance $d' = |f(u) - f(v)|$ between their images satisfy $$(1-\varepsilon)\,d^2 \le d'^2 \le (1+\varepsilon)\,d^2 .$$

Take a random $k \times n$ matrix $A$ with independent normal entries of variance $1/k$. For one fixed vector $x$, the quantity $|Ax|^2/|x|^2$ is the average of $k$ independent squared standard normal variables. Its expectation is 1, and the probability that it is off by more than $\varepsilon$ is at most $2e^{-k(\varepsilon^2/2 - \varepsilon^3/3)/2}$; with $k$ as above this is at most $2/N^2$. There are fewer than $N^2/2$ pairs of points, so the probability that any pair is distorted is less than one. Hence a suitable matrix exists, and a random one works with positive probability. This short version of the proof is due to Dasgupta and Gupta (2003). ∎

The main thing about the formula is what it lacks: $k$ does not depend on the original dimension $n$. A million points from a space of dimension $10^9$ fit into a few thousand dimensions: with $\varepsilon = 0.2$ the formula gives $k \approx 3200$, with $\varepsilon = 0.1$ about $11\,800$. The constants are conservative, but the order $\varepsilon^{-2}\log N$ cannot be improved: Larsen and Nelson proved it optimal in 2017. Random projections built this way are used to speed up similarity search and in numerical linear algebra.

Embeddings: meaning as a direction

Today the most widespread use of high-dimensional geometry is embeddings — vector representations of words, texts and images. A model turns an object into a list of hundreds or thousands of numbers, and similarity between objects becomes geometry. Some concrete sizes: the word2vec model trained at Google on news text (Mikolov and colleagues, 2013) gives 300-dimensional vectors for three million words and phrases; the hidden vectors of BERT-base have 768 dimensions and those of BERT-large 1024; current text-search embedding models output, for example, 1536 or 3072 numbers.

Similarity is usually measured by the cosine of the angle: $$\cos\theta = \frac{\langle a, b\rangle}{|a|\,|b|}.$$ The length of a vector often reflects side properties (such as how frequent a word is), while the direction carries the meaning. This is where everything above comes in. Two random directions in 768 dimensions have a cosine of about $\pm 1/\sqrt{768} \approx \pm 0.036$, so a cosine of $0.5$ is far from chance. Mikolov’s famous “king − man + woman ≈ queen” is about directions too: a difference of vectors encodes a relation.

Two caveats, to avoid overselling the picture. First, real embeddings are not isotropic: in many models the vectors occupy a narrow cone (shown, among others, by Ethayarajh in 2019), so even unrelated texts can have a noticeably positive cosine. Absolute thresholds like “similarity above 0.8” do not transfer from one model to another; it is safer to compare which candidate is closer. Second, because distances concentrate, exact nearest-neighbour search is hard to speed up: search trees that work beautifully in the plane degenerate into near brute force in hundreds of dimensions. Vector databases therefore use approximate search — HNSW graphs (Malkov and Yashunin, 2016), for example — trading a little accuracy for speed.

Sphere packing and kissing numbers

The kissing number is the largest number of equal non-overlapping balls that can all touch a central ball of the same size. On a line it is 2, in the plane 6: six coins around a seventh. In three dimensions 12 balls fit around one easily, with so much room to spare that they can be moved around. Might a thirteenth squeeze in? In 1694 in Cambridge, Isaac Newton and the astronomer David Gregory discussed the question. Tradition has it that Newton said 12 and Gregory 13, though Gregory’s note can be read more than one way. The first rigorous proof that the answer is 12 came only in 1953, from Kurt Schütte and Bartel van der Waerden; John Leech published a short sketch in 1956.

In four dimensions the answer is 24: the balls sit at the vertices of the 24-cell (you can spin it in the showcase). Oleg Musin proved it; the preprint appeared in 2003 and the paper in the Annals of Mathematics in 2008. Then comes a miracle. In eight dimensions the kissing number is 240, in twenty-four it is 196,560; both values were proved in 1979, independently, by Andrew Odlyzko with Neil Sloane and by Vladimir Levenshtein. The arrangements there are unique and come from exceptional lattices: $E_8$ and the Leech lattice. In five dimensions, meanwhile, the answer is still unknown — somewhere between 40 and 44. Kissing numbers are known exactly only in dimensions 1, 2, 3, 4, 8 and 24. Lower bounds in other dimensions keep improving: in 2024 Henry Cohn and Anqi Li made progress in dimensions 17–21, and in 2025–2026 AI-assisted programs joined the search.

A closely related question is the densest packing: what is the largest fraction of space that equal non-overlapping balls can fill?

dimensionkissing numberdensest packingdensity
26hexagonal$\pi/\sqrt{12} \approx 0.907$
312the greengrocer’s stack of oranges (Kepler conjecture; Hales, 1998–2005)$\pi/\sqrt{18} \approx 0.740$
424not proved
8240the $E_8$ lattice (Viazovska, 2016)$\pi^4/384 \approx 0.254$
24196,560the Leech lattice (Cohn, Kumar, Miller, Radchenko, Viazovska, 2016)$\pi^{12}/12! \approx 0.0019$

Thomas Hales proved the Kepler conjecture on packing in three dimensions: the proof was announced in 1998 and published in 2005, and a full computer verification (the Flyspeck project) was completed in 2014. The proofs in 8 and 24 dimensions turned out to be far shorter. In March 2016 Maryna Viazovska proved that $E_8$ gives the densest packing in eight dimensions, and a week later, with Cohn, Kumar, Miller and Radchenko, she carried the method over to 24 dimensions. The key was a special function built from modular forms — the “magic” function whose existence Cohn and Elkies had conjectured. Viazovska received the Fields Medal for this work in 2022. In 2026 both proofs were fully formalized in the Lean proof assistant.

Look at the density column: it falls fast. It is the same story as the ball inscribed in a cube: balls in high dimensions are “skinny”, and almost all of space is inevitably left between them. These problems are not just pretty. Coding messages for a noisy channel is the same kind of packing: each message is a point of a high-dimensional space, noise moves it within a small ball, and the balls of different messages must not overlap. Claude Shannon introduced this geometric picture in 1949.

Key idea

In high dimensions almost everything is concentrated: a ball’s volume near its surface and near every equator, the angles between random vectors near $90°$, the distances between random points near a single value. That hurts (the curse of dimensionality) but it also helps: random projections preserve distances, and the directions of embeddings are easy to tell apart.

Summary

  • The layer of thickness $\varepsilon$ at the surface holds $1-(1-\varepsilon)^n$ of an $n$-ball’s volume; for large $n$ the ball is almost all peel.
  • For a random point of the ball $\mathbb{E}[x_1^2] = 1/(n+2)$, so almost all of the volume is also near every equator.
  • In the cube $[-2,2]^n$ the ball between the $2^n$ corner balls has radius $\sqrt{n}-1$ and pokes out of the cube for $n \ge 10$.
  • For random unit vectors $\mathbb{E}[\cos^2\theta] = 1/n$: the angle between them is $90°$ with a spread of $57.3°/\sqrt{n}$. Distances between random points of the cube are close to $\sqrt{n/6}$, give or take about $0.24$.
  • The curse of dimensionality: grids grow like $10^n$, neighbourhoods stop being local, the nearest neighbour loses its meaning.
  • The Johnson–Lindenstrauss lemma: $N$ points can be mapped linearly into $O(\varepsilon^{-2}\log N)$ dimensions, changing squared distances by a factor of at most $1 \pm \varepsilon$.
  • Kissing numbers are known exactly only in dimensions 1, 2, 3, 4, 8 and 24; densest packings are proved in dimensions 2, 3, 8 and 24.

Next: where dimensions live — spacetime, robot configurations, colour, data, and the extra dimensions of physics.

Sources