CPU·IV The machine Chapter 31 of 65

Memory and the clock

The adder from the last chapter forgets its answer the moment its inputs change. Here we pick up an oscilloscope and watch a loop of two gates begin to remember. Along the way we find out why a processor needs a metronome and how the program of the Apollo lunar module was woven into wire.

Basics 55 minutes Computer architecture History

Builds on: 30 · The machine does arithmetic

What you will take away

  • read timing diagrams and predict what a latch, a flip-flop, a register or a counter will do on the next clock edge
  • explain how feedback turns gates into memory, and how a clock signal turns them into a machine that takes steps
  • understand what registers and gigahertz mean in a processor’s specs, and how RAM and ROM are built

Chapter 30 built the ALU of Iskra-8 and ended on an experiment that leaves you uneasy. Put 200 and 100 on its inputs, and 44 appears at the output, while whatever didn’t fit in the byte goes to the carry flag. Take the inputs away, and the answer is gone. To add up ten numbers, you have to put the running total somewhere and pick it up again on the next step. There is nowhere to put it: the circuit has no place where a number could stay.

In Python the question never comes up. In Chapter 4 we set up a piggy bank, total, and added number after number to it, and the value waited patiently for the next step. The circuits of Chapters 29 and 30, on the other hand, behave like the pure functions of Chapter 5: the output depends only on what is at the inputs right now. Yesterday’s inputs don’t exist for them.

To compute step by step, the machine lacks two things. It needs a place where a number lies until somebody asks for it: memory. And it needs a signal that means “now,” at which all parts of the machine take their next step together: time. In the workshop of Chapter 29 you couldn’t run a wire from a part’s output back to its own input, because loops were banned. Here we allow them, and a loop will give us both memory and a metronome. All of this happens in time, so the main instrument of the chapter is an oscilloscope. We will watch signals, guess the next jump and check our guesses.

The sweep: how to read an oscilloscope

An oscilloscope draws the voltage on a wire as a function of time: time runs from left to right, and voltage goes up. A digital circuit has only two levels, so the picture comes out in steps: the high level is 1, the low level is 0. Several such tracks, stacked one under another on a shared time axis, are called a timing diagram. A jump from low to high is called a rising edge, and a jump from high to low a falling edge. Almost everything in this chapter happens on edges.

The first experiment is the silliest circuit imaginable. Take an inverter, a NOT gate, and connect its output to its own input. If the output is 1, the input is 1 too, and the inverter has to put out 0. But then the input is 0, and it has to put out 1. The logic contradicts itself.

What will the oscilloscope show at the output of an inverter wired to itself? A hint: the gate answers after a delay, however tiny.

In a digital model with delay, the contradiction turns into an oscillation. The output changes; one delay later the change reaches the input; one more delay later the gate responds to it, and so on forever: 0, 1, 0, 1 with a period of two delays. A lone inverter on a chip behaves differently: its output hangs halfway between 0 and 1. A gate is an amplifier at heart, and in such a short loop it finds a balance. That is why in practice rings are built from at least three inverters, and then the oscillation is stable.

Delay is the hero of this chapter. Without it, a circuit with a loop would be an equation with no solution; with it, the circuit becomes a process you can follow step by step. We will follow it the way Chapter 18 simulated the emergency room: every switch is an event in a calendar, and the calendar is a heap with the nearest event on top. The program below builds rings of one, two, three and five inverters and prints the moments when the output of the last one changes.

With three inverters the output changes every three delays, with five every five: a single edge runs round the ring, and a full period is two of its laps, $2 \cdot n$ delays. Such a ring is called a ring oscillator. Chip designers put one right on the die when they want to measure how fast their gates turned out: a frequency is easy to measure, and the delay follows from it at once.

The ring of two inverters, though, stays silent. In the widget below this is mode “2”: the first inverter puts out 1, the second 0, and each agrees with its input. There is nothing to change, and the circuit will stay like this for as long as it has power. It remembers the state it was left in, and that is one bit of memory.

A loop of gates. At the top is the circuit: a green wire carries 1, a dark one 0, and a glowing dot is an edge on its way to the next gate while the delay lasts. At the bottom is the oscilloscope. In mode “2” you can push a wire with your finger.

Push a wire in the ring of two. The edge goes round the loop once, comes back to where it started and agrees with the new value. The loop has been rewritten. Here, once again, the output is fed back to the input, and this kind of connection has a name: feedback. With an odd number of inverters in the loop it gives oscillation; with an even number, memory. One small thing is missing: pushing a wire with your finger is no way to write.

A loop with doors

In Chapter 29 we built an inverter from a NAND with one signal on both its inputs. There is another way: a NAND with 1 on its second input also inverts the first, because $\overline{x \cdot 1} = \overline{x}$. Replace both inverters of the loop with such NANDs. As long as the free inputs hold ones, nothing has changed: the loop of two inverters keeps its bit. But now it has doors. Put 0 on the free input of the top gate, and that NAND puts out 1 whatever comes to it round the loop, since a NAND with a zero on either input answers 1. The top output has become 1, the edge has run round the loop, and the bottom output has become 0. Put 1 back on the door, and the loop stays in its new position.

What we have built is a latch. The top output is called $Q$, the stored bit, and the bottom one $\overline{Q}$, its negation. The input that sets $Q = 1$ is written $\overline{S}$ (for set; the bar is there because it acts on a zero), and the other input, $\overline{R}$ (for reset), resets $Q$ to 0. These two inputs give the circuit its name: the SR latch.

$\overline{S}$$\overline{R}$what happens to $Q$
11the old value is kept
01$Q = 1$
10$Q = 0$
00$Q = \overline{Q} = 1$, the forbidden state

The last row is a trap. Open both doors, and both gates put out 1, so the outputs stop being negations of each other. That is only half the trouble. The worse half comes when both doors close at the same moment: both gates see ones at their inputs and both drop to 0 together, then both see zeros and jump back to 1. In the code below both gates switch at once, step by step, until the circuit settles.

In the ideal model the latch oscillates forever. Real gates are never quite identical: sooner or later one gets slightly ahead of the other, and the latch falls into one of its two states. Which one, and when, can’t be told in advance, and for a while the output may even hang between 0 and 1. This is called metastability, and it is a practical problem. When a signal comes into a circuit from outside, a button press for instance, it is passed through two memory cells in a row, so that the second one receives a value that has already settled. The widget above has a “latch” mode: press both doors, then “Release both at once,” with the “real gates” box ticked and without it.

The first trace to predict. You are given the inputs $\overline{S}$ and $\overline{R}$; draw the output $Q$ by running your finger along the bottom track: above the middle draws 1, below it 0. Then press “Check.” The “New” button gives you other inputs.

One data wire and an enable

The SR latch is awkward in two ways: it has a forbidden combination of inputs, and to write a bit you have to decide which of the two doors to press. Two gates placed in front cure both. There are still two inputs, but different ones: $D$, what to write, and $E$ (enable), whether writing is allowed. The doors receive $\overline{S} = \overline{D \cdot E}$ and $\overline{R} = \overline{\overline{D} \cdot E}$. While $E = 0$, both doors hold ones and the latch keeps its value. When $E = 1$, one of the doors is open ($D$ decides which), and the latch copies $D$. The forbidden state can never arise.

This cell is called a D latch, and an open one is said to be transparent: while $E = 1$, the output $Q$ repeats $D$, like a window. Predict the output yourself.

A D latch: while $E = 1$, $Q$ follows $D$; when $E$ drops to 0, $Q$ freezes at the last value.

Transparency looks like a convenience, but it breaks what we started all this for. Remember the piggy bank: a register, an adder and a loop between them, so that every step performs total = total + 5. Let the register be eight D latches. We open them for an instant to write the sum. The sum passes into the register and immediately appears at the adder’s input; sixteen delays later the adder’s output already reads $total + 10$, and the latches, still open, let that through as well. How many fives get added in one step depends on how many times the sum manages to run round the loop while the window is open. Worse, the adder’s bits settle at different times, and the register may catch a mixture of old and new bits.

The metronome

The fix works like a lock on a canal. Put two D latches in a row and open them in turn: the first when the control signal is 0, the second when it is 1. There is never a straight way through, because one of the two is always closed. While the signal is 0, the first latch listens to $D$ and the second holds the old value. At the moment the signal jumps from 0 to 1, the first one closes, keeping $D$ as it was an instant before the edge, and the second one opens and shows that value at the output. While the signal is 1, $D$ may change as it likes: the first latch is closed.

Such a pair is called a D flip-flop. It changes its output only at the moment of a rising edge, and only once per period. The control signal, which jumps steadily 0, 1, 0, 1, is called the clock, and one of its periods is a clock cycle, or tick. All the flip-flops in a processor hang on one clock signal, and on every edge they all take their new values at once.

A D flip-flop. $Q$ changes only on the rising edges of CLK (the yellow marks) and takes the value $D$ had at that moment. Compare it with the transparent latch above.

Between two edges the circuits without memory (engineers call them combinational) have time to finish: the adder gets new inputs right after an edge, and by the next edge its output holds a ready answer. This is the main rule of a synchronous circuit: the clock period must be longer than the slowest path from one flip-flop to another. In Chapter 30 the carry took sixteen delays to run through the adder, and those sixteen delays are what limit the clock rate of Iskra-8.

Now you can read a line from a processor’s specs. “3 GHz” means three billion clock cycles a second, so one cycle lasts a third of a nanosecond; in that time light travels about ten centimeters, or four inches. It all starts with a quartz crystal of some tens of megahertz. From its vibrations the board derives a reference clock, usually 100 MHz, and a multiplier on the processor die raises it further: a multiplier of 36 in your computer’s settings means $36 \times 100$ MHz $= 3.6$ GHz. Overclocking a processor is a game played with the margin the engineers left. Push the frequency too high, and at some point the longest path stops arriving before the edge. Then a half-computed value lands in a register, and the program starts doing strange things.

The register: eight flip-flops on one clock

Eight D flip-flops on a common clock hold a byte. That is a register. Usually each flip-flop gets a switchman from Chapter 29 in front of it, all of them sharing one control input, LOAD. When LOAD = 1, the new value goes to input $D$; when LOAD = 0, the flip-flop’s own output does, and on the edge the flip-flop overwrites itself with the same value. This way the register takes a number only when asked to, and on all other cycles it holds the old one.

Iskra-8 has six eight-bit registers: four working ones, R0–R3, and two special ones, PC and SP, which belong to the next chapter. That makes 48 flip-flops, plus one for each of the ALU’s three flags. The processor in your laptop has 16 general-purpose registers (the x86-64 architecture) or 31 (ARM64), 64 bits each. That isn’t many: registers are the fastest and the most expensive memory, and they sit right next to the ALU.

The timing diagram of a register. The eight data wires are drawn as a single track, a bus, with the number written on it. The register takes $D$ only on a rising edge of CLK and only when LOAD = 1. Tap the pieces of the Q track to choose a value.

A register plus an adder plus a wire back, and the piggy bank of Chapter 4 exists in hardware: on every edge, $total \leftarrow total + x$. The rule “all flip-flops take their new values at once” has a twin in Python, the multiple assignment of Chapter 2: first the whole right-hand side is computed from the old values, and only then do all the names change together. The line a, b = b, a + b is a clock edge for two registers. Split it into two lines, and the new a leaks into the second one, as if through a transparent latch.

With an edge we get the Fibonacci numbers; without one, powers of two, which by the eighth step overflow the eight-bit register and turn to zeros. Conway’s Game of Life works the same way, and so does any simulation where all the cells change together: the new generation is computed from the old one in full and only then swapped in. Update the cells one at a time, and you get a different game, and a wrong one.

The counter and the watch on your wrist

Put a one into the piggy bank on every cycle, and you get a counter: a register whose output feeds a “+1” circuit, whose output goes back to the register’s input. A “+1” circuit is simpler than an adder. When you add one, a bit changes if and only if all the bits below it are 1: $0111 + 1 = 1000$. Bit zero changes on every cycle, bit one on every second cycle, bit two on every fourth. This observation turns into a circuit made of ANDs and XORs alone, and you will build it in the task “A counter without plus”.

There is also a thoroughly lazy counter. Connect each flip-flop’s output to its own input through an inverter, so that it toggles on every edge. Then use the inverted output of each flip-flop as the clock of the next one. The first toggles on every cycle, the second whenever the first falls from 1 to 0, that is, half as often, and the third half as often again. Each flip-flop divides the frequency by two.

A frequency divider. $Q_0$ toggles on every rising edge of CLK, $Q_1$ every time $Q_0$ falls from 1 to 0. Read $Q_1 Q_0$ as a two-digit binary number, and you get 0, 1, 2, 3, 0…

A chain like this ticks inside every quartz watch. The quartz crystal in a watch vibrates 32,768 times a second, and the number wasn’t chosen by chance: $32\,768 = 2^{15}$. Fifteen flip-flops in a row halve the frequency fifteen times, down to one hertz, and once a second the last of them nudges the hand. The cell below checks it.

The chain returns to zero on the 32,768th vibration, on the 65,536th and on the 98,304th: once a second. Incidentally, the inner while loop is a carry running along the bits, as in the adder of Chapter 30. The lazy counter has a price: its bits change one after another, and while the wave runs, wrong numbers flash at the outputs for an instant. A watch doesn’t mind; a processor does, so the counters in a processor are synchronous, with all their flip-flops on one clock.

The traffic light: a machine made of a register and a table

A register, a combinational circuit and a clock are enough to build a device whose behavior depends on what happened before. Take a traffic light. Right now the green is on; in a few cycles the yellow will come on, then the red, then red and yellow together, which in Britain and much of Europe warns that green is coming. What to light on the next cycle depends on what is lit now and on how long it has been lit. Those two numbers are kept in registers. The table of transitions “from which state to which” is a combinational circuit that computes the registers’ next values from their outputs. The clock edge overwrites the registers, and everything repeats.

A device with a finite number of states that at every step picks its next state from the current one and its input is called a finite-state machine, or finite automaton. You have met one already: in Chapter 27 the KMP algorithm ran along a text remembering a single number, how many letters of the pattern had matched so far. In hardware a state machine is always a state register and a transition table. Elevators, washing machines and keyboard controllers are built this way. In Chapter 54 automata become a mathematical object, and we will find out what they can do and what they can’t.

Our traffic light has no inputs: it goes round and round and listens to nobody. In the task “A traffic light with a button” a button for pedestrians appears, along with one more bit of memory, to remember that the button was pressed even if the green for the cars hasn’t yet run its full time.

RAM: 256 numbered boxes

Iskra-8 has 256 bytes of memory, from 0x00 to 0xFF. That is 256 registers of eight flip-flops each, 2048 bits. There are many registers, and we want few wires to them: eight address lines, eight data lines and a “write” signal. How do you reach one of 256 by its number?

Writing needs a decoder: a circuit with eight inputs and 256 outputs, and any given address lights only one of them. Line 42 is an AND of the eight address bits, with inverters where the zeros are: $42 = 00101010_2$, so the line is lit when $\overline{a_7}\,\overline{a_6}\,a_5\,\overline{a_4}\,a_3\,\overline{a_2}\,a_1\,\overline{a_0}$. The decoder’s output, passed through an AND together with the “write” signal, becomes the LOAD of the right register, and on the clock edge the byte on the data lines lands in that cell and nowhere else.

Reading needs the opposite: pick one of 256 bytes and put it on the data lines. This is the tree of switchmen promised in Chapter 29. The bottom level of 128 switchmen uses the lowest bit of the address to pick one cell out of each pair, the next level uses the next bit to pick one out of each pair of winners, and after eight levels a single cell is left.

Two thousand switchmen for reading alone is expensive. Commercial memory chips economize: they lay the cells out in a square and split the address in half. The top four bits choose one of 16 rows, the bottom four one of 16 columns, and instead of 256 decoder lines, 16 + 16 will do. The memory of Iskra-8 in the widget below is built this way. Choose an address with the switches or with your finger on the grid, set a byte and write it.

The RAM of Iskra-8: 16 rows × 16 columns. The top half of the address lights a row, the bottom half a column, and the cell where they cross is open. Addresses 0xF0–0xF7 are wired to the 8 × 8 screen: each byte is a row of pixels. Write something there.

Memory in which you can both write and read any cell by its number in a single access is called random-access memory, or RAM: cell number 200 comes out as fast as cell number 0. Chapter 14 relied on this property when it explained why a[i] costs the same in a Python list for any i.

Commercial memory uses thriftier circuits than NAND gates. A cell of static RAM (SRAM), the kind processor caches are made of, is six transistors: the same loop of two inverters plus two switches for writing and reading. Register cells get more switches, so that several numbers can be read in one cycle. Dynamic RAM (DRAM), the ordinary gigabytes of a computer’s memory, stores a bit as the charge of a tiny capacitor next to a single transistor. The charge leaks away, and every cell has to be read and written back again, usually every 64 milliseconds. Nearly all the main memory in the world is this kind of memory, which keeps forgetting and keeps remembering.

ROM: a program woven into wire

Some memory never needs rewriting: a computer’s startup program, a font table, the firmware of a washing machine. It is simpler to build. The address decoder stays, but no flip-flops are needed: each output line of the decoder runs past the eight data lines and is connected to the ones where a 1 should be, and not to the others. The contents are fixed at manufacture by the pattern of connections and can’t be changed. This is read-only memory, or ROM.

The most famous ROM in history was woven by hand.

A hundred and sixty years earlier, on Jacquard’s loom from Chapter 4, a needle went through wherever the card had a hole and raised its thread. In rope memory a wire goes through a core wherever the program has a one. Ada Lovelace wrote that the Analytical Engine “weaves algebraical patterns”; the Apollo programs were woven in earnest.

Rope memory had another advantage: density. One wire threaded through a core is a bit, and many wires went through each core, so the rope held several times more bits per cubic centimeter than the rewritable core memory of the same computer. To catch a weaver’s mistake, every word got a parity bit: a sixteenth wire went through the core or past it so that the word always had an odd number of ones. A wrong pass changes the number of ones by one, the parity breaks, and the computer notices the damage. It is a relative of the Luhn check digit from Chapter 5.

A core rope loom. Each core is one letter of the word, wires 7…0 are its bits, and wire P is the parity. In the card below, a dot means “through the core.” Weave a word, then tap a letter above the card to read its core. The “A weaver’s slip” button spoils one pass.

Tasks

Four tasks, and in each a circuit with memory lives by the clock. Signals are lists of zeros and ones, one value for each moment of time, like the tracks of an oscilloscope.

A signal is written as a list: d[t] is the value on the wire at moment $t = 0, 1, 2, \ldots$ Write two functions that take the inputs and return the list of values of the output $Q$ at the same moments.

latch(en, d) is a D latch: at moment $t$, if en[t] == 1, the output equals d[t]; otherwise it stays as it was. flipflop(clk, d) is a D flip-flop: the output changes only at the moment of a rising edge, that is, when clk[t - 1] == 0 and clk[t] == 1, and then it becomes equal to d[t]. Before the record starts, both $Q$ and clk were 0. The lists can be hundreds of thousands of values long.

Both functions walk through the moments of time and keep the current value of $Q$ in a variable; that variable is the flip-flop. At every moment they decide whether to update it or leave it alone, and append it to the answer.

To spot an edge, you need one more bit of memory: the value of clk at the previous moment. Start it at zero, and clk[0] == 1 will count as an edge too.

A latch needs one bit of state, $Q$. A flip-flop needs two: $Q$ and the previous value of the clock. In hardware this second bit is kept by the first latch of the pair, which remembers what came before the edge.

The state of a four-bit counter is a tuple (q3, q2, q1, q0), high bit on the left. Write step(q, en): when en == 1 it returns the next state ($q + 1$ modulo 16), and when en == 0 the same state. Then write decade(q, en), a decimal counter like the ones in a digital clock: it counts 0, 1, …, 9 and back to 0, and returns a pair (new state, carry), where the carry is 1 only on the step 9 → 0.

You may use only gates: the operations &, | and ^, your own functions and tuple unpacking. No +, no if, no comparisons and no not: “not $x$” is x ^ 1. The checker will chain two of your decimal counters, so that the carry of the lower one becomes the en of the upper one, and make sure the display counts up to 99 and comes back to 00.

Bit $i$ changes when counting is enabled and all the lower bits are 1. Define toggle signals: $t_0 = en$, $t_1 = t_0 \cdot q_0$, $t_2 = t_1 \cdot q_1$, $t_3 = t_2 \cdot q_2$. The new bit is $q_i \oplus t_i$.

Of the states 0–9, only nine, $1001_2$, has ones in both $q_3$ and $q_0$. The carry is wrap = en & q3 & q0. When it is 1, every bit of the result has to be cleared: AND each bit of step(q, en) with wrap ^ 1.

The chain $t_0, t_1, t_2, t_3$ is the same carry that ran through the adder in Chapter 30, except that the number being added is always one. Feeding the carry of one decimal counter into the en of the next is how digits are joined in clocks, in odometers and on stadium scoreboards. For seconds from 00 to 59, the upper digit needs its own reset at six.

A state machine for the car signals at a pedestrian crossing. Its state is a triple (light, t, request): which light is on, for how many ticks it has been on before the current one (t = 0 when the light changes), and the request bit, whether anybody has pressed the button. Write step(state, button), which takes the state and the button in the current tick (0 or 1) and returns the state for the next tick.

lightstays onthen
greenat least 3 ticks, and until there is a requestyellow
yellow1 tickred: pedestrians cross
red3 ticksred-yellow
red-yellow1 tickgreen

A press sets request = 1 in any light except red, and the request counts in the same tick. On the change to red the request is cleared: the pedestrian has had their turn. While red is on, the button does nothing.

Compute done = t + 1, the number of ticks the light has been on, the current one included. Time is up when done reaches the duration. Then comes the new light with t = 0; otherwise it is the same light with t = done.

The trap is a press that comes before the minimum green has run out. If you look only at button in the current tick, a pedestrian who pressed once will wait forever. So first request = request | button, and only then the decision about the transition.

In hardware request is the SR latch from the start of the chapter: the button sets it to 1, and the change to red resets it. The rest is the light register, the tick counter and the transition table. A press during red-and-yellow is remembered and starts the next round as soon as the green has run its three ticks.

A core rope memory of $n$ cores: core $i$ stores the byte data[i]. There are nine wires. Wire $k$, from 0 to 7, carries bit $k$ (weight $2^k$) and passes through core $i$ if that bit of data[i] is 1. The ninth wire, number 8, is the parity wire: it passes through a core if the eight bits hold an even number of ones, so that an odd number of wires goes through every core, as in Apollo.

Write weave(data), which returns a list of nine lists of core numbers in increasing order, and read(wires, n), which turns the wires back into a list of $n$ bytes, with None in place of any core whose parity is broken. Words can be tens of thousands of cores long.

For weave, go through the bytes and through the eight bits of each one: byte >> k & 1. Count the ones as you go; their number decides the parity wire.

For read, don’t ask every core whether it is on every wire: i in wire on a list walks the whole list, and with 60,000 cores that adds up to billions of comparisons. Walk once along each wire, and for every number on it add the bit and add one to that core’s count of passes.

Both functions are linear: each pass of a wire through a core is touched once. The machine read the same way: a remagnetized core answers on all its wires at once, in a single pulse. Parity catches any single slip of the weaver, but not two at once: two errors in one word make the count odd again, and the damage goes unnoticed.

What next

Iskra-8 now has almost everything. The ALU of Chapter 30 adds, subtracts, compares and shifts. Registers R0–R3 hold numbers from one cycle to the next, the flags remember how the last result came out, the 256-byte RAM keeps anything you like at an address, and eight of its bytes show on the screen. There is a counter that adds one on every cycle, and a clock that says “step” again and again.

But all of it stands still. Someone has to decide what the ALU does on this cycle (add or subtract), which registers to take the numbers from, where to put the answer and what to do on the next cycle. So far we have decided that ourselves, by hand. The hint is in plain sight: a counter can point at a memory cell, and a cell can hold an order instead of a number. How the machine follows such orders on its own is the subject of Chapter 32: there Iskra-8 comes to life, and the first processor to run it will be you.