sdlc.dev

Computer science fundamentals for working engineers path · Inside the machine · Topic 1 of 13

How a CPU runs your code

Not started

A CPU core repeats one loop: fetch the instruction the program counter points to, decode it into the operation and its operands, execute it, write the result to a register or memory, and advance the program counter (or jump, if the instruction was a branch). A modern core still has that loop at heart, but runs it so aggressively in parallel that hundreds of instructions can be in flight at once, which is why the shape of your code can matter as much as how many operations it does.

Why it matters: Knowing that a modern CPU overlaps and predicts instructions explains why the same loop can run several times faster on sorted data, and why 'fewer instructions' is not the same as 'faster'.

  1. 1The basic cycle

    The CPU fetches the instruction at the program counter, decodes it, executes it, writes the result and moves on. Everything else is an optimisation of that loop.

  2. 2Pipelines overlap work

    Like an assembly line, different stages work on different instructions at the same time, so one instruction can finish per cycle or better.

  3. 3Branches are guessed

    To keep the pipeline full, the CPU predicts which way each branch goes and runs ahead. A wrong guess throws that work away and costs many cycles.

  4. 4Out of order, then in order

    Instructions run as soon as their inputs are ready, not strictly in program order, but their results become visible in program order.

1

How it works

The explanation, with examples you can run.

Pipelines

Executing one instruction takes several steps, and each needs different hardware. A pipeline lets them overlap: while instruction 1 executes, instruction 2 is being decoded and instruction 3 fetched, like cars on an assembly line. Modern cores go further. They are superscalar (several pipelines, so several instructions can start per cycle) and out of order: instructions wait in a buffer until their inputs are ready and then run, even if earlier instructions are still waiting for memory. Results are committed in program order, so the program can't tell, apart from the speed.

The enemy of a pipeline is not knowing what comes next. A load that misses the cache can take hundreds of cycles (the next topic), and a dependent instruction must wait for it. A branch is worse: until the condition is computed, the CPU doesn't know which instructions to fetch.

Branch prediction

Rather than wait, the CPU guesses. A branch predictor remembers how each branch behaved recently and predicts the same pattern again; the core then runs speculatively down the predicted path. A correct guess costs nothing. A wrong one means discarding the speculative work and restarting, typically 15 to 20 cycles on current desktop cores.

⌘/Ctrl + Enter
OutputNot run yet

Press Run to see what your code prints, in order.

Runs in this browser, in an isolated frame with no access to this page. console.log, info, warn and error are shown; code runs as a classic script, so use an async function for await.

On a Mac laptop with Node 22, the random array took about 30 ms per pass and the sorted one about 6 ms, the same work on the same numbers. With random values the if is a coin toss and the predictor is wrong about half the time. Sorted, it sees one long run of "skip" followed by one long run of "add". Your numbers will differ, and a compiler may remove the branch altogether (with a conditional move), in which case the gap disappears.

What this means day to day

  • Predictable code is fast code. Data-dependent branches inside hot loops are expensive when the data is random; sorting, partitioning or branch-free arithmetic can help.
  • Clock speed isn't everything. Performance is roughly clock rate × instructions per cycle (IPC). Profilers such as perf stat report IPC and branch-miss counts, which tell you whether the CPU is busy or waiting.
  • Speculation leaks. Spectre-class vulnerabilities (2018) showed that speculative execution leaves traces in the caches that attackers can measure. Operating systems, browsers and language runtimes now include mitigations that cost some performance.

Most of the time a core is not limited by arithmetic at all but by waiting for memory, which is the next topic.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.In this topic's example, summing values of 128 or more over the same numbers took about 30 ms in random order and about 6 ms sorted. Why?
  2. 2.What is a pipeline stall?
  3. 3.A CPU at 3 GHz retires on average 2 instructions per cycle. Roughly how many instructions per second is that?

0 of 3 right so far. Get all three to finish this topic.

Go deeper