Pipelining

Washing laundry: wash (30 minutes), dry (40), fold (20). Doing four loads one after another takes 4 × 90 = 360 minutes. But while load 1 dries, load 2 can wash. Overlapping the stages, the four loads finish in far less time, even though each load still takes 90 minutes. That is pipelining.

Five stages

The classic RISC pipeline splits every instruction into five stages, each taking one clock cycle:

In every cycle, each stage works on a different instruction: five instructions are in flight at once.

How much faster?

With k stages, n instructions take k + n − 1 cycles instead of k × n. For large n the speedup approaches k:

k = 5
for n in [1, 5, 100, 10000]:
    plain = k * n
    piped = k + n - 1
    print(n, "instructions:", plain, "cycles without a pipeline,", piped, "with; speedup", round(plain / piped, 2))

Pipelining improves throughput (instructions finished per second), not latency (the time for one instruction, which even grows a little because of the registers between stages).

Two things keep real pipelines below the ideal:

• Unbalanced stages: the clock must fit the slowest stage, so the others waste part of each cycle.
• Hazards: sometimes an instruction cannot go on because it needs a result that is not ready yet. That is the next lesson.

stages = {"IF": 250, "ID": 150, "EX": 200, "MEM": 300, "WB": 100}   # picoseconds
single_cycle = sum(stages.values())
clock = max(stages.values())
print("one long cycle per instruction:", single_cycle, "ps")
print("pipelined clock (slowest stage):", clock, "ps")
print("best speedup:", round(single_cycle / clock, 2), "(not 5, because MEM is slow)")

Modern processors have 10 to 20 stages, and are superscalar: they fetch and run several instructions per cycle, even out of program order when the results allow it.

Mark lesson complete