Branches & Speculation
A CPU cannot afford to wait to find out where a branch goes, so it guesses. What that buys, what a misprediction costs, and why unpredictable data hurts more than extra work.
A pipelined CPU must fetch an instruction every cycle, but at a conditional branch it does not yet know which instruction comes next. Waiting for the answer is unaffordable on a deep pipeline, which is why every high-performance CPU guesses instead.
The CPU has to supply a fetch address before it knows the branch outcome, so it predicts one from history. Modern predictors are accurate enough that well-behaved code pays essentially nothing for its branches — which is exactly why the badly-behaved cases stand out so sharply.
When the predictor is wrong, everything fetched and executed down the wrong path is discarded and the pipeline refills from the correct address. The cost is not the discarded work — it is the emptiness afterwards, and it scales with how deep the pipeline is.
Modern CPUs execute instructions they may have to discard, on the bet that the guess was right. It works because the guess usually is, and because discarded results never become architecturally visible. What they do leave behind — microarchitectural traces — turned out to matter enormously.
Replacing an unpredictable branch with arithmetic converts a variable cost into a fixed one. That is a win when the branch mispredicts often and the work it guards is trivial — and a loss in every other case, which is most of them.