Why pointer is faster than reference

Background

In my previous post I argued that a C++ reference can be faster than a pointer, because Clang attaches dereferenceable to a reference parameter and LLVM can then speculate loads above branches. At the end of that post I showed Intel APX's cfcmov, a conditional move whose memory operand does not fault when the condition is false, and I demonstrated it with this file:

// mux.cpp
struct S { int a, b, c, d; };
void by_ptr(S* s) { s->b = s->a ? s->c : 0; }
void by_ref(S& s) { s.b  = s.a  ? s.c  : 0; }

I only showed the pointer version there. Here is what the same command produces for both functions, on LLVM main (commit 3d91765f54e9):

clang++ -O3 -mapx-features=cf -masm=intel -S mux.cpp

_Z6by_ptrP1S:
        cmp     dword ptr [rdi], 0
        cfcmovne eax, dword ptr [rdi + 8]
        mov     dword ptr [rdi + 4], eax
        ret

_Z6by_refR1S:
        mov     eax, dword ptr [rdi]
        test    eax, eax
        je      .LBB1_2
        mov     eax, dword ptr [rdi + 8]
.LBB1_2:
        mov     dword ptr [rdi + 4], eax
        ret

The pointer gets the new branchless instruction. The reference, the version with more information for the compiler, gets a branch. So this post is about the opposite question: why is the pointer faster than the reference here, and what does it take to fix it?

Tracing every pass

Guessing is cheap and usually wrong, so I dumped the IR after every pass for both functions and diffed them. The middle end runs 94 passes on each function, the backend 61:

clang++ -O0 -Xclang -disable-O0-optnone -mapx-features=cf -fno-discard-value-names -emit-llvm -S mux.cpp -o mux.O0.ll
opt -O3 -S mux.O0.ll -o mux.opt.ll -print-after-all 2> mid.txt
llc -O3 -mtriple=x86_64 -mattr=+cf mux.opt.ll -o mux.s -print-after-all 2> back.txt

Both functions enter identical, as the same three-block triangle, and they part ways at exactly two passes.

SimplifyCFG, run 2 of 8 (the "GlobalCleanup" run right after mem2reg and InstCombine). Both functions look like this going in:

entry:
  %0 = load i32, ptr %s
  %tobool.not = icmp eq i32 %0, 0
  br i1 %tobool.not, label %cond.end, label %cond.true
cond.true:
  %1 = load i32, ptr %c
  br label %cond.end
cond.end:
  %cond = phi i32 [ %1, %cond.true ], [ 0, %entry ]

speculativelyExecuteBB asks whether the load in cond.true may be executed unconditionally. For the reference the answer is yes (the check), thanks to dereferenceable(16), so the block is flattened:

entry:
  %0 = load i32, ptr %s
  %tobool.not = icmp eq i32 %0, 0
  %1 = load i32, ptr %c
  %cond = select i1 %tobool.not, i32 0, i32 %1

For the pointer the answer is no, and the function leaves this pass unchanged.

SimplifyCFG, run 8 of 8. This is the last SimplifyCFG in the pipeline and the only one constructed with hoistLoadsStoresWithCondFaulting(true), the option that knows about cfcmov. The pointer still has its conditional load, so the pass rewrites it into a masked load with the branch condition as the mask:

  %3 = call <1 x i32> @llvm.masked.load.v1i32.p0(ptr align 4 %c, <1 x i1> %2, <1 x i32> zeroinitializer)

The reference has had no branch since run 2. There is nothing for this pass to do, and it does nothing.

That is the whole middle-end story: the cfcmov transformation looks for loads that are still inside a branch, and a reference's load never is, because the same attribute that makes it hoistable moved it out 76 passes earlier. No pass rejects cfcmov for the reference. The pass that produces cfcmov simply never sees it.

The backend. The pointer enters instruction selection as a CLOAD node (the lowering of the masked load) and selects straight to cmp + cfcmov. The reference enters as select of a plain load. Instruction selection does the natural thing and folds the load into a cmov:

%1 = MOV32rm  [%0]                      ; a
TEST32rr %1, %1
%2 = CMOV32rm %1(tied), [%0 + 8], NE    ; cmov with the load folded in

This is correct and branchless. Then a pass called X86CmovConversion runs and rewrites it:

TEST32rr %1, %1
JCC_1 %bb.2, E
%3 = MOV32rm [%0 + 8]
%2 = PHI %3, %bb.1, %1, %bb.0

That is the je in the listing above.

A heuristic from 2017

X86CmovConversion converts every cmov with a memory operand into a branch. The rule was added by Chandler Carruth in 2017 (commit 93a645525cf3), and the commit message explains it:

"We have seen periodically performance problems with cmov where one operand comes from memory. On modern x86 processors with strong branch predictors and speculative execution, this tends to be much better done with a branch than cmov. We routinely see cmov stalling while the load is completed rather than continuing, and if there are subsequent branches, they cannot be speculated in turn."

The reasoning is sound for a predictable branch: the CPU runs ahead of it, while a cmov cannot retire until its load returns. For an unpredictable branch it is exactly backwards, and unpredictable branches are the whole reason RTL simulation wants cmov in the first place. cfcmov escapes the rule for a trivial reason: the pass looks for CMOV opcodes, and CFCMOV is not one of them.

So the reference is slower here because it was more optimizable. Its load was speculated early, which took it away from the one transformation that knows about cfcmov, and handed it to a heuristic that does not trust cmov with memory operands.

Can't SimplifyCFG just hoist the reference's load the same way?

That was my first idea. The cfcmov hoist is only enabled on the last SimplifyCFG run, but a hidden flag enables it on all of them. With -mllvm -hoist-loads-stores-with-cond-faulting=true, run 2 does turn the reference's load into a masked load, exactly like the pointer. It survives one more pass and is gone by run 4:

The pass in between is InstCombine. In simplifyMaskedLoad it has:

  // If we can unconditionally load from this address, replace with a
  // load/select idiom.
  if (isDereferenceablePointer(LoadPtr, II.getType(), ...)) {
    LoadInst *LI = Builder.CreateAlignedLoad(...);
    return Builder.CreateSelect(II.getArgOperand(1), LI, II.getArgOperand(2));
  }

The same dereferenceable fact turns the masked load back into a plain load and a select. InstCombine deliberately knows nothing about the target, so it cannot know that on this one we would rather keep the conditional load. This is why the cfcmov hoist lives only in the last SimplifyCFG, after the last InstCombine, and by then the reference has no branch left.

Two policies with opposite defaults

Putting hints on the mux makes the disagreement visible:

The pointer path is governed by SimplifyCFG's isProfitableToSpeculate: with no profile data it hoists, and it backs off only when the profile says the load is usually skipped. Branchless by default. The reference path is governed by X86CmovConversion: with no profile data it branches, it ignores branch weights entirely, and the only thing that stops it is an unpredictable flag on the instruction. Branch by default.

Neither rule is wrong for the code it was tuned on. Intel's own numbers say the aggressive one loses without hardware PGO, which is why they removed cf from -mapxf and from -march=diamondrapids in LLVM 22; Chandler's numbers say the conservative one wins on predictable code. They just never agreed on a default, and the disagreement only shows when the same mux can reach either path depending on whether it was written with a pointer or a reference.

The hint that never arrived

The last row of the table is the one that bothered me. X86CmovConversion is supposed to leave a cmov alone when it is marked unpredictable, and __builtin_unpredictable is exactly the tool for an RTL mux. Yet the reference still got a branch. It turned out to be a bug: the unpredictable flag was dropped inside the backend before it could reach that pass. I fixed it in llvm/llvm-project#223469, and with it the hint works with no APX at all, on plain -O3:

void by_ref(S& s) { s.b = __builtin_unpredictable(s.a) ? s.c : 0; }

; before                                ; after
        mov     eax, dword ptr [rdi]            mov     eax, dword ptr [rdi]
        test    eax, eax                        test    eax, eax
        je      .LBB0_2                         cmovne  eax, dword ptr [rdi + 8]
        mov     eax, dword ptr [rdi + 8]        mov     dword ptr [rdi + 4], eax
.LBB0_2:                                        ret
        mov     dword ptr [rdi + 4], eax
        ret

Without the hint nothing changes, so this only affects code that asked for it.

A twist with -mapxf

While collecting the listings I noticed that under the realistic flag set, -mapxf -mapx-features=cf, the reference was already branchless on unmodified main:

_Z6by_refR1S:
        mov     eax, dword ptr [rdi]
        test    eax, eax
        cmovne  eax, dword ptr [rdi + 8]
        mov     dword ptr [rdi + 4], eax
        ret

Not because any policy changed. With NDD available, instruction selection picks the three-operand cmov and leaves the load as a separate instruction; X86CmovConversion sees a cmov without a memory operand and leaves it alone; and then the peephole optimizer, which runs after the converter, folds the load in. The 2017 rule is bypassed on APX targets by pass ordering, which nobody decided. It is a good example of how many independent decisions sit between a ?: in the source and the instruction that comes out.

The practical consequence is that on an APX machine built with its own -march (which includes NDD), the reference is already branchless and nothing in this post needs to change for it. The hint fix and the discussion above matter for every x86 CPU shipping today, where the rule still fires. And because the APX exemption is an accident of pass ordering rather than a policy, nothing pins it down; a change to the NDD selection patterns or to the pass order could bring the branch back without anyone noticing.

If you just want cmov everywhere

There is no -funpredictable-branches, but the three decisions have three hidden options, and together they emulate one:

clang++ -O3 -mllvm -predictable-branch-threshold=100 -mllvm -x86-cmov-converter=false ...

The threshold (default 99) is what SimplifyCFG and CodeGenPrepare compare profile probabilities against; at 100 no branch is ever "predictable", so neither pass forms or keeps a branch on profile grounds. The second option disables the converter. With both, every reference variant in the table above compiles to cmovne eax, [rdi + 8], hinted or not, on any x86-64; the pointer variants still branch without CF, since nothing can speculate their load. The caveats are the usual ones for -mllvm: these are internal, unversioned options, and the threshold also silences __builtin_expect for the middle end's own speculation decisions.

Conclusion

The pointer was faster in my APX example because it was less optimizable. Its load could not be speculated, so it stayed conditional long enough to reach the one transformation that emits cfcmov. The reference's load was speculated early, handed to instruction selection as an ordinary load, and then a tuning rule from 2017 turned the resulting cmov back into a branch. Between the two sit two profitability policies with opposite defaults, and an unpredictable hint that was silently dropped on the way.

Takeaways:

1. References still give the compiler the permission that pointers cannot; that part of the first post stands.
2. Whether that permission turns into cmov, cfcmov or a branch is decided by heuristics that can be told what you know: __builtin_unpredictable now works end to end for these muxes.
3. The heuristic that takes the cmov away is the next thing to teach. And when a compiler does something surprising, dumping every pass is faster than reasoning about it. It's especially important in vibe coding era for understanding which pass diverged from expectations, thus you will know exactly where to look, where to change, and how to guide your AI agent effectively.