Why reference is faster than pointer, How ISA improve this

Background

Two years ago, I submitted Verilator PR #5254 to replace the pointer to the huge model struct with a reference. With LLVM, it measured an 11% performance improvement and 9% fewer sampled branch instructions at runtime.

I have many insights to share regarding why using references can lead to better performance compared to pointer dereferences in certain scenarios, and how new ISA extensions like x86-APX can further optimize such access patterns.

Let's dig into the reasons behind this performance difference and explore how references can be more efficient than pointers in certain contexts.

How do references make a difference in LLVM IR?

A reference and a pointer are the same thing in LLVM IR: an opaque ptr. To see where they differ, compile two functions with an identical body, one taking S* and one taking S&, and diff the IR of the two functions.

// ref.cpp
struct S { int a, b, c, d; };

void by_ptr(S* s) { s->b = s->a ? (s->c & 0xff) : (s->d >> 3); }
void by_ref(S& s) { s.b  = s.a  ? (s.c  & 0xff) : (s.d  >> 3); }

Then we compile with the latest LLVM master branch, commit 8e84c4a901a7fe6ba80b3c305b200c3e9bbc4606.

clang++ -fno-discard-value-names -emit-llvm -S ref.cpp -o ref.ll
diff <(sed -n '/^define.*by_ptr/,/^}/p' ref.ll) <(sed -n '/^define.*by_ref/,/^}/p' ref.ll)

1c1
< define dso_local void @_Z6by_ptrP1S(ptr noundef %s) #0 {
---
> define dso_local void @_Z6by_refR1S(ptr noundef nonnull align 4 dereferenceable(16) %s) #0 {
5c5
<   %0 = load ptr, ptr %s.addr, align 8
---
>   %0 = load ptr, ptr %s.addr, align 8, !nonnull !5, !align !6
12c12
<   %2 = load ptr, ptr %s.addr, align 8
---
>   %2 = load ptr, ptr %s.addr, align 8, !nonnull !5, !align !6
19c19
<   %4 = load ptr, ptr %s.addr, align 8
---
>   %4 = load ptr, ptr %s.addr, align 8, !nonnull !5, !align !6
27c27
<   %6 = load ptr, ptr %s.addr, align 8
---
>   %6 = load ptr, ptr %s.addr, align 8, !nonnull !5, !align !6

A reference in C++ carries additional guarantees compared to a pointer, which the compiler encodes in the LLVM IR as the nonnull, align, and dereferenceable attributes.

• nonnull: a reference is always bound to an object.
• align 4: that object is a real S, so the address has the alignment of an S.
• dereferenceable(16): the 16 bytes starting at %s, i.e. the whole S, can be loaded without trapping.

C++ hands the compiler these facts for free. A reference must refer to a valid object while it is used; a pointer may be null, one past the end of an array, or anything else.

Now run the optimizer on the same file and diff the two functions again.

sed 's/ optnone//' ref.ll | opt -O3 -S -o ref.opt.ll
diff -u <(sed -n '/^define.*by_ptr/,/^}/p' ref.opt.ll) <(sed -n '/^define.*by_ref/,/^}/p' ref.opt.ll)

@@ -1,23 +1,14 @@
-define dso_local void @_Z6by_ptrP1S(ptr nofree noundef captures(none) initializes((4, 8)) %s) local_unnamed_addr #0 {
+define dso_local void @_Z6by_refR1S(ptr nofree noundef nonnull align 4 captures(none) dereferenceable(16) initializes((4, 8)) %s) local_unnamed_addr #0 {
 entry:
   %0 = load i32, ptr %s, align 4
   %tobool.not = icmp eq i32 %0, 0
-  br i1 %tobool.not, label %cond.false, label %cond.true
-
-cond.true:                                        ; preds = %entry
   %c = getelementptr inbounds nuw i8, ptr %s, i64 8
   %1 = load i32, ptr %c, align 4
   %and = and i32 %1, 255
-  br label %cond.end
-
-cond.false:                                       ; preds = %entry
   %d = getelementptr inbounds nuw i8, ptr %s, i64 12
   %2 = load i32, ptr %d, align 4
   %shr = ashr i32 %2, 3
-  br label %cond.end
-
-cond.end:                                         ; preds = %cond.false, %cond.true
-  %cond = phi i32 [ %and, %cond.true ], [ %shr, %cond.false ]
+  %cond = select i1 %tobool.not, i32 %shr, i32 %and
   %b = getelementptr inbounds nuw i8, ptr %s, i64 4
   store i32 %cond, ptr %b, align 4
   ret void

As we can see, with the dereferenceable attribute the compiler knows more about the memory behind the reference. There is no risk that an early load faults (SIGSEGV) because the pointer is null or points to invalid memory, so the compiler can safely hoist those loads above branches and optimize the code more aggressively.

Sometimes a branch is better, when it is easy to predict: the CPU speculates past it, skips the loads on the untaken side, and overall performance improves.

However, for Verilator with a large RTL design, the branch density is very high, and consecutive executions of the same branch are very far apart. For example, when Verilator evaluates a large RTL design like XiangShan, about 100M instructions execute per simulation cycle, and ~10% of them are branches. Even worse, when the BTB (Branch Target Buffer) inside the CPU cannot hold that many branches, every branch is effectively predicted not-taken. In XiangShan on Verilator, about 50% of branches are taken, leading to frequent mispredictions and a significant performance penalty.

Thus, in such scenarios, minimizing branches and relying on straight-line code with select instructions, as seen in the optimized LLVM IR for references, leads to better overall performance by reducing branch mispredictions.

For these reasons, using references in C++, with the attributes they carry, helps the compiler generate more efficient LLVM IR, which ultimately leads to better runtime performance in scenarios with high branch density.

Is the dereferenceable attribute actually safe?

dereferenceable(16) is a promise that the 16 bytes behind s can be loaded at any time without trapping.

However, C++ promises something weaker: a reference is bound to a valid object when the function is entered. Nothing in the language stops a callee from destroying that object. delete &s is legal if s was allocated with new, and a push_back on a std::vector invalidates every reference to its elements.

So, is the compiler allowed to speculate a load through a reference after a call? Here is a program that is well-defined C++ and tests exactly that:

// f.cpp
struct S { int a, b, c, d; };
void g();
int f(S& s, bool use) {
    g();                              // for all the compiler knows, g() may free the object behind s
    return use ? (s.a & 0xff) : 0;    // s is only touched when use is true
}

// main.cpp
#include <sys/mman.h>
#include <cstdio>
struct S { int a, b, c, d; };
int f(S& s, bool use);
static S* obj;
void g() { munmap(obj, 4096); }       // the object's page is gone after this
int main() {
    obj = static_cast<S*>(mmap(nullptr, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0));
    int r = f(*obj, /*use=*/false);   // well-defined: s is never read after g()
    printf("f returned %d\n", r);
    return 0;
}

f is called with use == false, so after g() unmaps the page the program never reads s again. The C++ abstract machine is happy with this. Now compile f.cpp twice with the same trunk Clang: once as-is, and once with the hidden flag that restores the way LLVM up to version 22 interpreted the attribute.

clang++ -O3 -c f.cpp -o f.o
clang++ -O3 -mllvm -use-dereferenceable-at-point-semantics=0 -c f.cpp -o f_old.o
clang++ main.cpp f.o     -o demo     && ./demo;     echo "exit $?"
clang++ main.cpp f_old.o -o demo_old && ./demo_old; echo "exit $?"

f returned 0
exit 0
exit 139

Exit code 139 is SIGSEGV. The old interpretation hoisted the load above the use test, and the load read the unmapped page:

; f_old.o: how LLVM <= 22 compiled f
        call    _Z1gv@PLT
        movzx   eax, byte ptr [r14]       ; speculative load, runs even when use == false
        test    ebx, ebx
        cmove   eax, ebx

; f.o: LLVM 23.1 and later
        call    _Z1gv@PLT
        xor     eax, eax
        test    ebp, ebp
        je      .LBB0_2
        movzx   eax, byte ptr [rbx]       ; only when use == true
.LBB0_2:

So the honest answer is: the attribute was not safe the way LLVM used it for years, and LLVM fixed that in July 2026 with PR #204795 "Enable dereferenceable-at-point semantics", which shipped in LLVM 23.1. The LangRef now says:

"The dereferenceable attribute only implies dereferenceability at the point of the attribute (i.e. on function entry for arguments or at the point of the call for return values). The underlying object may still get freed after that point. Other attributes such as nofree can be used to exclude frees."

The motivation given in the PR is exactly the C++ case: "C++ does not actually make any guarantees that the underlying memory does not get freed during the execution of the function". The same hole existed for malloc return values, which LLVM also marks dereferenceable.

What "at the point" means in practice (see isDereferenceableAndAlignedPointer in llvm/lib/Analysis/Loads.cpp and willNotFreeBetween in ValueTracking.cpp): when LLVM wants to speculate a load through a dereferenceable argument, it first asks whether the pointer can be freed at all. It cannot if the whole function is nofree, if the argument is nofree noalias, or (new in LLVM 24) if it is nofreeobj. Otherwise LLVM walks backwards from the load to the function entry, and every call on the way must be nofree, nothing on the way may synchronize, and the walk gives up after 32 instructions (a hard-coded MaxInstrsToCheckForFree). One opaque call, or one long enough straight-line block, and the fact is gone.

In the example above, g() is exactly such a call. If g were visible and inferred nofree, or if f called nothing at all, the cmove would still be there.

This has a direct consequence for my Verilator PR. The generated code does not take a reference parameter; instead, it does

auto& vlSelfRef = std::ref(*vlSelf).get();

which works because std::reference_wrapper<T>::get() returns T&, so the call to get() carries dereferenceable(sizeof(T)) on its result, and the SimplifyCFG run that sits right before the inliner uses it. After inlining, the call is gone and so is the attribute, so that one pass is the only chance. Under the at-point semantics a call result is something that "can be freed", so every later load needs the 32-instruction, nofree-only walk back to get(). A Verilator eval function is hundreds of statements long and calls trigger helpers before the first mux, so the walk never succeeds. Compiling the code Verilator generates for a small design with four muxes and a register bank, once as emitted (reference) and once with vlSelfRef. rewritten to vlSelfRef-> (pointer), gives:

With LLVM 23.1 or newer, the std::ref trick buys nothing on this design. I have submitted Verilator PR #8339 to replace the pointer parameter with a reference parameter, which fixes the performance regression.

How x86-APX improves performance further

Everything so far comes down to one problem: a load or store that the program did not ask for may fault, so the compiler needs a proof that it cannot. References supply that proof, and the previous section showed how brittle the proof is. Intel APX (Advanced Performance Extensions) attacks the problem from the hardware side with a new instruction family, CFCMOVcc, which will appear in Intel Nova Lake and Diamond Rapids. The CF prefix means conditionally faulting: it is cmov, but when the condition is false and the operand is in memory, the memory access is not performed and any fault it would have raised is suppressed. The forms, with the names LLVM uses in X86InstrCMovSetCC.td:

Only 16, 32 and 64-bit integer operands exist. With this instruction the compiler no longer needs to know anything about the pointer. The hardware simply does not touch memory when the condition is false, and since it is not a branch, nothing is predicted and nothing can be mispredicted.

Take a simpler mux through a pointer, s->b = s->a ? s->c : 0, and add -mapx-features=cf:

// mux.cpp
struct S { int a, b, c, d; };
void by_ptr(S* s) { s->b = s->a ? s->c : 0; }
void by_ref(S& s) { s.b  = s.a  ? s.c  : 0; }

clang++ -O3 -mapx-features=cf -S mux.cpp

; -O3
_Z6by_ptrP1S:
    cmpl    $0, (%rdi)
    je  .LBB0_1
    movl    8(%rdi), %eax
    movl    %eax, 4(%rdi)
    retq
.LBB0_1:
    xorl    %eax, %eax
    movl    %eax, 4(%rdi)
    retq

; -O3 -mapx-features=cf
_Z6by_ptrP1S:
    cmpl    $0, (%rdi)
    cfcmovnel   8(%rdi), %eax
    movl    %eax, 4(%rdi)
    retq

As we can see, the pointer version now uses the cfcmov instruction, which conditionally moves the value without a branch, improving performance if the branch is hard to predict.

However, you may have noticed that I am using a new file, mux.cpp, for this demonstration instead of the earlier ref.cpp. That is because ref.cpp cannot be optimized with cfcmov yet: its conditional assignment involves two memory accesses, and most of the time branch prediction handles it well enough.

Thus, making the compiler optimize such patterns with cfcmov is left as future work.

Can we make Verilator's case faster even without x86-APX?

Even after replacing pointers with references, we only reduce branches by 8.75% and get a 10.19% improvement in overall performance in Verilator with XiangShan. Moreover, this fragile optimization stopped working two months ago with LLVM's fix, until my recent PR made Verilator's code generation use references fully. Luckily, there is always one more optimization: PGO (Profile-Guided Optimization)!

Profile-guided optimization improves performance further by giving the compiler runtime information about branch behavior, so it can determine more accurately whether a branch is likely to be taken. My findings are:

• With LLVM's -fprofile-instr-generate (frontend PGO), Verilator's XiangShan model gets a ~1.5x speedup.
• With LLVM's -fprofile-generate (backend PGO) or BOLT's -reorder-blocks=ext-tsp, it gets a ~2x speedup with only ~0.2% branch MPKI. BOLT is much faster to apply with hardware branch sampling (Intel LBR or AMD LbrExtV2), since it does not require recompilation as long as the binary was linked with -Wl,--emit-relocs.

For these reasons, profile-guided optimization can significantly amplify the benefit of references and other code optimizations by giving the compiler accurate runtime information about branch behavior.