Nvidia Vera: 1.228 TB/s Memory, 164 MB L3 and the Cost of Latency

By Amara Osei

Nvidia's Vera server chip moves up to 1.228 TB/s of DRAM bandwidth through a 1024-bit LPDDR5X-9600 interface — more than any DDR5-based peer — while holding memory latency under 200 ns even at full bandwidth load. That combination, measured with SPEC CPU2026, defines the chip's position: fewer, faster cores fed by a memory subsystem that no density-oriented competitor matches.

The design carries tradeoffs. Cross-socket memory latency reaches 309.27 ns, core-to-core latency across sockets sits near 660 ns, and Vera falls behind Amazon's Graviton 5 at the socket level in cache-friendly throughput.

What feeds the Olympus cores?

Vera's second-generation Scalable Coherency Fabric (SCF) connects cores, cache and off-chip I/O, filling the same role as Arm's CMN mesh or Intel's server mesh. Like the first-generation SCF in Grace, Vera's fabric most likely uses a mesh topology.

Three node types make up the fabric:

• Coherency Switch Nodes (CSNs) host cores and cache. Each CSN carries a pair of cores, keeping node count down — the same tactic Ampere used in Altra Max, where each fully coherent request node hosts a DynamIQ Shared Unit with two Neoverse N1 cores.
• Memory Switch Nodes (MSNs) bridge SCF to the LPDDR5X controllers, delivering up to 1.228 TB/s.
• Bridge Switch Nodes (BSNs) carry 88 PCIe lanes (665.5 GB/s total) plus the cross-socket NVLink C2C link, rated at 1.8 TB/s theoretical bandwidth.

In short, Nvidia built a chip-wide network that prioritizes bandwidth over node count.

What changed since Grace?

Grace pioneered LPDDR5X in servers but delivered mediocre DRAM results: memory latency above 200 ns and bandwidth below contemporary DDR5 parts. Vera reverses both figures. Latency now matches DDR5-based setups, and a pointer-chasing thread stays under 200 ns even while other threads saturate the memory subsystem — no "hockey stick" latency spike near the controller's limit.

Vera also moves to SOCAMM2 modules, making memory field-serviceable and potentially upgradeable. Grace used soldered memory, where a failed chip meant trouble. Nvidia's latency control under load should also dampen noisy-neighbor effects and let Vera out-scale rivals in DRAM-bound workloads despite fewer cores.

How much cache does Vera have?

A 164 MB L3 is shared across all cores and kept non-inclusive with the 2 MB core-private L2 caches, yielding up to 340 MB of effective last-level capacity when no data is duplicated. Frequently shared addresses are stored once in L3, saving capacity.

L3 read bandwidth totals roughly 3.35 TB/s, or 33.1 GB/s per core. A single uncontended core reaches 88.5 GB/s, so contended bandwidth drops significantly — but the 2 MB L2 caches should absorb much of the demand. AMD takes the opposite approach: Genoa-X delivers 90 GB/s of L3 bandwidth to one core and 85 GB/s with all cores loaded, but splits 1248 MB of L2+L3 into octa-core clusters. A single Genoa-X thread can only fill its 96 MB L3 instance; a Vera thread can theoretically use the entire 164 MB.

How does the cross-socket link perform?

NVLink C2C links two Vera sockets into a dual-NUMA system. Cross-node memory reads land at 309.27 ns load-to-use — 177.41 ns more than a local access. Reaching that figure oddly requires a few dozen GB/s of traffic; an isolated pointer-chasing test measures over 370 ns, suggesting the link only shifts into a higher-performance state once requests back up.

Bandwidth is class-leading: reads exceed 587 GB/s, and a read-modify-write pattern reaches 780 GB/s. That is more cross-socket bandwidth than most servers have to their own local memory. Latency is the weak spot — AMD's Genoa-X handles cross-socket accesses in 222.7 ns, and latency spikes to 780 ns under full read-modify-write load.

What about latency between cores?

Within a socket, a bouncing-atomic test averages 148.5 ns, ranging from 87.5 to 199.3 ns depending on the core pair; SMT siblings stay under 10 ns. That intra-socket latency is consistent — an advantage Nvidia cites in its whitepaper — but consistently high, worse than Grace and worse on average than Graviton 4 and Genoa-X.

Across sockets, roughly 660 ns. Genoa-X (EPYC 9V33X) achieves about 209 ns, and Graviton 4 does even better.

What do SPEC CPU2026 numbers show?

Running one copy per hardware thread, Vera beats Graviton 5 node-for-node, and also beats a lower-core-count Zen 5 part configured with one NUMA node spanning both sockets. At the socket level, Graviton 5 wins — it has more than twice the cores, and density-optimized designs deliver higher throughput when enough work exists to fill them.

Bandwidth-heavy tests flip the result. In 749.fotonik3d, a workload that saturates DRAM bandwidth even single-threaded on a Ryzen 7 9800X3D, Vera holds nearly a 20% lead over Graviton 5 despite the core-count deficit, with the advantage extending to several other SPEC CPU2026 floating-point tests.

Vera trades core count for per-core performance, and for jobs that must finish fast on partly loaded machines, that trade pays. For the Arm server market, it adds a second axis of competition beyond density.