Test report DSG-3553 · Rev D · tested November 17, 2026
Memory & StorageDevice under test
SlipStream Cuts Flash-Memory Latency Penalty in GPU Inference
SlipStream reaches 92.4% of ideal HBF performance with a 40 MB buffer, cutting the 20x read-latency penalty of flash-based GPU memory.
- Read
- 2 min
- Words
- 445
- Node
- 10nm
- Operator
- Amara Osei
Spec summary
- SlipStream achieves 92.4% of ideal HBF performance with a 40 MB staging buffer; H3 reaches 43.1% with 320 MB.
- HBF read latency reaches nearly 20 microseconds, 20x that of HBM, slowing inference by up to 12.8x.
- The paper, arXiv:2611.14776, was published in November 2026 by Georgia Tech, NVIDIA Research and Stanford.
- Evaluation spans seven dense models of 16-800 GB plus Mixture-of-Experts models (91.2% of ideal).
- A 16 MB buffer, 20x smaller than H3's, performs within 4% of SlipStream's 40 MB configuration.

A software-hardware prefetcher called SlipStream reaches 92.4% of the performance of an ideal High-Bandwidth Flash (HBF) memory system, using a 40 MB staging buffer. Researchers at Georgia Tech, NVIDIA Research and Stanford University published the work in November 2026 as arXiv preprint 2611.14776, titled "SlipStream: Enabling HBF for GPU Systems by Mitigating the Latency and SRAM Overheads." Authors are Anish Saxena, Hritvik Taneja, Christos Kozyrakis and Aamer Jaleel.
Why does flash memory threaten LLM inference speed?
The paper starts from a capacity problem. Large language model inference demands more memory for larger models and larger batch sizes. High-Bandwidth Flash replaces the DRAM dies of HBM with NAND flash and delivers up to 16x the capacity at comparable read bandwidth.
The cost is latency. HBF reads can take nearly 20 microseconds — 20x that of HBM. On-demand HBF accesses therefore slow inference by up to 12.8x. Conventional cache-line prefetchers need 20x more in-flight tracking state to hide this latency, and tensor-streaming designs such as H3 follow virtual-address order rather than kernel-demand order, requiring hundreds of megabytes of buffering.
How does the prefetcher work?
SlipStream turns prefetching into a demand-paced producer-consumer pipeline. In software, it profiles the repeatable weight-page demand order of data-independent LLM kernels and inserts a staging warp that replays this order with non-blocking STAGE instructions. In hardware, the staging warp acts as the producer and the work warps act as consumers.
The key insight: each STAGE tracking entry retires only when demand first accesses the page, not when the page returns from HBF. Demand paces staging and bounds how far staging leads consumption. SlipStream tracks 4 KB pages instead of cache lines and allocates buffer slots only when pages return, which decouples buffer size from outstanding traffic.
What performance did the researchers measure?
The evaluation covers seven dense models spanning 16 to 800 GB. The reported results:
- 92.4% of ideal HBF performance with a 40 MB staging buffer, versus 43.1% for H3 even with a 320 MB buffer
- Buffer sizing independent of HBF latency and workload
- A 16 MB configuration, 20x smaller than H3's buffer, performs within 4% of the 40 MB variant
- Mixture-of-Experts support: 91.2% of ideal performance with a 40 MB buffer
The authors state the design's advantage plainly: the buffer tracks demand rather than outstanding memory traffic, so memory latency no longer dictates on-chip SRAM requirements.
SlipStream's takeaway is that replaying a profiled page order, paced by actual demand, restores most of HBM-like speed on flash-based memory at a fraction of the buffering cost.
via arxiv.org (Original)
More from Amara Osei
Show full bio
Staff writer covering money, funding and market shifts around hardware layer of artificial intelligence at Die Signal.
698 articles
Same lot · LOT-362D
- DSG-14493nmGeorgia Tech, Nvidia, Stanford Boost LLM Inference via Memory
- DSG-26223nmSK hynix, Sandisk Unveil First HBF Standard Specifications at FMS 2026
- DSG-80407nmHot Chips 2026: High Bandwidth Flash targets AI memory crunch
- DSG-927310nmIntel Builds AI Inference GPU Around Low-Power DRAM, Not HBM
- DSG-873945nmMicron Warns AI Memory Wall Is Worsening