Why CSPs Are Turning to QLC SSDs for KV Cache Storage

∙ Paid 8 Share We’ve shared a series of articles on Substack mapping the memory hierarchy as AI shifted from training to inference, and introduced the SSD POD as a new tier between local SSD and shared storage. This piece looks at what is going into that tier. TLC SSD plus HDD has been the default, but TrendForce has observed some operators adding QLC SSD to the mix. This article explains what is driving that change, and which operators have already begun purchasing QLC for their SSD PODs.

From Training to Inference: A Paradigm Shift in the Memory Hierarchy TrendForce · Jul 22

Prefill: The input prompt is processed in parallel in a single pass. The Key and Value vectors for each layer are computed and stored in the KV Cache, and the first output token is generated at the same time.