Hyperscalers are implementing techniques that could compress memory usage by up to 40x

Everyone is continuing to pile into DRAM etf, MU, and SNDK right now after a monster earnings report by MU. The primary driver of the revenue, earnings, and margin growth has been the drastic increase in DRAM/ NAND prices amidst increasing demand for memory as part of the AI buildout. As a result, memory companies have been a direct beneficiary of the insane CapEx spending being done by the hyperscalers. The new narrative has been that the historically cyclical subsector will no longer be cyclical moving forward, as there will now be a constant, continuous demand for memory as chips/ data-centers continue to evolve. Again, an interesting part about Micron’s earnings was that shipment growth did not drastically drive revenue growth (shipments even decreased in some business units), but the massive increase in ASPs is what drove the blowout sales #s. Now DRAM/ NAND flash spot prices have already seemed to top out, and they’ll likely continue to move downward as several Chinese memory players have been growing their presence in the market. Additionally, in past DRAM cycles, memory stock prices peaked about 5-8 months before actual DRAM prices peaked. But what’s not discussed AS MUCH in the “cyclicality is dead” debates is the measures that the big memory **buyers** have been working on and planning to implement/ build upon to optimize their memory usage moving forward. The attached table shows techniques developed by US big tech companies that could compress memory usage by 20-40x. A brief description of NVDA’s KV Cache Transform Coding (KVTC) method: *KVTC is a method that borrows from classical media compression to dramatically reduce the size of the Key-Value (KV) cache in Large Language Models (LLMs). By applying PCA and entropy coding, it shrinks memory demands up to 20× (and up to 40× for certain use cases) without modifying model weights*