Custom Memory, Deeper into the Die, Shallower the Cycle
Damnang
· Damnang’s Substack
· July 13, 2026 at 15:54
· ⏱ 12 min read
| Read on Substack ↗
Summary
The memory business is transitioning from commodity to custom silicon, driven by AI's insatiable bandwidth demands. Compute is moving closer to memory (CIM, PIM, PNM) to reduce data movement, which dampens the traditional memory cycle by replacing spot-market volatility with contract-based margins. The article explains the technology and argues that the shift benefits memory makers with custom design capabilities.
•Memory and compute are separated by the von Neumann architecture due to incompatible process technologies; SRAM is fast but dense, DRAM is dense but can't be on the same die as logic.
•AI inference flips the compute-to-data ratio: each token requires reading all model weights, making memory bandwidth the bottleneck and motivating compute-in-memory solutions.
•The three categories (CIM, PIM, PNM) are distinguished by where compute sits relative to memory cells: CIM inside the array, PIM on the same DRAM die, PNM on a separate adjacent logic die.
•Putting compute inside memory incurs two taxes: a dependency tax (processor and memory become co-designed) and an area tax (logic on a DRAM process is less dense and slower).
•Custom memory breaks the interchangeability of commodity DRAM, allowing suppliers to charge premiums and sign long-term contracts instead of riding spot prices.
•The author quantifies that moving compute closer to memory can halve the data crossing the boundary (e.g., adding three pairs of numbers locally versus fetching all six), effectively doubling usable bandwidth without faster pins.