HBM Density Peak: Why AI Value Is Moving From Memory to Connectivity

Damnang · Damnang’s Substack · August 20, 2026 at 16:53 · ⏱ 19 min read  | Read on Substack ↗
Summary
NVIDIA's Rubin Ultra appears to be stepping down from HBM4E 12-Hi to HBM4 8-Hi 192GB because supply tightness and package cost are pushing the optimization point from local memory density to scale-up connectivity. The article argues this makes optics/CPO and memory disaggregation the next value pool, with Marvell, SK hynix and NVIDIA aligned on the same system-level architecture.
  • TrendForce (Aug. 4) reports NVIDIA is evaluating several lower-capacity HBM configurations and the final Rubin Ultra spec is unset; SemiAnalysis sees the main SKU moving to HBM4 8-Hi 192GB with compute held and memory bandwidth slightly up.
  • The article's sensitivity math: moving from 12-Hi to 8-Hi creates a 1.5x stacking-time benefit and 1.084x yield benefit, or ~1.63x effective output ceiling; total HBM revenue lands at ~81.5% of plan if 8-Hi is priced at 50% of planned HBM4E, and ~98% at 60%, assuming all extra stacks become GPU shipments.
  • A Nature Electronics review with an SK hynix corresponding author says compute throughput tripled every ~2 years while interconnect bandwidth advanced only ~1.4x over the same period, pushing the bottleneck from HBM bandwidth to data movement between compute and memory.
  • NVL576 is an 8-rack, 576-GPU NVLink domain with direct optical rack-to-rack connections; SemiAnalysis reads it as NPO, and the paper's roadmap moves optical conversion closer via 2.5D CPO, 3D TSV, hybrid bonding and monolithic integration.
  • Rubin local HBM bandwidth is ~21-22TB/s vs NVLink 6 at ~3.6TB/s per GPU; software prefetch/token-routing keeps performance while weights and KV cache spread to peer HBM.
  • Marvell's August FMS disclosure targets a shared warm KV cache tier up to 32TB over an optical fabric within 50m, with 2-3x token throughput — explicitly vendor targets, not commercial deployment.
  • SK hynix and Marvell co-developed CMM-Ax, combining Marvell Structera A with SK hynix DRAM and software for CXL-attached compute, disclosed in August 2026 as the first concrete electrical disaggregation product.
  • NVIDIA invested $2B in Marvell and announced co-development of NVLink Fusion-compatible scale-up networking and silicon photonics in March 2026; Marvell also acquired Celestial AI's Photonic Fabric in February 2026.
Read time 19 min
Length 19,780 chars
Category finance
Ideas
Damnang Substack author, Damnang’s Substack
Article says Marvell is co-developing NVLink Fusion-compatible scale-up networking and silicon photonics with NVIDIA, co-developing CMM-Ax with SK hynix, and owns Celestial AI's Photonic Fabric — plac
Article says Marvell is co-developing NVLink Fusion-compatible scale-up networking and silicon photonics with NVIDIA, co-developing CMM-Ax with SK hynix, and owns Celestial AI's Photonic Fabric — placing it at the exact intersection of optical I/O and CXL memory disaggregation the article identifies as the next AI value pool. Risk: Much of the optical/memory roadmap is still research or vendor targets; execution and adoption risk remains high.
Damnang Substack author, Damnang’s Substack
Article positions SK hynix's role expanding from DRAM die/HBM to custom base die, CXL/CMM-Ax pooled memory, and eventually a photonic memory pool, which could increase value-added and customer stickin
Article positions SK hynix's role expanding from DRAM die/HBM to custom base die, CXL/CMM-Ax pooled memory, and eventually a photonic memory pool, which could increase value-added and customer stickiness beyond selling memory bits. Risk: HBM stack-height reduction cuts per-GPU bit content; if 8-Hi pricing falls sharply, article's sensitivity shows HBM revenue could fall to ~81.5% of plan.
Damnang Substack author, Damnang’s Substack
The article argues the HBM4 8-Hi 192GB config works because NVL576 scale-up bandwidth and software tiering absorb memory-capacity loss, while compute holds and memory bandwidth rises slightly — valida
The article argues the HBM4 8-Hi 192GB config works because NVL576 scale-up bandwidth and software tiering absorb memory-capacity loss, while compute holds and memory bandwidth rises slightly — validating NVIDIA's system-level architecture despite lower local HBM density. Risk: Final Rubin Ultra spec is unconfirmed; if software/scale-up cannot fully compensate for lower capacity in memory-bound workloads, NVIDIA could face a capability gap.
Damnang Substack author, Damnang’s Substack
Article says DRAM is likely to stay tight through 2027 and lowering stack height raises the number of stacks built from the same DRAM/stacking capacity, so tighter supply and higher stack count can su
Article says DRAM is likely to stay tight through 2027 and lowering stack height raises the number of stacks built from the same DRAM/stacking capacity, so tighter supply and higher stack count can support HBM-dollar revenue even as per-GPU bits fall; Micron is one of the HBM suppliers to that market. Risk: If 8-Hi is priced at 50% of planned HBM4E, article's sensitivity shows industry HBM revenue could fall to ~81.5% of plan, pressuring MU if GPU shipments do not absorb all capacity.
Damnang Substack author, Damnang’s Substack
Article says as optics moves from pluggable through NPO/CPO to photonic interposer, volume growth in optical engines and laser sources comes first; Coherent is a major supplier of lasers and photonic
Article says as optics moves from pluggable through NPO/CPO to photonic interposer, volume growth in optical engines and laser sources comes first; Coherent is a major supplier of lasers and photonic components exposed to CPO/optical interconnects. Risk: CPO timing is uncertain; the paper's architecture is a research direction, not a commercial roadmap, so optical volume may ramp later than implied.
More from Damnang’s Substack

This newsletter, published August 20, 2026, features Damnang discussing MRVL, 000660.KS, NVDA, MU, COHR. 5 trade ideas extracted by AI with direction and confidence scoring.

Speakers: Damnang  · Tickers: MRVL, 000660.KS, NVDA, MU, COHR