The Inference Chip Architecture Map: 12 Companies Sorted by Three Bottlenecks

Damnang · Damnang’s Substack · July 01, 2026 at 07:55 · ⏱ 30 min read  | Read on Substack ↗
Summary
The inference chip market is not a monolithic race; it splits into three bottlenecks (memory bandwidth, flexibility cost, deployment/power) and four memory camps (SRAM-centric, low-cost DRAM, tiered, HBM-centric). Startups target gaps that Nvidia and hyperscalers leave, but the durable investment thesis is not in picking individual chip companies but in the common foundation: foundry wafers (TSMC), memory (HBM, DDR5), packaging (CoWoS), and power/cooling infrastructure, which grow regardless of which architecture wins.
  • •Inference has three bottlenecks: memory bandwidth (decode phase reads all weights per token), cost of flexibility (GPU generality wastes cycles), and deployment/power (interconnect, cooling, rack integration).
  • •12 companies sorted by memory camp: SRAM-centric (Groq, Cerebras), low-cost DRAM (d-Matrix, Tenstorrent), tiered (SambaNova, Positron), HBM-centric (Rebellions, FuriosaAI, Etched, MatX).
  • •Groq licensed inference tech to Nvidia for ~$20B in Dec 2025; Cerebras went public in May 2026 with ~$80B intraday market cap.
  • •Tenstorrent in acquisition talks with Qualcomm at $8–10B; d-Matrix's Corsair entered mass production in June 2026 with 90% US customers.
  • •Nvidia's response is system-level: Vera Rubin NVL72 rack bundles GPU, CPU, NVLink, networking; HBM4 triples bandwidth; CUDA ecosystem locks in switching costs.
  • •Hyperscalers' own ASICs (TPU, Trainium, Maia, MTIA) absorb inference internally, with one estimate putting Nvidia's inference share falling from 90%+ to 20–30% by 2028.
  • •SRAM-centric designs shift memory demand from DRAM to foundry silicon; low-cost DRAM camp avoids HBM/CoWoS supply constraints by using DDR5/LPDDR on organic substrates.
  • •The tiered camp (SambaNova, Positron) stacks SRAM, HBM, and DDR, selling multiple memory types per chip; HBM camp (Rebellions, Etched) keeps HBM focus but moves compute toward reconfigurable middle ground.
Read time 30 min
Length 30,555 chars
Category finance
Ideas
Damnang Substack author, Damnang’s Substack
Article identifies multiple memory demand channels: HBM camp (Rebellions, Etched, FuriosaAI) drives HBM demand, 'low-cost DRAM camp carries DDR5 and LPDDR together' (d-Matrix uses LPDDR5X, Tenstorrent
Article identifies multiple memory demand channels: HBM camp (Rebellions, Etched, FuriosaAI) drives HBM demand, 'low-cost DRAM camp carries DDR5 and LPDDR together' (d-Matrix uses LPDDR5X, Tenstorrent uses GDDR6), and tiered camp 'carries HBM and DDR5 together.' Micron is a key supplier of both HBM and commodity DRAM, benefiting from volume growth even as architecture mix shifts. Risk: HBM supply tightness could benefit Micron but also invites competition from SK Hynix and Samsung; pricing cycles remain volatile.
MU
Damnang Substack author, Damnang’s Substack
Article cites an estimate that Nvidia's inference share could fall from 90%+ to 20–30% by 2028 as hyperscaler ASICs (TPU, Trainium, Maia) absorb internal volume, and startups eat into specific bottlen
Article cites an estimate that Nvidia's inference share could fall from 90%+ to 20–30% by 2028 as hyperscaler ASICs (TPU, Trainium, Maia) absorb internal volume, and startups eat into specific bottlenecks. Despite Nvidia's system-level response, the structural threat from custom chips is quantified. Risk: Nvidia's CUDA moat and rack-level integration may slow the erosion; actual share decline depends on hyperscaler adoption pace and startup exits.
NVDA
Damnang Substack author, Damnang’s Substack
Article emphasizes that SRAM-centric designs (Cerebras, Groq) 'send volume to the foundry' because on-chip SRAM is printed on logic wafers, not separate DRAM. Even as startups compete, they all pass t
Article emphasizes that SRAM-centric designs (Cerebras, Groq) 'send volume to the foundry' because on-chip SRAM is printed on logic wafers, not separate DRAM. Even as startups compete, they all pass through TSMC for advanced nodes (N4P for Etched, wafer-scale for Cerebras). The common foundation 'grows with it' regardless of architecture winner. Risk: Geopolitical risk or capacity allocation shifts; TSMC's exposure to demand swings in AI chip startups.
TSM
Damnang Substack author, Damnang’s Substack
Article states hyperscalers' own ASICs (Amazon's Trainium) are growing at mid-40% annual rate, absorbing inference internally. Amazon deployed 'more than a million Trainium2 chips' for Anthropic and B
Article states hyperscalers' own ASICs (Amazon's Trainium) are growing at mid-40% annual rate, absorbing inference internally. Amazon deployed 'more than a million Trainium2 chips' for Anthropic and Bedrock. This internal volume is a direct revenue/cost-savings driver for Amazon, reducing dependency on Nvidia and capturing more AI workload margin. Risk: Custom chip development costs and execution risk; Nvidia's CUDA ecosystem may retain some workloads.
AMZN
Damnang Substack author, Damnang’s Substack
Article's third bottleneck is 'deployment and power' — how chips fit into data-center racks, cooling, and power handling. All startups converge on rack-level systems, and Nvidia's Rubin NVL72 is a com
Article's third bottleneck is 'deployment and power' — how chips fit into data-center racks, cooling, and power handling. All startups converge on rack-level systems, and Nvidia's Rubin NVL72 is a complete rack platform. This drives demand for thermal management, power delivery, and data-center infrastructure, which Vertiv supplies (cooling, power distribution, CDUs). While not named, the thesis directly maps to VRT's AI exposure. Risk: Vertiv's valuation already reflects AI data-center buildout; downside if startup deployment disappoints or Nvidia's own rack integration displaces third-party parts.
VRT
More from Damnang’s Substack
Lumentum Is Starting Its Second Engine
Jun 29, 01:24
AAOI: Down 37% From Its Peak. Opportunity or Trap?
Jun 24, 09:56
SPHBM4: The HBM Packaging Playbook Is Changing
Jun 22, 07:54
How the Memory Tax Gets Solved
Jun 19, 01:01
Coherent: The Company That Gets Stronger in the CPO Era
Jun 17, 07:24

This newsletter, published July 01, 2026, features Damnang discussing MU, NVDA, TSM, AMZN, VRT. 5 trade ideas extracted by AI with direction and confidence scoring.

Speakers: Damnang  · Tickers: MU, NVDA, TSM, AMZN, VRT