Tae Kim
· Key Context by Tae Kim
· September 03, 2026 at 00:24
· ⏱ 10 min read
| Read on Substack ↗
Summary
Google is splitting its eighth-generation TPU family into an inference-optimized 8i and a training-optimized 8t, arguing that interconnect bandwidth and latency—not raw FLOPS—are now the key constraints for AI workloads ranging from small agentic serving to 100k+-chip pre-training. The presentation's details on 384MB SRAM, boardfly topologies, a 134,400-TPU Virgo scale-out network, HBM3E and liquid-cooling reliability signal that hyperscaler custom silicon is advancing quickly, with direct implications for AI accelerator, HBM and data-center cooling demand.
•Google says it is now introducing two TPU chips per year instead of one: the inference-optimized TPU 8i and the training-optimized TPU 8t, driven by more varied AI workloads such as large-scale pre-training and small model serving.
•TPU 8i includes 384MB of SRAM, higher HBM capacity/bandwidth, and doubled ICI bandwidth to support low-latency serving of mixture-of-experts and agentic models.
•For inference latency, TPU 8i uses a boardfly network with a maximum of 7 hops versus up to 16 hops in a 3D Torus, and performs collectives in the IO die to achieve 5x faster on-chip reductions.
•The training-focused TPU 8t keeps a 3D Torus because training prioritizes bisection bandwidth, and introduces the Virgo network, a non-blocking scale-out fabric connecting roughly 134,400 TPUs in a single domain.
•Google reports that customers are already training "largest frontier models" on tens of Ironwood superpods and that future demand targets trillion-to-tens-of-trillion-parameter models trained on more than one million TPU chips.
•Google chose HBM3E over HBM4 for the 8t because at 200,000-400,000 chip scale, inter-chip connectivity matters more than per-chip HBM bandwidth; it also highlighted liquid cooling as a way to address HBM temperature-driven failure rates.
Read time10 min
Length10,368 chars
Categoryfinance
Ideas
Tae KimSenior writer, Barron's; author of The Nvidia Way
The article presents concrete evidence of Google's custom TPU hardware advance and says customers are already "training their largest frontier models on tens of Ironwood super pods," supporting Google
The article presents concrete evidence of Google's custom TPU hardware advance and says customers are already "training their largest frontier models on tens of Ironwood super pods," supporting Google Cloud/Alphabet's AI infrastructure credibility.
Risk: TPU hardware success is not the same as Alphabet profitability; AI capex intensity and cloud competition could pressure returns.
Tae KimSenior writer, Barron's; author of The Nvidia Way
Google's TPU 8i "increased the HBM capacity and also the bandwidth" and the 8t remains on HBM3E, showing continued large-scale HBM procurement for AI accelerators — a positive read-through to HBM supp
Google's TPU 8i "increased the HBM capacity and also the bandwidth" and the 8t remains on HBM3E, showing continued large-scale HBM procurement for AI accelerators — a positive read-through to HBM suppliers such as Micron.
Risk: Memory/HBM pricing cycles, customer concentration, and the possibility Google moves to HBM4 in later generations.
Tae KimSenior writer, Barron's; author of The Nvidia Way
The article quotes Google saying "we've been deploying liquid cooling for almost a decade now" and links it to HBM reliability and Arrhenius failure-rate benefits, reinforcing the AI data-center liqui
The article quotes Google saying "we've been deploying liquid cooling for almost a decade now" and links it to HBM reliability and Arrhenius failure-rate benefits, reinforcing the AI data-center liquid-cooling demand narrative that benefits Vertiv.
Risk: No named supplier relationship with Google; hyperscaler self-managed cooling or an AI data-center capex slowdown could limit the benefit.
This newsletter, published September 03, 2026,
features Tae Kim
discussing GOOGL, MU, VRT.
3 trade ideas extracted by AI with direction and confidence scoring.