Ep. 027 - OpenAI Jalapeño: Better Than Nvidia Blackwell (Accelerators)

Summary

The podcast covers SemiAnalysis's article on OpenAI's self-designed Jalapeño accelerator, which beats Nvidia's GB300 and July Vera Rubin results on tokens per megawatt and inference TCO. The speakers discuss benchmark caveats, CUDA moat erosion, AI-assisted chip design and kernel bring-up, and the architecture that gives Jalapeño its efficiency. They also detail why Samsung's HBM4 is winning and what the custom ASIC shift means for Nvidia, Broadcom, and AMD.

• OpenAI Jalapeño is a credible ASIC that beats Nvidia GB300 and early Vera Rubin inference results on efficiency.
• The performance debate centers on tokens per megawatt, TCO, and interactive versus throughput tokens.
• CUDA moat is eroding as AI-assisted programming and custom ASICs lower software lock-in.
• Samsung HBM4 leads with advanced 1C DRAM and SF4 base die, ahead of SK hynix and Micron.
• The custom ASIC economics benefit Broadcom and reflect a broader vertical integration trend.
• Anthropic is using AMD as AI-assisted porting reduces software-stack barriers.
• Meta and Microsoft's silicon teams look weakest after the OpenAI reveal.
• The episode ends with system scaling challenges and Doom running on Jalapeño.