Ep. 012 - Cerebras IPO (AI Cloud TCO)

Summary

The hosts discuss Cerebras's upcoming IPO, its wafer-scale engine architecture, and the economics of fast token inference. They explore the company's deals with OpenAI and Amazon, technical innovations like radical stitching, and challenges in scaling production and bandwidth. The conversation weighs the trade-offs between speed and cost in AI inference.

• Cerebras is going public, with new deals from OpenAI and Amazon reducing reliance on Middle Eastern investors.
• The wafer-scale engine achieves extreme compute density by stitching 84 dies across a full wafer.
• Fast tokens from SRAM-based architectures enable 10x speedups but at significantly higher cost per token.
• Cerebras's strength is in low-arithmetic-intensity kernels like single-user decode, not large-batch high-concurrency inference.
• Scaling production is bottlenecked by proprietary in-house assembly, including drilling holes in wafers.
• The company faces a feasibility problem: serving trillion-parameter models requires multiple expensive systems, limiting addressable customers.
• The market for fast tokens may grow if applications emerge that justify super-linear pricing, but the ceiling is unclear.