Why next-gen AI scale-up needs CPO

Summary

The video explains the trade-offs between copper and co-packaged optics (CPO) for AI datacenter networking. It covers scale-up and scale-out networks, current limitations of copper at high speeds, and the technical benefits of CPO in reducing power and latency. The speaker discusses industry adoption dynamics, noting that hyperscalers are cautious due to vendor lock-in while neoclouds embrace CPO, and concludes that copper will be used as long as possible but CPO will eventually dominate scale-up networks.

• Copper is used for short-distance scale-up networks (e.g., Nvidia NVL72).
• Pluggable transceivers require DSPs that consume 60%+ power and 90%+ of latency.
• CPO eliminates the DSP by placing optical engines closer to the ASIC.
• CPO for scale-out reduces latency and energy, but hyperscalers worry about vendor lock-in.
• Neoclouds prefer turn-key CPO solutions (e.g., Nvidia CPO switches).
• Next-gen 448G interconnects push copper to its limit, requiring CPO or higher PAM levels.
• Nvidia's Vera-Rubin NVL576 uses a mixed copper-optical scale-up network.
• Copper will be used until its physical limits, after which CPO will take over scale-up.