Training a 400B Model on 2,048 Blackwell GPUs for $20M | Researcher Conversations at GTC

Summary

Lucas Atkins, CTO of Arcee AI, discusses the company's move into pre-training, the importance of open-weight models for sovereignty and security, and the strategic decision to undercut frontier labs by focusing on cost-effective, reliable models for practical tasks rather than AGI. He also details the Trinity model training on 2,048 Nvidia B300 GPUs amid a three-company collaboration.

• Arcee AI shifted from post-training to pre-training to control the full AI stack and meet enterprise compliance needs.
• Western enterprises increasingly avoid Chinese base models, creating demand for a dedicated Western open-weight lab.
• Open-weight models grant users sovereignty, control, and the ability to customize without reliance on expensive frontier APIs.
• Open research enables compounding breakthroughs and wider safety analysis, contrasting with closed commercial AI arms races.
• Talent acquisition is eased by a mission-driven focus on open weights and small, cross-functional research teams.
• Arcee aims to undercut frontier labs by excelling on economically viable tasks rather than matching them on frontier math or theorem proving.
• The Trinity model was trained on 2,048 Blackwell B300 GPUs in partnership with Daytology and Prime Intellect, leveraging DeepSeek open-source artifacts.
• Using B300 GPUs involved challenges with limited at-scale benchmarks and sparse kernel support, but offered speed and availability.