Ep. 14 - Finding Miscompiles For Fun, Not Profit | Justin Lebar & Jordan Nanos

Summary

Justin Lebar discusses his project of finding compiler bugs in NVIDIA PTXAS, LLVM AMD GPU, and x86 backends using fuzzing and LLM-assisted techniques. He highlights the efficiency gains from modern AI coding tools and the potential for improving software quality.

• Justin Lebar spent $10,000 on a compiler bug-finding project covering NVIDIA PTXAS, LLVM AMD GPU, and x86 backends.
• He used traditional fuzzing and LLM-assisted code reading to identify miscompiles.
• A high-severity x86 bug was found where an atomic operation splits into two non-atomic operations.
• LLMs, especially with Anthropic's Opus 4.8 and UltraCode mode, greatly improved bug detection speed and token efficiency.
• Manual code review for such bugs is nearly impossible; LLMs make it practical.
• Lebar calls for more organizations to use LLMs proactively to find bugs in critical software infrastructure.
• The project is presented as a case study, not a scientific benchmark.
• The conversation touches on the broader potential of LLMs for improving code quality beyond security vulnerabilities.