What rebuilding AlphaGo teaches us about self-play, RL, and future of LLMs - Eric Jang

Summary

Eric Jang explains how AlphaGo works and how he rebuilt it with modern tools, covering Monte Carlo tree search, policy and value networks, self-play, and reinforcement learning. The conversation contrasts AlphaGo's stable learning signal with the inefficiency of current LLM policy-gradient RL and discusses what automated AI research can and cannot do today. The episode is technical and does not state direct actionable investment trades.

• Eric Jang details Go rules, Monte Carlo tree search, policy/value networks, and how MCTS generates improved per-move training labels.
• AlphaGo's self-play is framed as stable supervised learning on improved labels, avoiding sparse RL credit assignment.
• The discussion contrasts MCTS with modern LLM RL and argues policy-gradient RL is sample-inefficient.
• Eric describes rebuilding AlphaGo with limited compute, using best-response training against KataGo and small-board pretraining.
• Automated AI research is described as strong at running experiments and hyperparameter optimization but weak at choosing research direction.
• The conversation touches on scaling laws, compute efficiency, and whether GPUs and compute scaling dominate algorithmic ideas.