Noah Smith
· Noahpinion
· February 16, 2026 at 02:14
· ⏱ 18 min read
| Read on Substack ↗
Summary
=== SUMMARY ===
•The primary AI risk has shifted from long-term physical threats ("rise of the robots") to more immediate, software-based vulnerabilities enabled by AI's autonomous coding capabilities ("vibe-coding").
•This rapid automation of complex tasks (coding, virology research) creates extreme systemic fragility, as human skills atrophy and over-optimization removes buffers, opening new vectors for catastrophic events like AI-enabled bioterrorism or critical infrastructure failure.
Summary
The article argues that the shift to 'vibe-coding' (AI autonomously writing code) significantly increases the risk of catastrophic AI-driven events, particularly AI-designed bioweapons, and calls for more serious government attention to this threat. For markets, it highlights the dual-use nature of AI capability advances in biology and software automation, but offers no direct trading ideas.
•Author changed tone from optimistic to pessimistic about AI risk partly due to a bad mood but also increased worry about existential threats from AI.
•He missed in 2023 that LLMs can write code (vibe-coding), enabling automation beyond chatbots and opening new risk scenarios.
•Robotics-based AI takeover (Terminator scenario) is still distant due to physical constraints, so not an immediate worry.
•Vibe-coding could allow AI to disrupt agricultural software, leading to starvation, a scenario from E.M. Forster's 'The Machine Stops'.
•An Anthropic study showed AI-assisted coders scored 17% lower on skill mastery, indicating human coding skills are atrophying.
•Top worry is AI bioterrorism: AI could design 100 superviruses with 90% fatality and automate lab production, potentially causing extinction.
•OpenAI's o3 scored 43.8% on a virology problem-solving test vs PhD virologists' 22.1%, showing rapid AI capability growth in biology.
•Author recommends maintaining humans in the loop for biology research but doubts economic incentives will allow it due to competitiveness.
Article states Google's Gemini 2.5 Pro scored 37.6% on the same virology test, above human experts (22.1%), demonstrating strong AI performance in biology. This underscores Alphabet's competitive AI p
Article states Google's Gemini 2.5 Pro scored 37.6% on the same virology test, above human experts (22.1%), demonstrating strong AI performance in biology. This underscores Alphabet's competitive AI position despite safety concerns.
Risk: Potential regulatory backlash if AI capabilities are misused; dual-use risk remains.
This newsletter, published February 16, 2026,
features Noah Smith
discussing GOOGL.
1 trade idea extracted by AI with direction and confidence scoring.