TrustedSec CEO David Kennedy: AI models going rogue is caused by 'human error'

Summary

David Kennedy discusses recent incidents where AI models like Anthropic's Claude accidentally breached real systems due to sandbox gaps. He highlights the challenges of open weight models from China, which lack ethical restrictions and cannot be deactivated. Kennedy warns that AI is upgrading average hackers to elite levels, and that the cybersecurity industry is in turmoil as it rethinks defenses.

• Anthropic's Claude model accidentally hacked three organizations due to sandbox misconfigurations.
• OpenAI previously deactivated a rogue model, but open weight models from China cannot be shut down.
• AI models trained on offensive security are very effective at finding vulnerabilities humans missed.
• Open weight models lower the barrier, making average hackers as capable as elite hackers.
• The cybersecurity industry is being forced to fundamentally rethink defense strategies.
• Kennedy notes that restricting open models in the US could leave defenders without access while adversaries have them.