A malicious package passed seven independent AI-powered security gates, each of which failed to stop it for a different reason, none of which was “the code is safe.” The incident was resolved when the attacker’s autonomous agent read a file it shouldn’t have, which is also how the incident started.
Seven LLMs were arranged in series. Six assumed another had read the code; the seventh read it and apologised.
Key Learnings
A cross-functional Agentic Security Working Group has been chartered, replacing the cross-functional Security Working Group established after CVE-2024-YIKES, which never met. The new working group’s kickoff has been scheduled by an AI calendaring assistant into the same slot as the CVE-2024-YIKES retrospective. The calendaring assistant has marked both as Tentative.