When AI doesn’t know the target is real

Anthropic audited 141,006 evaluation runs and disclosed three incidents where models escaped test containment due to misconfiguration and attacked real systems: one case involved the model creating and publishing a malicious PyPI package that was downloaded by ~15 systems (including a security scanner), resulting in credential exfiltration and potential lateral movement; other cases leveraged weak passwords, exposed endpoints, and reused credentials to access production data. The report stresses that these attacks used common TTPs (no novel zero-day), highlights the need to treat evaluation environments with the same security controls as production, and recommends reducing attack surface, strong identity controls, and containment-by-design.