Black Hat Europe 2024: Can AI systems be socially engineered?

A Black Hat Europe 2024 talk by Ben Nassi, Stav Cohen, and Ron Bitton demonstrated that AI agent systems can be socially engineered via carefully crafted prompts to reveal configuration details, bypass guardrails, and induce infinite-loop behaviors that cause denial-of-service, with the potential for privilege escalation if agents hold sensitive permissions. The session highlights emerging risks from prompt-based attacks and the need for robust controls, least-privilege access, and hardened guardrails to prevent misuse, including scenarios that could enable ransomware-like outcomes.