NVIDIA and Lakera AI Propose Unified Framework for Agentic System Safety

Researchers from NVIDIA and Lakera AI propose a unified framework for securing agentic AI systems, emphasizing safety as an emergent property of interactions among models, orchestration, tools, and data. The work introduces AI-driven red teaming to probe agents for risks like prompt injection and tool misuse, arguing that traditional security approaches (e.g., CVSS) are insufficient for these systems. They also release the Nemotron-AIQ Agentic Safety Dataset 1.0, containing 10,000+ traces to advance evaluation and mitigation of agentic AI risks.