Why the pentesting playbook doesn’t fit: belief, assumptions, and non-determinism

This post explains why conventional security testing and pentesting approaches fail for AI systems: unlike deterministic software, AI behavior depends on training data, inference-time context, and non-deterministic generation, which makes exhaustive input/output testing ineffective. It highlights prompt injection as a prevalent and hard-to-solve risk, describes the need to test behavioral bounds and defense-in-depth (guardrails, output checks, least privilege), and situates the guidance within evolving regulatory expectations (EU AI Act, NIST AI RMF). The series will follow with concrete test guidance, examples, and a threat/coverage model.