Unearned Confidence: AI Security Reviewers Don’t Really Get It

This Checkmarx post evaluates AI-based security review reliability by testing Claude Opus 4.6 against OpenEMR patches for two CVEs: a file-upload fix (CVE-2022-4506) and an XSS fix (CVE-2022-4733). The model produced a false positive about the file-upload hardening and suggested bypasses for the XSS patch (one of which worked in testing), illustrating that LLMs can sound authoritative yet miss contextual details like whitelisting and default configuration; the author concludes AI findings should be treated as hypotheses, not final verdicts.