From data to deployment: A deep dive into building our AI Resolutions (part two)

This report presents AI Resolutions, an LLM-driven approach to automatically generate precise and defensible close comments for benign security alerts by leveraging structured feature engineering from Expel Workbench evidence, robust evaluation metrics (grade, completeness, BERT F1, correctness, errors, situational awareness, adoption), and a hybrid prompt design combining Chain-of-Thought and Self-Reflection. It documents extensive experimentation and model comparisons, human analyst annotation workflows for oversight, and operational safeguards to prevent data leakage, cross-customer data mixing, and prompt-hacking to ensure reliable, secure analyst adoption.