Project Naptime: Evaluating Offensive Security Capabilities of Large Language Models

Project Zero presents 'Naptime', an LLM-powered vulnerability research framework that equips models with specialised tools (code browser, Python interpreter, debugger, reporter) to iteratively discover and verify memory-safety bugs. Using this architecture the authors significantly improved results on the CyberSecEval2 benchmark for buffer overflow and advanced memory corruption tests and demonstrate a concrete proof-of-concept buffer overflow exploit in a sample C++ challenge, validated with AddressSanitizer.