Broken Hill : An Automated Penetration Testing Tool To Trick AI Chatbots

Broken Hill, released by Bishop Fox, is an automated tool that uses the Greedy Coordinate Gradient (GCG) attack to produce tailored prompts capable of circumventing LLM safety restrictions; the report outlines its capabilities, demonstrations (including bypassing Phi-3), potential use for penetration testing and research, and ethical considerations regarding responsible use and disclosure.