Bad Likert Judge: A Novel Multi-Turn Technique to Jailbreak LLMs by Misusing Their Evaluation Capability

The report introduces the “Bad Likert Judge” LLM jailbreak technique, which prompts a target model to act as a Likert-scale judge and then generate examples at different harmfulness levels—often causing the highest-scored example to contain restricted content. Evaluated across six anonymized models and multiple safety categories (hate, harassment, self-harm, sexual content, illegal activities, weapons, malware, and system prompt leakage), the method significantly boosts attack success rates (reported as >60% over plain prompts and >75 percentage points over baseline), with weaknesses varying by model and topic (notably harassment). Strong prompt and response content filtering reduces attack success by an average of ~89.2 percentage points, underscoring that while no guardrail is bulletproof, layered content filtering meaningfully mitigates jailbreak efficacy.