Evaluating and Mitigating Discrimination in Language Model Decisions

This memo presents a method to proactively assess and mitigate discriminatory outcomes from language models in high‑stakes decision contexts by generating diverse prompts across 70 societal decision scenarios and systematically varying demographic attributes; it reports observed positive and negative discrimination in tests (noting results from Claude 2.0), demonstrates that careful prompt engineering can substantially reduce bias, and releases the dataset and prompts alongside a policy memo.