Constitutional AI: Harmlessness from AI Feedback

This memo presents "Constitutional AI," a training framework where AI systems supervise other AIs using a set of rules to produce harmless, non-evasive assistants. It describes a supervised phase of self-critiques and revisions followed by an RL phase that trains on model-generated preferences (RLAIF) to reduce harmful outputs without human labels.