Anthropic outlines 10 safety eval rules

AI safety claims have long suffered from benchmark gaming and uncalibrated metrics, making Anthropic's push for statistical rigor and reproducible evals essential for the ecosystem. By candidly acknowledging that their own single-turn evals are saturated and that LLM graders silently fail, Anthropic sets a much higher standard for safety disclosures.

• –Refusal is a flawed safety proxy: Simply declining to answer fails crisis-intervention scenarios where empathetic engagement and de-escalation are required rather than outright silence.
• –Saturation forces multi-turn testing: With top models achieving 94–98% harmless response rates on single-turn benchmarks, complex multi-turn simulations are necessary to expose gradual behavioral drift.
• –Grader validation is critical: LLM judges must be validated against human inter-annotator agreement and audited for length and position bias rather than trusted as infallible evaluators.
• –Preregistration and uncertainty intervals: Exposing confidence intervals—which often swing ±10% in multi-turn runs—will prevent teams from celebrating marginal score differences that are merely sampling noise.