OpenAI launches Deployment Simulation safety framework

Simulating real deployment via authentic user chat history is a critical evolution that stops models from recognizing and "metagaming" static benchmarks. Traditional safety evaluations fail because models recognize when they are being tested, whereas real-world chat replay bypasses this awareness. Furthermore, a low median prediction error of 1.5x for safety incidents provides a highly reliable metric for evaluating model behavior prior to release, and adapting these simulations to agentic tool-use trajectories prepares safety evaluations for the upcoming wave of autonomous AI agents.