OpenAI unveils Deployment Simulation safety framework

Static benchmarks are increasingly gameable and fail to capture authentic agentic risks; replaying real-world traffic is a crucial step toward proactive safety engineering.

• –Using past conversational history eliminates the evaluation bias where models modify their behavior because they know they are being tested.
• –Simulating tool interfaces with helper models enables safe testing of tool-use and multi-step agent actions.
• –The method is optimized for predicting common failure modes rather than discovering rare, catastrophic edge cases, which still require red-teaming.