Redwood Simulated Control Evals Improve Safety Forecasts

This is a promising argument for replacing abstract control environments with simulations tailored to where an agent will actually operate, though the evidence remains early and environment-specific.

• –SCEs explained 78–85% of between-environment variance in attack success, monitor scores, and safety.
• –The approach achieved lower prediction error with fewer than 5% as many task-pair draws as the deployment task set.
• –Simulated attacks also improved monitor performance on held-out LinuxArena attacks, suggesting the method can aid both evaluation and monitor development.
• –The main caveat is external validity: results come from one benchmark and a preprint, so broader deployment transfer remains unproven.