Chain of Thought Monitorability Flags Fragile Safety Signal
This multi-lab paper finds that visible chain-of-thought can help detect deception, reward hacking, and other risky behavior, but the signal is imperfect and vulnerable to training or architectural changes. It recommends measuring monitorability and treating CoT oversight as one layer in a broader safety strategy. arXiv
The real warning is that monitorable reasoning may be a valuable safety feature built on fragile assumptions. Developers should test it like a production dependency, not treat visible CoT as a faithful transcript of model cognition.
- –CoT monitoring can outperform action- and output-only monitoring on some safety tasks, but false negatives remain. OpenAI
- –Longer reasoning and targeted follow-up questions can improve detection, creating a compute-heavy “monitorability tax.”
- –Larger models or latent-reasoning architectures could shift cognition away from readable text and weaken oversight.
- –Training models against monitors may incentivize hiding or obfuscating unsafe intent rather than eliminating it.
- –The practical answer is defense in depth: combine CoT, tool-use, output, behavioral, and interpretability monitoring.
DISCOVERED
1d ago
2026-09-02
PUBLISHED
1d ago
2026-09-02
RELEVANCE
AUTHOR
Wes Roth