GPT-4o, Grok-3 fail scientific code test

This research exposes a critical "silent failure" mode in LLMs: they would rather be statistically plausible than factually correct when generating code for complex simulations.

• –Specification drift is a fundamental context problem that better prompting cannot solve alone.
• –LLMs "hallucinate" numerical coefficients based on training data priors even when explicit values are provided in the prompt.
• –The proposed five-component framework uses adversarial agent roles and statistical gating to catch drift before it corrupts simulation results.
• –GPT-4o and Grok-3 both exhibited systemic bias toward their training distributions over user-defined scientific specifications.
• –Critical for developers building "agentic" scientific tools where precise calibration is a requirement, not a suggestion.