MIT Total Uncertainty catches overconfident LLMs

This is not a magic truth meter, but it is a much better abstention signal than asking the model twice and trusting the repeat answer. The useful shift here is treating uncertainty as a cross-model problem, which is exactly where confident hallucinations get exposed.

• –Works on generated text alone, so it can be applied to closed models without logits or hidden states.
• –Best on tasks with a single correct answer, like factual QA, translation, and math; open-ended prompts will stay messy by nature.
• –The paper finds a small, scale-matched ensemble from different companies works best, which is a pragmatic way to estimate epistemic uncertainty.
• –More auxiliary models improve calibration, but in production this still means extra API calls and vendor coordination.
• –Strong fit for selective abstention, routing, and safety checks where a confident hallucination is worse than saying "I’m not sure."