Gemini confidence gaps expose calibration failure

This looks less like a Gemini-specific bug than a general failure mode of verbalized confidence: these models can sound certain long after the answer quality has fallen off a cliff.

• –Domain-specific questions are exactly where prior knowledge, retrieval gaps, and hallucinated fluency can decouple confidence from correctness
• –Smaller local models may look more cautious, but lower confidence is not the same thing as better calibration
• –The useful metric here is proper calibration error or reliability curves, not whether the model says “high confidence” in a nice-sounding way
• –For production use, the fix is domain evals, abstention, and retrieval checks, not trusting self-reported certainty
• –If this pattern holds across frontier and local systems, confidence should be treated as a UI signal, not a safety guarantee