Contradish flags inconsistent LLM answers

This is a small but genuinely useful category, closer to a unit-test harness for response stability than a vanity benchmark.

• –Semantic variants are a better match for real user drift than exact-match tests, especially when wording changes but intent stays the same
• –CI thresholds turn consistency into a release gate for prompt edits, model swaps, or policy updates
• –The benchmark framing gives teams a shared metric, and the Python API and CLI lower the friction, so it can actually live inside existing eval workflows
• –The strongest fit is support, policy, and agent workflows where contradictory answers are a trust and liability problem
• –It measures consistency, not truth, so it should complement retrieval and grounding checks rather than replace them