ThermoQA benchmark flips thermodynamics rankings

This looks like a genuinely useful eval, not just another leaderboard screenshot: it tests exact numerical reasoning in a domain where small mistakes compound fast.

• –The three-tier design is the right move, because it separates rote steam-table recall from multi-step cycle reasoning
• –Supercritical water and R-134a appear to be the hardest regimes, which suggests current models are brittle around nonlinear property regions and weaker training coverage
• –The run-to-run variance is as important as the mean score; a model that swings a lot is harder to trust in workflows that need repeatable answers
• –Open dataset plus code makes ThermoQA easy to reproduce, extend, and use as a targeted stress test for reasoning agents
• –The benchmark also hints that “knows thermodynamics” and “can solve thermodynamics” are not the same capability