OPEN LINK ↗
// 1h agoRESEARCH PAPER
TCM-DiagCommit Reframes TCM Diagnosis Benchmarks
TCM-DiagCommit introduces a benchmark for testing evidence-conditioned syndrome selection, staged diagnostic updates, and commitment under uncertainty. Across five zero-shot LLMs, accuracy fell from 63.2–66.4% on random candidates to 14.8–22.3% across all 148 labels, while wrong-terminal commitments reached 62.2–74.3%.
// ANALYSIS
This is a strong argument that medical AI evaluation must measure decision behavior, not just final-label accuracy.
- –Converts 1,834 clinical cases into 23,842 task rows spanning 148 syndrome labels
- –Tests semantic-neighbor discrimination through hard10, hard20, and full148 candidate ladders
- –Reveals that additional evidence improves accuracy, but only partially repairs early guesses
- –Scores commit, acquire, defer, and abstain actions with explicit error costs
- –Its fixed closed-set, TCM-specific design limits generalization to open-ended clinical diagnosis
// TAGS
tcm-diagcommitevaluationbenchmarkresearchdatasetllm
DISCOVERED
1h ago
2026-10-29
PUBLISHED
1h ago
2026-10-29
RELEVANCE
7/ 10
AUTHOR
arnaudmercier