GardenBench Tracks Daily LLM Drift

GardenBench’s strongest idea is treating model evaluation as observability rather than a launch-day scorecard.

• –Runs 105 image and text tests across 11 practical capability suites.
• –Daily measurements reveal performance fluctuations, model drift, and ranking changes after deployment.
• –The full benchmark costs about $50 and takes up to an hour per day, making recurring evaluation accessible.
• –Exact-match scoring keeps results interpretable, but limits nuanced assessment of open-ended responses.
• –The unpublished dataset improves resistance to contamination while reducing reproducibility for outside researchers.