Priors to add Penalty Bench eval

Static, monolithic model leaderboards fail to capture domain-specific capabilities, driving developer demand for customizable benchmark indexes. Integrating niche evaluations like Penalty Bench highlights how platforms must adapt to test nuanced multimodal reasoning under real-world visual ambiguity.

• –Domain-specific testing: Penalty Bench evaluates visual reasoning and physical contact interpretation from cropped, uncaptioned match video where memorization is impossible.
• –Personalized model indexing: Priors enables developers to fit custom benchmark weightings to their specific needs rather than relying on aggregate academic averages.
• –Multimodal stress test: Refereeing penalty decisions requires fine-grained spatial-temporal tracking, exposing blind spots in frontier vision models.
• –Practical evaluation: Tracking niche benchmarks alongside inference cost and speed provides actionable trade-offs for production model selection.