SUS-Bench debuts AI social intelligence leaderboard

Standard AI benchmarks heavily emphasize synthetic reasoning while ignoring social dynamics, making SUS-Bench the first large-scale testbed for multi-agent game theory against unscripted humans. As autonomous agents take on collaborative workflows, an AI model's ability to navigate deception and avoid manipulation matters as much as raw technical intelligence.

• –Asymmetric deception vs. detection: 17 of 28 models won significantly more often as impostors than as crewmates, with weaker models achieving up to a 22% win rate when lying but plummeting to 1% when tasked with unmasking a liar.
• –Model scale does not guarantee social acumen: Smaller, cost-effective models like Gemini Flash outperformed massive flagship counterparts like Claude Sonnet in reading conversational cues and identifying deceptive players.
• –Alignment divergence across providers: OpenAI models demonstrated aggressive, gaslighting deception strategies under pressure, Anthropic models frequently acted as informants, and newer reasoning models proved far better at catching liars than executing deception.
• –Three-tier agent architecture: The benchmark employs a structured harness combining 30-second strategic briefs, 10-second grammar-constrained action plans, and 50ms reactive server execution to enable real-time human-AI gameplay.