Mercor Maps AI Benchmark Blind Spots

The map exposes a serious measurement problem: strong benchmark scores may describe a narrow slice of economically valuable work, not broad workplace capability.

• –Only 208 occupations have even one task represented by at least five benchmark prompts.
• –Benchmark coverage favors AI-exposed tasks, high-wage-bill occupations, and highly important tasks.
• –Six of 22 job families have no occupation meeting Mercor’s coverage threshold.
• –The estimate depends on benchmark selection, automated task matching, and modeled time-use data.
• –Developers should treat benchmark scores as task-specific evidence, not proof that an agent can perform an entire job.