AI
AI benchmarks cover only a small share of the work done across the US economy
A Mercor analysis mapping 29 AI benchmarks onto the US Department of Labor’s 923 occupations finds that only 208 occupations have tasks represented by at least five benchmark prompts. It estimates that 90% of American working hours involve tasks with little or no coverage in the selected benchmarks, leaving major gaps in how model capabilities are assessed. The authors stress that the estimate is provisional, shaped by benchmark selection, LLM-based task mapping and limited coverage of physical work.
Editorial summary · Publisher article
What matters
- Benchmarks disproportionately cover AI-exposed, high-wage and high-importance tasks, with software engineering and financial analysis better represented than many other jobs.
- The analysis uses an LLM to map benchmark prompts to O*NET tasks and validates part of that tagging with human experts.
- The authors say current methods remain underpowered for many occupations and do not adequately measure human-AI augmentation or physical tasks.
Context & caveats
- The 90% estimate comes from an initial, non-comprehensive list of 29 benchmarks and may change as coverage expands.
- The analysis is produced by Mercor and relies partly on LLM-derived mappings and working-time estimates from another study.
Why this made the edition
The research supplies a broad dataset, explicit methodology, validation details and extensive limitations, making its central finding useful despite its provisional and publisher-produced nature.
Source material