90% of US Working Hours Spent on Tasks Poorly Covered by AI Benchmarks
Recent research reveals that 90% of US working hours are spent on tasks with little to no coverage from existing AI benchmarks. This highlights a significant disconnect between how artificial intelligence capabilities are evaluated and the actual work Americans perform. This mismatch makes it exceptionally difficult to determine how proficient AI models truly are at the labor tasks that dominate the economy. Furthermore, the findings suggest that current benchmark prompts disproportionately target higher-wage jobs rather than tasks representative of overall human labor. The study analyzed O*NET tasks and discovered that even after controlling for AI-exposure, occupations with larger wage bills are significantly overrepresented in benchmark evaluations. This indicates a potential bias toward automating higher-value positions rather than comprehensively tracking all forms of work.
## BACKGROUND
AI benchmarks are standardized tests used to measure the reasoning, factual accuracy, and capabilities of large language models (LLMs). O*NET is a database developed by the US Department of Labor that serves as a comprehensive source of occupational descriptors and task data.