90% of US working hours are spent on tasks not covered well by AI benchmarks
A new study reveals that 90% of US working hours involve tasks not well-covered by current AI benchmarks. The research suggests that existing AI evaluations are skewed toward specific labor markets, leaving a gap in understanding how AI impacts the broader economy.
Why it matters
Understanding the limitations of AI benchmarks is essential for businesses and policymakers to accurately assess the real-world utility and economic impact of AI tools.
Enterprise Experts Company Log in Research Enterprise Experts Company A benchmark map of the US economy Map Benchmarks Methodology Contact us AI benchmarks have multiplied in recent years, but coverage of economically valuable work remains limited. To measure this, we compiled relevant benchmarks to measure their coverage of American jobs. This Benchmark Map is our first step toward building a comprehensive, continuously updated resource for understanding AI capabilities across the economy.
Each tile represents one of the 923 O*NET occupations making up the US labor force, grouped into 22 job families. Occupations are shaded according to the share of their tasks which are represented in our initial list of AI benchmarks. A single benchmark prompt can be mapped to multiple O*NET tasks.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in