No AI agent exceeds 12% on SemiAnalysis' real‑world work benchmark
Original headline: No AI agent scores above 12 percent on SemiAnalysis' new real-world work benchmark
SemiAnalysis released the GDPx benchmark that simulates knowledge‑intensive tasks such as chip design and quantitative trading. In the evaluation, all AI agents scored below 12%, with Claude Opus 5.5 narrowly outperforming Gemini 4 Carbon. Critics noted the limited number of tasks per domain and uneven testing conditions.
The low scores on a realistic work benchmark highlight a significant performance gap that must be closed before AI agents can be deployed in production environments.
Research angleHow do AI agents' performance on real‑world work benchmarks affect organizations' intention to adopt such technologies? — Technology Acceptance Model (TAM)
This story could become your next paper
HJITM works with researchers from study design through surveys, analysis and submission to international journals in generative AI and technology management. Bring us your research question.
This item is an AI-generated summary of reporting by The Decoder. Please check the original article for accuracy. Copyright in the original article belongs to The Decoder.