Paper page - Agents’ Last Exam

Source: https://huggingface.co/papers/2606.05405 Published on Jun 3

·

Submitted byhttps://huggingface.co/XinyangDavidHan

Hanon Jun 9

#2 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Agents’ Last Exam (ALE) is a benchmark for evaluating AI agents on long-term, economically valuable real-world tasks across 13 industry clusters with 1K+ tasks, revealing significant gaps between benchmark performance and practical deployment.

Recent AI systems have achieved strong results on a wide range ofbenchmarks, yet these gains have not translated into economically meaningful deployment across many professional domains. We argue that this gap is largely an evaluation problem: widely usedbenchmarks lack sustained performance measurement on real and economically valuable workflows. This paper introduces Agents’ Last Exam (ALE), abenchmarkdesigned to evaluateAI agentson long-horizon, economically valuable,real-world taskswith verifiable outcomes. Developed in collaboration with 250+ industry experts, ALE covers non-physical industries defined with reference toO*NET/SOC 2018(the U.S. federal occupational taxonomy). It is organized around atask taxonomywith 55 subfields grouped into 13industry clusterscovering 1K+ tasks. Current results show that the hardest tier remains far from saturated: across mainstream harness and backbone configurations, the averagefull pass rateis 2.6%. ALE is designed as aliving benchmark: its task pool grows continuously as new workflows and industries are onboarded. More broadly, ALE is intended not merely as another leaderboard, but as an instrument for closing the gap betweenbenchmarksuccess and GDP-relevant impact.

View arXiv pageView PDFProject pageGitHub183Add to collection

Get this paper in your agent:

hf papers read 2606\.05405

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.05405 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.05405 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.05405 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.