Agents' Last Exam (ALE)
by Berkeley RDI (Center for Responsible, Decentralized Intelligence) with industry experts
What is Agents' Last Exam (ALE)? Agents' Last Exam (ALE) is an agents & tool use eval made by Berkeley RDI (Center for Responsible, Decentralized Intelligence) with industry experts. Tests computer-use agents on long, economically valuable professional work in real Windows and Linux sandboxes running the actual software (for example After Effects, Siemens NX, Unreal Engine), mixing GUI and command-line steps. Tasks come from projects experts already completed, are mapped to the O*NET / SOC 2018 occupation taxonomy (55 subdomains, 13 industry clusters) and are graded against hidden references. The v1.1 run set has 122 active tasks, with a CLI-only subset (ALE-CLI) for terminal agents.
Use it when: You want to measure computer-use agents on long professional jobs in real software such as After Effects, Siemens NX or Unreal Engine.
Key fact: The project says it is led by Berkeley RDI with 300+ industry experts, spans all 55 targeted sub-industries and has collected 1,500+ tasks toward a 5,000-task target.
Our note: The task pool is still growing, so scores from different releases are not directly comparable. Note which version (for example v1.1) a result was run on. Terminal-only agents can use the ALE-CLI subset.
Editor note
Tests computer-use agents on long, economically valuable professional work in real Windows and Linux sandboxes running the actual software (for example After Effects, Siemens NX, Unreal Engine), mixing GUI and command-line steps. Tasks come from projects experts already completed, are mapped to the O*NET / SOC 2018 occupation taxonomy (55 subdomains, 13 industry clusters) and are graded against hidden references. The v1.1 run set has 122 active tasks, with a CLI-only subset (ALE-CLI) for terminal agents.
- What it tests
- Long-horizon professional computer work with verifiable deliverables.
- Original source
- agents-last-exam.org/
- Last updated
- v1.1 release docs updated 2026-10-24; repo last push 2026-10-27
- License
- Apache-2.0 (code); CC BY 4.0 (data)
Related agents & tool use evals
- GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
- WebArena — Autonomous web navigation and task completion.
- OSWorld — Computer-use agents operating real operating systems and apps.
- Terminal-Bench — Agents working in the shell end to end.
Discussion
No one has discussed Agents' Last Exam (ALE) yet. Have you run it? Share what it caught, what it missed or how you set it up.