AI agent and tool-use evals, compared
Agent evals check whether a model can take multi-step actions: browse, use tools, operate a computer or a terminal, or handle a support conversation under policy.
| Eval | Made by | What it tests | License | Last updated | Source |
|---|---|---|---|---|---|
| GAIA | Meta, Hugging Face & collaborators | General assistant skills: browsing, file handling, multi-step reasoning. | — | — | arxiv.org |
| τ-bench (tau-bench) | Sierra Research | Policy-following support agents with tools, plus consistency across repeated runs. | MIT | Repo last commit 2026-03-18 | github.com |
| WebArena | Carnegie Mellon University | Autonomous web navigation and task completion. | — | — | webarena.dev |
| OSWorld | XLANG Lab & collaborators | Computer-use agents operating real operating systems and apps. | — | — | os-world.github.io |
| Terminal-Bench | Terminal-Bench team | Agents working in the shell end to end. | — | — | tbench.ai |
| BrowseComp | OpenAI | Deep, creative web research by browsing agents. | — | Published 2025-04-10 | openai.com |
| MLE-bench | OpenAI | End-to-end ML engineering: data prep, training, experimentation and submission quality. | — | Repo last commit 2026-04-24 | github.com |
| Berkeley Function Calling Leaderboard (BFCL) | UC Berkeley (Gorilla) | Correct function/tool calling across simple, parallel, multi-turn and agentic tasks. | Apache-2.0 | Repo last commit 2026-04-13 | gorilla.cs.berkeley.edu |
In short
GAIA — Questions that are easy for a person with a browser and tools but hard for AI assistants, each with one short, checkable answer.
τ-bench (tau-bench) — Simulates a customer talking to an AI agent in domains like retail and airlines. The agent must follow company policy and use tools; the final database state is checked.
WebArena — Self-hosted copies of realistic websites (shop, forum, code host and more) where an agent must complete tasks by clicking and typing.
OSWorld — Puts an agent in a real computer environment and asks it to do tasks across desktop apps, checked by scripts.
Terminal-Bench — Tasks an agent must finish inside a command-line terminal, such as setting up, debugging or building software.
BrowseComp — 1,266 very hard-to-find facts with short answers that are easy to verify, built to test persistent web-browsing agents.
MLE-bench — 75 real Kaggle machine-learning competitions where an agent must prepare data, train models and submit results, scored against human leaderboard medal thresholds.
Berkeley Function Calling Leaderboard (BFCL) — Measures how accurately a model calls tools and functions, including parallel calls, multi-turn tool use and agentic scenarios, with an actively updated public leaderboard.
Browse the full Agents & tool use and Customer support categories, or see other comparisons.