WorkflowArena
by Zheng Jiang, Fang Niu, Bo Peng, Huan-ang Gao et al. (Einsia)
What is WorkflowArena? WorkflowArena is an agents & tool use eval made by Zheng Jiang, Fang Niu, Bo Peng, Huan-ang Gao et al. (Einsia). A benchmark for multi-agent systems that must plan, split up and carry out long pieces of real work, not just answer questions. It has 145 tasks across eight domains (software engineering, enterprise operations, science, formal maths, finance, law, content design and 3D modelling) in 17 task forms, and every task shares a 14,400-second (four-hour) agent budget. Each deliverable is checked against the task's acceptance criteria, giving a strict resolved rate (all criteria met) and a 0-100 continuous score for partial progress. The paper tests nine models in 25 model-and-effort setups; the best overall resolved rate it reports is 36.95%, from Claude Opus 5.5 at max effort in Claude Code. The repo ships task data and evaluation materials, not the execution frameworks.
Editor note
A benchmark for multi-agent systems that must plan, split up and carry out long pieces of real work, not just answer questions. It has 145 tasks across eight domains (software engineering, enterprise operations, science, formal maths, finance, law, content design and 3D modelling) in 17 task forms, and every task shares a 14,400-second (four-hour) agent budget. Each deliverable is checked against the task's acceptance criteria, giving a strict resolved rate (all criteria met) and a 0-100 continuous score for partial progress. The paper tests nine models in 25 model-and-effort setups; the best overall resolved rate it reports is 36.95%, from Claude Opus 5.5 at max effort in Claude Code. The repo ships task data and evaluation materials, not the execution frameworks.
- What it tests
- Whether an agent team can organise dependent, multi-hour work and hand in deliverables that actually pass checks.
- Original source
- github.com/Einsia/WorkflowArena
- Last updated
- Repo last push 2026-11-10; arXiv 2611.12856 v1 2026-11-09
- License
- Not checked yet — see source
Related agents & tool use evals
- GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
- WebArena — Autonomous web navigation and task completion.
- OSWorld — Computer-use agents operating real operating systems and apps.
- Terminal-Bench — Agents working in the shell end to end.
Discussion
No one has discussed WorkflowArena yet. Have you run it? Share what it caught, what it missed or how you set it up.