Agents & tool use

WorkflowArena

by Zheng Jiang, Fang Niu, Bo Peng, Huan-ang Gao et al. (Einsia)

What is WorkflowArena? WorkflowArena is an agents & tool use eval made by Zheng Jiang, Fang Niu, Bo Peng, Huan-ang Gao et al. (Einsia). A benchmark for multi-agent systems that must plan, split up and carry out long pieces of real work, not just answer questions. It has 145 tasks across eight domains (software engineering, enterprise operations, science, formal maths, finance, law, content design and 3D modelling) in 17 task forms, and every task shares a 14,400-second (four-hour) agent budget. Each deliverable is checked against the task's acceptance criteria, giving a strict resolved rate (all criteria met) and a 0-100 continuous score for partial progress. The paper tests nine models in 25 model-and-effort setups; the best overall resolved rate it reports is 36.95%, from Claude Opus 5.5 at max effort in Claude Code. The repo ships task data and evaluation materials, not the execution frameworks.

Page last updated 2026-10-18 · Evals Review

Editor note

A benchmark for multi-agent systems that must plan, split up and carry out long pieces of real work, not just answer questions. It has 145 tasks across eight domains (software engineering, enterprise operations, science, formal maths, finance, law, content design and 3D modelling) in 17 task forms, and every task shares a 14,400-second (four-hour) agent budget. Each deliverable is checked against the task's acceptance criteria, giving a strict resolved rate (all criteria met) and a 0-100 continuous score for partial progress. The paper tests nine models in 25 model-and-effort setups; the best overall resolved rate it reports is 36.95%, from Claude Opus 5.5 at max effort in Claude Code. The repo ships task data and evaluation materials, not the execution frameworks.

What it tests
Whether an agent team can organise dependent, multi-hour work and hand in deliverables that actually pass checks.
Original source
github.com/Einsia/WorkflowArena
Last updated
Repo last push 2026-11-10; arXiv 2611.12856 v1 2026-11-09
License
Not checked yet — see source

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related agents & tool use evals

  • GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
  • WebArena — Autonomous web navigation and task completion.
  • OSWorld — Computer-use agents operating real operating systems and apps.
  • Terminal-Bench — Agents working in the shell end to end.

All Agents & tool use evals · Comparison table

Discussion

No one has discussed WorkflowArena yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions