Agents & tool use

WebArena-Pro

by McGill NLP (Imene Kerboua, Fatemeh Pesaran Zadeh, Xing Han Lù, Weijian Qi et al.)

What is WebArena-Pro? WebArena-Pro is an agents & tool use eval made by McGill NLP (Imene Kerboua, Fatemeh Pesaran Zadeh, Xing Han Lù, Weijian Qi et al.). A reproducible browser-agent benchmark built on 20 self-hosted web apps across six domains (productivity, communication, analytics, business operations, design and media, travel), each modelled on a well-known service. Unlike earlier WebArena-style suites, audio and video are first-class inputs, with dedicated actions for pulling information out of them, and tasks include stateful, cross-site workflows. The paper (arXiv:2611.17955) scores agents on 300 test tasks; the repo also ships 100 dev tasks and runs through BrowserGym/AgentLab, Harbor or an MCP browser service. The authors report Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 tied at 62.3%, while the best generic vision-language agents reach 30.3%. These are the authors' numbers.

Page last updated 2026-10-18 · Evals Review

Editor note

A reproducible browser-agent benchmark built on 20 self-hosted web apps across six domains (productivity, communication, analytics, business operations, design and media, travel), each modelled on a well-known service. Unlike earlier WebArena-style suites, audio and video are first-class inputs, with dedicated actions for pulling information out of them, and tasks include stateful, cross-site workflows. The paper (arXiv:2611.17955) scores agents on 300 test tasks; the repo also ships 100 dev tasks and runs through BrowserGym/AgentLab, Harbor or an MCP browser service. The authors report Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 tied at 62.3%, while the best generic vision-language agents reach 30.3%. These are the authors' numbers.

What it tests
Whether web agents can finish multi-step, multimodal (text, image, audio, video) tasks across realistic self-hosted apps, reproducibly.
Original source
webarena-pro.github.io
Last updated
arXiv:2611.17955 (13 Nov 2026); repo last push 2026-11-16 (GitHub API, read 17 Nov 2026)
License
MIT (GitHub API, read 17 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related agents & tool use evals

  • GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
  • WebArena — Autonomous web navigation and task completion.
  • OSWorld — Computer-use agents operating real operating systems and apps.
  • Terminal-Bench — Agents working in the shell end to end.

All Agents & tool use evals · Comparison table

Discussion

No one has discussed WebArena-Pro yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions