VisualWebArena

Use it when: Your web agent has to use images and page layout (product photos, listings, forum posts), not only page text.

Key fact: The maintainers added an Amazon Machine Image with all VisualWebArena and WebArena websites pre-installed, so the sites no longer have to be set up one by one. (Repo news dated 5 Aug 2024, Official repository README)

Our note: Setup is the main cost: the tasks run against self-hosted sites that must be reset between runs. Budget time for the environment before comparing agents.

Based on the linked source, not a hands-on run. Updated 2 Nov 2026.

Editor note

A visual follow-up to WebArena: 910 web tasks on self-hosted classifieds, shopping and forum sites where the agent has to understand images and page layout, not just text. Success is checked by executing against the site state.

What it tests

Multimodal web agents on visually grounded browsing tasks.

Original source

jykoh.com/vwa

Last updated

Repo last push 2024-11-09

License

MIT

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Related agents & tool use evals

• GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
• WebArena — Autonomous web navigation and task completion.
• OSWorld — Computer-use agents operating real operating systems and apps.
• Terminal-Bench — Agents working in the shell end to end.

All Agents & tool use evals · Comparison table