Agents & tool use

TwinLab-Bench

by Yang Wan and Linchao Zhu (ZJUSCL)

What is TwinLab-Bench? TwinLab-Bench is an agents & tool use eval made by Yang Wan and Linchao Zhu (ZJUSCL). Tests AI-scientist agents on experiments they design themselves. Each of the 52 tasks is rebuilt from a real laboratory's records (20 physical sciences, 20 life sciences, 12 engineering): the agent gets the notebooks, raw data and interim conclusions the lab had at one point, requests experiments within a budget, and writes a report that is compared with a hidden answer taken from what the lab found later. A per-task environment agent returns the lab's recorded result when the experiment was actually run and otherwise uses calculators built from the lab's records; closed-book probes check for answers recalled from memory. The authors report the best model tested averaging 45 out of 100 (arXiv:2611.26248).

Page last updated 2026-10-18 · Evals Review

Editor note

Tests AI-scientist agents on experiments they design themselves. Each of the 52 tasks is rebuilt from a real laboratory's records (20 physical sciences, 20 life sciences, 12 engineering): the agent gets the notebooks, raw data and interim conclusions the lab had at one point, requests experiments within a budget, and writes a report that is compared with a hidden answer taken from what the lab found later. A per-task environment agent returns the lab's recorded result when the experiment was actually run and otherwise uses calculators built from the lab's records; closed-book probes check for answers recalled from memory. The authors report the best model tested averaging 45 out of 100 (arXiv:2611.26248).

What it tests
Open-ended experimental design and reporting by AI-scientist agents, grounded in real lab records.
Original source
twinlab-bench.github.io/
Last updated
arXiv:2611.26248 (19 Nov 2026); repo last pushed 2026-11-20 (GitHub API) and HF dataset ZJUSCL/TwinLab-Bench updated 2026-11-20 (read 22 Nov 2026)
License
Not checked yet — see source

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related agents & tool use evals

  • GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
  • WebArena — Autonomous web navigation and task completion.
  • OSWorld — Computer-use agents operating real operating systems and apps.
  • Terminal-Bench — Agents working in the shell end to end.

All Agents & tool use evals · Comparison table

Discussion

No one has discussed TwinLab-Bench yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions