TandemBench
by Baohang Li, Xiaocheng Feng, Bing Qin
What is TandemBench? TandemBench is an agents & tool use eval made by Baohang Li, Xiaocheng Feng, Bing Qin. A multiple-choice benchmark for decision models (models that pick among fixed options) in multi-agent cooking scenarios: 30 tasks, 200 scenarios and 1,200 questions. Models either pick the best team plan from candidates or say whether a better plan exists that is missing from the options; reference answers come from execution and exact search. The repo's leaderboard (results as of 3 Nov 2026) covers 40 decision models and 10 general LLMs; the paper reports a best decision accuracy of 62.5% and near-chance optimality checks. The release is the test set plus a separate answer key, without a model-serving adapter.
Editor note
A multiple-choice benchmark for decision models (models that pick among fixed options) in multi-agent cooking scenarios: 30 tasks, 200 scenarios and 1,200 questions. Models either pick the best team plan from candidates or say whether a better plan exists that is missing from the options; reference answers come from execution and exact search. The repo's leaderboard (results as of 3 Nov 2026) covers 40 decision models and 10 general LLMs; the paper reports a best decision accuracy of 62.5% and near-chance optimality checks. The release is the test set plus a separate answer key, without a model-serving adapter.
- What it tests
- Whether a decision model can choose a good multi-agent plan and notice when none of the offered plans is optimal.
- Original source
- github.com/lililibaohang/TandemBench
- Last updated
- Repo last push 2026-11-05; arXiv v1 2026-11-04
- License
- CC BY 4.0 (data)
Related agents & tool use evals
- GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
- WebArena — Autonomous web navigation and task completion.
- OSWorld — Computer-use agents operating real operating systems and apps.
- Terminal-Bench — Agents working in the shell end to end.
Discussion
No one has discussed TandemBench yet. Have you run it? Share what it caught, what it missed or how you set it up.