TheAgentCompany
by Carnegie Mellon University & collaborators
What is TheAgentCompany? TheAgentCompany is an agents & tool use eval made by Carnegie Mellon University & collaborators. A simulated small software company with its own GitLab, Plane, ownCloud and RocketChat, where agents do work tasks by browsing, coding, running programs and messaging simulated coworkers. In the paper, the strongest baseline agent finished 30% of tasks on its own.
Editor note
A simulated small software company with its own GitLab, Plane, ownCloud and RocketChat, where agents do work tasks by browsing, coding, running programs and messaging simulated coworkers. In the paper, the strongest baseline agent finished 30% of tasks on its own.
- What it tests
- Agents doing realistic, multi-step workplace tasks end to end.
- Original source
- the-agent-company.com
- Last updated
- Repo last push 2025-11-17
- License
- MIT
Related agents & tool use evals
- GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
- WebArena — Autonomous web navigation and task completion.
- OSWorld — Computer-use agents operating real operating systems and apps.
- Terminal-Bench — Agents working in the shell end to end.
Discussion
No one has discussed TheAgentCompany yet. Have you run it? Share what it caught, what it missed or how you set it up.