Coding

FrontierCS 2

by FrontierCS team (Qiuyang Mang, Wenhao Chai et al.; 61 authors)

What is FrontierCS 2? FrontierCS 2 is a coding eval made by FrontierCS team (Qiuyang Mang, Wenhao Chai et al.; 61 authors). A research-and-development benchmark for coding agents: 48 released tasks in 13 areas such as databases, compilers, GPU kernels, high-performance computing, security and robotics. Each task starts from a published paper's own code with the paper's key contribution removed; the agent gets five hours in the repository, can request scores on development workloads, and passes only if its final patch matches or beats the paper's method on every metric of hidden held-out workloads. The paper (arXiv:2610.29932) reports 12 agents, with Claude Opus 5.5 passing 58% of its runs and 16 tasks passed by no run. More than 100 further tasks are kept private for later release, and results are mirrored on a Kaggle leaderboard.

Page last updated 2026-10-18 · Evals Review

Editor note

A research-and-development benchmark for coding agents: 48 released tasks in 13 areas such as databases, compilers, GPU kernels, high-performance computing, security and robotics. Each task starts from a published paper's own code with the paper's key contribution removed; the agent gets five hours in the repository, can request scores on development workloads, and passes only if its final patch matches or beats the paper's method on every metric of hidden held-out workloads. The paper (arXiv:2610.29932) reports 12 agents, with Claude Opus 5.5 passing 58% of its runs and 16 tasks passed by no run. More than 100 further tasks are kept private for later release, and results are mirrored on a Kaggle leaderboard.

What it tests
Whether agents can improve on published computer-science methods inside real research codebases, judged on hidden workloads.
Original source
github.com/FrontierCS/FrontierCS-2
Last updated
arXiv:2610.29932 (21 Oct 2026); repo last push 2026-11-17 (GitHub API, read 18 Nov 2026); Kaggle board updated 14 Nov 2026
License
CC-BY-4.0 (GitHub API, read 18 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related coding evals

All Coding evals · Comparison table

Discussion

No one has discussed FrontierCS 2 yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions