FrontierCode 2.0
by Cognition
What is FrontierCode 2.0? FrontierCode 2.0 is a coding eval made by Cognition. A private coding benchmark from Cognition (the team behind Devin) that asks whether a repository maintainer would actually merge an agent's change. Open-source maintainers wrote tasks and detailed grading rubrics from the repositories they maintain; any attempt that fails a blocking check scores 0, and tasks stay private to limit contamination. Version 2.0 (27 Oct 2026) adds more repositories and categories such as security patches and performance work, and its tasks touch more files than 1.1. Artificial Analysis reports 132 tasks and gives it a 15% weight in its Intelligence Index v5 (27 Oct 2026).
Use it when: You care whether a coding agent's change would actually be merged by a maintainer, not just whether tests pass.
Key fact: Compared with FrontierCode 1.1, reference solutions in 2.0 modify 67% more files and 89% more lines of code on average, in codebases that are 25% larger. Attempts that fail blocking criteria score 0.
Our note: The tasks are private, so you cannot run it yourself; you can only read published results. Cognition builds Devin, so read its leaderboard alongside an independent coding benchmark.
Editor note
A private coding benchmark from Cognition (the team behind Devin) that asks whether a repository maintainer would actually merge an agent's change. Open-source maintainers wrote tasks and detailed grading rubrics from the repositories they maintain; any attempt that fails a blocking check scores 0, and tasks stay private to limit contamination. Version 2.0 (27 Oct 2026) adds more repositories and categories such as security patches and performance work, and its tasks touch more files than 1.1. Artificial Analysis reports 132 tasks and gives it a 15% weight in its Intelligence Index v5 (27 Oct 2026).
- What it tests
- Whether coding agents write mergeable, well-scoped changes from informal specs in real codebases.
- Original source
- cognition.com/blog/frontier-code-2.0
- Last updated
- Version 2.0 released 2026-10-27
- License
- Not checked yet — see source
Related coding evals
- SWE-bench — Fixing real bugs in real repositories, judged by unit tests.
- HumanEval — Writing a short correct function from a docstring.
- LiveCodeBench — Code generation, self-repair and test prediction on fresh problems.
- Aider leaderboards (Polyglot) — Practical multi-language code editing.
Discussion
No one has discussed FrontierCode 2.0 yet. Have you run it? Share what it caught, what it missed or how you set it up.