Coding

Accelerate Bench

by Bowen Liu, Yuhan Li, Nuo Chen et al.

What is Accelerate Bench? Accelerate Bench is a coding eval made by Bowen Liu, Yuhan Li, Nuo Chen et al.. 32 CPU optimisation tasks adapted from scientific applications (for example GROMACS, OpenFOAM and Quantum ESPRESSO) and supercomputing competitions. Instead of reproducing one known performance commit, the agent gets a whole working program, a fixed workload and CPU and memory limits, and eight hours to make it faster; a separate private verifier rebuilds the result, checks it on hidden inputs and times it against the original. The headline Score@1 averages correctness-gated runtime savings. On version 1.1 the paper (arXiv:2611.20882) reports 16 agents between 25.5% and 67.4%, led by Claude Sonnet 5.5 in Claude Code. Tasks ship as Harbor packages.

Page last updated 2026-10-18 · Evals Review

Editor note

32 CPU optimisation tasks adapted from scientific applications (for example GROMACS, OpenFOAM and Quantum ESPRESSO) and supercomputing competitions. Instead of reproducing one known performance commit, the agent gets a whole working program, a fixed workload and CPU and memory limits, and eight hours to make it faster; a separate private verifier rebuilds the result, checks it on hidden inputs and times it against the original. The headline Score@1 averages correctness-gated runtime savings. On version 1.1 the paper (arXiv:2611.20882) reports 16 agents between 25.5% and 67.4%, led by Claude Sonnet 5.5 in Claude Code. Tasks ship as Harbor packages.

What it tests
Application-level performance engineering by coding agents without breaking the required computation.
Original source
acceleratebench.com
Last updated
v1.1; arXiv:2611.20882 (16 Nov 2026); repo last push 2026-11-17 (GitHub API, read 18 Nov 2026)
License
Code Apache-2.0 (GitHub API, read 18 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related coding evals

All Coding evals · Comparison table

Discussion

No one has discussed Accelerate Bench yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions