SWE-PolyBench

Name: SWE-PolyBench
Creator: Amazon Science
License: https://spdx.org/licenses/MIT.html

by Amazon Science

What is SWE-PolyBench? SWE-PolyBench is a coding eval made by Amazon Science. Repository-level coding tasks from 2,110 curated issues in Java, JavaScript, TypeScript and Python, with a 500-issue stratified subset and a 382-issue Verified split released in August 2025. Besides pass rate it reports file and code-node retrieval metrics.

Page last updated 2026-10-18 · Evals Review

Editor note

Repository-level coding tasks from 2,110 curated issues in Java, JavaScript, TypeScript and Python, with a 500-issue stratified subset and a 382-issue Verified split released in August 2025. Besides pass rate it reports file and code-node retrieval metrics.

What it tests: Coding agents across four languages at repository scale.
Original source: amazon-science.github.io/SWE-PolyBench/
Last updated: Repo last push 2026-08-14
License: MIT

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Related coding evals

SWE-bench — Fixing real bugs in real repositories, judged by unit tests.
HumanEval — Writing a short correct function from a docstring.
LiveCodeBench — Code generation, self-repair and test prediction on fresh problems.
Aider leaderboards (Polyglot) — Practical multi-language code editing.

All Coding evals · Comparison table

Discussion

No one has discussed SWE-PolyBench yet. Have you run it? Share what it caught, what it missed or how you set it up.

Be specific and kind. No promotion or affiliate links. Community rules · All discussions