Best Programming Language for AI Coding Agents: What 4 Benchmarks Show

Results by benchmark and language

Each row is one language in one benchmark under one agent setup. Scores are only comparable within the same benchmark and setup: tasks, repositories and sample sizes differ between benchmarks.

Same data as the CSV download (CC BY 4.0, last checked 27 Oct 2026).

What each benchmark shows

• SWE-bench Multilingual (source; our page on the parent benchmark: SWE-bench): 300 issues, 42 repositories, 9 languages in 7 groups, no Python. Rust ranks first at 58.14% (one model/agent setup only; 43 Rust tasks), C/C++ last at 28.57%. The authors say gaps may reflect task difficulty, and there are no human difficulty labels.
• Multi-SWE-bench (paper, Table 4; our page): 1,632 tasks in 8 languages. Python is first for all three setups (45.80–52.20%); TypeScript or JavaScript is last for two of them. The authors note the agent methods were first built for Python.
• SWE-PolyBench (paper, Table 3; our page): 2,110 tasks in 4 languages. Python is first for all three agents (20.1–24.1%); JavaScript or TypeScript is last for each. Java beats TypeScript even though Java tasks are structurally the most complex in the set.
• MultiPL-E (paper; built on HumanEval): single-function code generation translated into 18 more languages, tested on 2022 models (Codex, CodeGen, InCoder), not agents. Codex did about as well on JavaScript, C++, Scala and TypeScript as on Python, and static typing had no overall effect. We list no MultiPL-E numbers in the table because its results are per model and prompt, not agent setups.
• Aider polyglot (leaderboard; our page): 225 exercises in 6 languages, but no per-language breakdown is published, so it is not counted among the four benchmarks above.

Caveats

• Small samples. SWE-bench Multilingual has 42–44 tasks per language group, so one task moves a rate by about 2.3 points. SWE-PolyBench has 165 Java and 199 Python tasks.
• Different tasks. Each benchmark uses different repositories and difficulty mixes; never compare a score from one benchmark with a score from another.
• Python-first agents. Multi-SWE-bench and SWE-PolyBench both adapted agents originally built for Python.
• Older models. The per-language results use Claude 3.5/3.7 Sonnet and OpenAI o1 era setups; newer leaderboard entries rarely publish a per-language split.
• Contamination. All of these draw on public GitHub data that may be in training sets, as the SWE-PolyBench authors note.

Context

This question was prompted by a 25 Oct 2026 post by @mattpocockuk on X on what a language designed for agents should look like; it is context, not evidence. The language it mentions, Bend, is covered by none of these benchmarks.

FAQ

Sources (all read 27 Oct 2026)

• https://www.swebench.com/multilingual.html
• https://arxiv.org/abs/2504.02605
• https://arxiv.org/abs/2504.08703
• https://arxiv.org/abs/2208.08227
• https://aider.chat/docs/leaderboards/

Related: coding benchmarks compared · AI agent evals · state of LLM evals