Multi-SWE-bench
by ByteDance Seed
What is Multi-SWE-bench? Multi-SWE-bench is a coding eval made by ByteDance Seed. SWE-bench-style issue resolving beyond Python: 1,632 instances across Java, TypeScript, JavaScript, Go, Rust, C and C++, filtered from 2,456 candidates by 68 expert annotators. Smaller mini (400) and flash (300) sets exist for cheaper runs. Accepted to the NeurIPS 2025 Datasets and Benchmarks track.
Editor note
SWE-bench-style issue resolving beyond Python: 1,632 instances across Java, TypeScript, JavaScript, Go, Rust, C and C++, filtered from 2,456 candidates by 68 expert annotators. Smaller mini (400) and flash (300) sets exist for cheaper runs. Accepted to the NeurIPS 2025 Datasets and Benchmarks track.
- What it tests
- Coding agents fixing real issues in non-Python repositories.
- Original source
- multi-swe-bench.github.io
- Last updated
- Repo last push 2025-12-18
- License
- Apache-2.0
Related coding evals
- SWE-bench — Fixing real bugs in real repositories, judged by unit tests.
- HumanEval — Writing a short correct function from a docstring.
- LiveCodeBench — Code generation, self-repair and test prediction on fresh problems.
- Aider leaderboards (Polyglot) — Practical multi-language code editing.
Discussion
No one has discussed Multi-SWE-bench yet. Have you run it? Share what it caught, what it missed or how you set it up.