KernelBench
by Stanford Scaling Intelligence Lab
What is KernelBench? KernelBench is a coding eval made by Stanford Scaling Intelligence Lab. Asks a model to rewrite PyTorch programs as GPU kernels (CUDA and other kernel languages) and checks two things: the output must match the reference PyTorch code on random inputs, and the kernel should run faster. Problems come in levels, from 100 single operators and 100 fusion patterns to 50 full model architectures and Hugging Face models (ICML 2025).
Editor note
Asks a model to rewrite PyTorch programs as GPU kernels (CUDA and other kernel languages) and checks two things: the output must match the reference PyTorch code on random inputs, and the kernel should run faster. Problems come in levels, from 100 single operators and 100 fusion patterns to 50 full model architectures and Hugging Face models (ICML 2025).
- What it tests
- Writing correct and fast GPU kernels from PyTorch reference code.
- Original source
- github.com/ScalingIntelligence/KernelBench
- Last updated
- Repo last push 2026-03-24
- License
- MIT (per the repo's LICENSE file)
Related coding evals
- SWE-bench — Fixing real bugs in real repositories, judged by unit tests.
- HumanEval — Writing a short correct function from a docstring.
- LiveCodeBench — Code generation, self-repair and test prediction on fresh problems.
- Aider leaderboards (Polyglot) — Practical multi-language code editing.
Discussion
No one has discussed KernelBench yet. Have you run it? Share what it caught, what it missed or how you set it up.