DataKernelBench

Editor note

Tests whether LLMs can speed up analytical database queries on GPUs. Each of the 22 TPC-H queries is turned into a validated PyTorch program, and the model optimizes either its core or the whole query in CUDA or Triton, with execution-guided repair. The paper reports a best overall speedup of 2.11x over the compiled PyTorch baseline at a 100% pass rate on one H100.

What it tests

Generating correct, faster CUDA or Triton kernels for database queries.

Original source

kerneldf.github.io/datakernelbench/

Last updated

Repo last push 2026-09-29; paper listed 24 Oct 2026

License

Apache-2.0

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Related coding evals

• SWE-bench — Fixing real bugs in real repositories, judged by unit tests.
• HumanEval — Writing a short correct function from a docstring.
• LiveCodeBench — Code generation, self-repair and test prediction on fresh problems.
• Aider leaderboards (Polyglot) — Practical multi-language code editing.

All Coding evals · Comparison table