Coding

AscendGYM

by SLAI-AITP (AI Training Platform Team; Bingwei Zhang, Jing Li et al., 27 authors)

What is AscendGYM? AscendGYM is a coding eval made by SLAI-AITP (AI Training Platform Team; Bingwei Zhang, Jing Li et al., 27 authors). A kernel-optimisation benchmark for Huawei Ascend NPUs. Rather than writing kernels from scratch, the model or agent must speed up existing kernels, and each kernel is tested on data types and tensor shapes taken from real training and inference jobs, so a gain on the benchmark maps to a real workload. The paper (arXiv:2611.17621) describes 48 kernels across three difficulty levels; the public repository currently releases Batch 1 (17 tasks), with later batches announced. It supports two modes, scoring a bare model through a built-in harness or scoring your own agent harness, and proposes a Speedup-Token Score that weighs speedup against token cost. A public leaderboard compares models and harnesses.

Page last updated 2026-10-18 · Evals Review

Editor note

A kernel-optimisation benchmark for Huawei Ascend NPUs. Rather than writing kernels from scratch, the model or agent must speed up existing kernels, and each kernel is tested on data types and tensor shapes taken from real training and inference jobs, so a gain on the benchmark maps to a real workload. The paper (arXiv:2611.17621) describes 48 kernels across three difficulty levels; the public repository currently releases Batch 1 (17 tasks), with later batches announced. It supports two modes, scoring a bare model through a built-in harness or scoring your own agent harness, and proposes a Speedup-Token Score that weighs speedup against token cost. A public leaderboard compares models and harnesses.

What it tests
Whether models and agent harnesses can make real-workload Ascend NPU kernels faster without breaking correctness, and at what token cost.
Original source
github.com/SLAI-AITP/AscendGYM
Last updated
arXiv:2611.17621 (13 Nov 2026); repo last push 2026-11-17 (GitHub API, read 19 Nov 2026)
License
Apache-2.0 (GitHub API, read 19 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related coding evals

All Coding evals · Comparison table

Discussion

No one has discussed AscendGYM yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions