ARC-AGI-3
by ARC Prize Foundation
What is ARC-AGI-3? ARC-AGI-3 is the ARC Prize Foundation's interactive reasoning benchmark: turn-based games on a 64x64 grid with no instructions, where an agent must explore, infer the goal, model the rules and finish each level. It scores action efficiency against first-time human players, and humans solve 100% of the environments.
Editor note
Interactive, turn-based game environments with no instructions: an agent must explore, work out the goal and the rules, and finish each level in as few actions as a first-time human. 25 public demo environments, 55 semi-private and 55 fully private; scored by action efficiency against human baselines (RHAE).
- What it tests
- Exploration, goal inference, world-modelling and planning in new environments.
- Original source
- arcprize.org/arc-agi/3
- Last updated
- Launched Mar 2026
- License
- Not checked yet — see source
When to use ARC-AGI-3
- You want to test learning by acting, not static question answering: the agent is never told the objective.
- You want a score that penalises brute force: each completed level scores (human actions ÷ AI actions)², capped at 1.15.
- You need a benchmark where frontier models are far from saturation: under 1% at launch in March 2026.
Limitations
- Public demo environments are easier than the private ones and are never used for official scores, so public-set results say little.
- Official leaderboard runs use one fixed system prompt and no harness; harness results go on a separate, self-reported community leaderboard. Check which one a number comes from.
- Runs are costly; the official leaderboard caps each level at 5x the human-median action count.
How to run it
- Install the ARC-AGI-3 toolkit from the official repository and play or connect an agent to the 25 public demo environments.
- Score with RHAE: per-level efficiency squared, weighted by level number, averaged over games.
- For official scores, the ARC Prize Foundation tests models on the semi-private set; the fully private set is reserved for the Kaggle competition.
Notable results
- Mar 2026 launch (semi-private set): Opus 4.6 (Max) 0.50%, Gemini 3.1 Pro Preview 0.40%, GPT 5.4 (High) 0.20%, Grok-4.20 0.10%. [ARC-AGI-3 technical paper (arXiv)]
- Human calibration: 486 participants across 414 candidate environments; each environment was tried by 10 people and kept only if at least two solved it on first sight. [ARC-AGI-3 technical paper (arXiv)]
- Jul–Aug 2025 preview competition: the winner (StochasticGoose, CNN + reinforcement learning) scored 12.58% on hidden preview games. [ARC-AGI-3 technical paper (arXiv)]
Sources
FAQ
What is ARC-AGI-3?
An interactive benchmark of novel, abstract, turn-based games. Agents get no instructions and must work out the goal and rules by acting. Scores compare the AI's action count with first-time human players.
How is ARC-AGI-3 scored?
RHAE (Relative Human Action Efficiency). For each completed level: (human baseline actions ÷ AI actions)², capped at 1.15. Levels are weighted by number, so later levels count more, and a game's score is capped by the share of levels completed.
What do frontier models score on ARC-AGI-3?
Under 1% at launch in March 2026; the best was Opus 4.6 (Max) at 0.50% on the semi-private set.
How big is ARC-AGI-3?
25 public demo environments, 55 semi-private and 55 fully private, each with at least six levels.
Compare ARC-AGI-3
Reviews
No reviews yet.
Reviews from practitioners who sign in with GitHub open soon.
Related reasoning & knowledge evals
- GPQA — Expert-level science reasoning.
- Humanity's Last Exam — Frontier academic knowledge and reasoning.
- LiveBench — Hard, contamination-limited tasks with verifiable ground-truth answers (no LLM judge).
- MMLU — World knowledge and problem solving across many school and professional subjects.