MathArena

Editor note

Evaluates LLMs on math competitions soon after the problems are released, such as AIME, HMMT, USAMO and IMO, to limit training-data contamination. Final answers are checked by a rule-based parser with manual review, and proof problems are graded by two human judges. Its paper reports strong signs of contamination in AIME 2024.

What it tests

Competition math reasoning and natural-language proof writing.

Original source

matharena.ai/

Last updated

Repo last push 2026-10-16

License

MIT (code); data CC BY-NC-SA 4.0

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Related reasoning & knowledge evals

• GPQA — Expert-level science reasoning.
• Humanity's Last Exam — Frontier academic knowledge and reasoning.
• LiveBench — Hard, contamination-limited tasks with verifiable ground-truth answers (no LLM judge) in v1; LiveBench 2.0 (14 Oct 2026) adds long-horizon real-work tasks.
• MMLU — World knowledge and problem solving across many school and professional subjects.

All Reasoning & knowledge evals · Comparison table