MMLU-Pro
by TIGER-AI-Lab
What is MMLU-Pro? MMLU-Pro is a reasoning & knowledge eval made by TIGER-AI-Lab. A harder follow-up to MMLU with over 12,000 exam and textbook questions in 14 subject areas and ten answer options instead of four. The authors report scores 16% to 33% lower than on MMLU and less sensitivity to prompt wording.
Editor note
A harder follow-up to MMLU with over 12,000 exam and textbook questions in 14 subject areas and ten answer options instead of four. The authors report scores 16% to 33% lower than on MMLU and less sensitivity to prompt wording.
- What it tests
- Expert-level knowledge plus multi-step reasoning across subjects such as law, physics, health and economics.
- Original source
- github.com/TIGER-AI-Lab/MMLU-Pro
- Last updated
- Repo last commit 2026-03-18
- License
- Apache-2.0
Compare MMLU-Pro
Reviews
No reviews yet.
Reviews from practitioners who sign in with GitHub open soon.
Related reasoning & knowledge evals
- GPQA — Expert-level science reasoning.
- Humanity's Last Exam — Frontier academic knowledge and reasoning.
- LiveBench — Hard, contamination-limited tasks with verifiable ground-truth answers (no LLM judge).
- MMLU — World knowledge and problem solving across many school and professional subjects.