AI benchmark comparisons
20 head-to-head pages. Each gives a short verdict, a fact table, when to use which, and dated numbers linked to the original paper, repository or leaderboard.
Coding
- SWE-bench vs SWE-bench-Live: SWE-bench or SWE-bench-Live?
- HumanEval vs BigCodeBench: HumanEval or BigCodeBench?
- HumanEval vs LiveCodeBench: HumanEval or LiveCodeBench?
- SWE-Bench Pro vs SWE-bench Verified: SWE-Bench Pro or SWE-bench Verified?
- LiveCodeBench vs SWE-bench: LiveCodeBench or SWE-bench?
Agents & tool use
- WebArena vs OSWorld: WebArena or OSWorld?
- GAIA vs BrowseComp: GAIA or BrowseComp?
- Terminal-Bench vs SWE-bench: Terminal-Bench or SWE-bench?
- Terminal-Bench 2.1 vs Terminal-Bench 4.0: Terminal-Bench 2.1 or Terminal-Bench 4.0?
Writing & instruction following
- AlpacaEval vs MT-Bench (LLM-as-a-judge paper): AlpacaEval or MT-Bench?
- MT-Bench (LLM-as-a-judge paper) vs LMArena (Chatbot Arena): MT-Bench or LMArena (Chatbot Arena)?
- LLM-as-a-Judge vs Agent-as-a-Judge: LLM-as-a-Judge or Agent-as-a-Judge?
Safety & factuality
- HarmBench vs StrongREJECT: HarmBench or StrongREJECT?
RAG & retrieval
- BEIR vs MTEB Leaderboard: BEIR or MTEB?
- Ragas vs DeepEval: Ragas or DeepEval?
Reasoning & knowledge
- MMLU vs GPQA: MMLU or GPQA?
- MMLU vs MMLU-Pro: MMLU or MMLU-Pro?
- GPQA vs Humanity's Last Exam: GPQA or Humanity's Last Exam?
- ARC-AGI-2 vs ARC-AGI-3: ARC-AGI-2 or ARC-AGI-3?
Rubric libraries & frameworks
- DeepEval vs Promptfoo: DeepEval or Promptfoo?