Reasoning & knowledge

LongBench v2

by LongBench team (THUDM, Tsinghua)

What is LongBench v2? LongBench v2 is a reasoning & knowledge eval made by LongBench team (THUDM, Tsinghua). 503 hard multiple-choice questions over contexts from 8k to 2M words, covering single- and multi-document QA, long in-context learning, long dialogue history, code repositories and structured data. The format keeps scoring simple and reliable.

Page last updated 2026-10-18 · Evals Review

Editor note

503 hard multiple-choice questions over contexts from 8k to 2M words, covering single- and multi-document QA, long in-context learning, long dialogue history, code repositories and structured data. The format keeps scoring simple and reliable.

What it tests
Deep understanding and reasoning over very long inputs.
Original source
longbench2.github.io/
Last updated
Repo last commit 2025-01-15
License
MIT

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Reviews from practitioners who sign in with GitHub open soon.

Related reasoning & knowledge evals

  • GPQA — Expert-level science reasoning.
  • Humanity's Last Exam — Frontier academic knowledge and reasoning.
  • LiveBench — Hard, contamination-limited tasks with verifiable ground-truth answers (no LLM judge).
  • MMLU — World knowledge and problem solving across many school and professional subjects.

All Reasoning & knowledge evals · Comparison table