RAG & retrieval

FinanceBench

by Patronus AI

What is FinanceBench? FinanceBench is a rag & retrieval eval made by Patronus AI. Open-book questions about public companies' financial filings, with human-written answers and evidence strings. The full set has 10,231 questions; 150 annotated examples are open source. In the paper, GPT-4-Turbo with retrieval answered wrongly or refused on 81% of the sampled questions.

Page last updated 2026-10-18 · Evals Review

Editor note

Open-book questions about public companies' financial filings, with human-written answers and evidence strings. The full set has 10,231 questions; 150 annotated examples are open source. In the paper, GPT-4-Turbo with retrieval answered wrongly or refused on 81% of the sampled questions.

What it tests
RAG and long-context systems answering questions from financial reports.
Original source
github.com/patronus-ai/financebench
Last updated
Repo last commit 2024-12-03
License
CC-BY-NC-4.0 (Hugging Face dataset)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Reviews from practitioners who sign in with GitHub open soon.

Related rag & retrieval evals

  • Ragas — Faithfulness, relevance and retrieval quality of your own RAG system.
  • BEIR — Zero-shot retrieval across diverse domains.
  • MTEB Leaderboard — Embedding model quality for search and RAG.
  • CRAG — End-to-end RAG answer correctness and hallucination.

All RAG & retrieval evals · Comparison table