Customer support

τ²-bench / τ³-bench (tau2-bench)

by Sierra Research

What is τ²-bench / τ³-bench (tau2-bench)? τ²-bench (now released as τ³-bench) is Sierra Research's simulation framework for testing customer-service agents. It added a telecom domain where the agent and a simulated user both use tools on a shared system, so the agent must coordinate with and guide the user; τ³ adds a banking knowledge-retrieval domain and full-duplex voice evaluation.

Page last updated 2026-10-17 · Evals Review

Editor note

The successor to τ-bench. Its telecom domain lets both the agent and the simulated user act with tools on a shared system, so the agent has to guide the user through fixes. The current τ³ release adds a banking knowledge-retrieval domain and full-duplex voice tests.

What it tests
Support agents that follow policy, use tools and coach a user who also takes actions.
Original source
github.com/sierra-research/tau2-bench
Last updated
Repo last commit 2026-10-16
License
MIT

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators.

When to use τ²-bench / τ³-bench (tau2-bench)

  • Your support agent has to walk users through actions on their side (dual control), not only act alone.
  • You want airline, retail, telecom or banking-knowledge domains with a public leaderboard at taubench.com.
  • You need voice-agent tests with realtime audio providers.

Limitations

  • The user is simulated by an LLM, so results depend on the user model you choose; report it.
  • Grading changed in v1.0.1 (July 2026) for banking_knowledge: results from versions before 1.0.1 are not comparable on that domain.

How to run it

  1. Clone the repository and install with uv (Python 3.12 or 3.13); add extras for voice or the banking_knowledge domain.
  2. Run a domain with tau2 run, choosing the agent and user models.
  3. Browse results saved in data/simulations/ with tau2 view.
uv sync
tau2 run --domain airline --agent-llm <model> --user-llm <model>

Notable results

  • τ²-bench paper: agents' performance dropped sharply when moving from a no-user setting to dual control, where they must guide the user. [Paper (arXiv)]
  • README (read 17 Oct 2026): τ³ fixes 75+ tasks across airline, retail and banking, based on the SABER analysis. [Official repository]

Sources

Everything above comes from the sources listed, read on 15–16 Oct 2026. Commands are copied from the official instructions; check the source for changes.

Reviews

No reviews yet.

Reviews from practitioners who sign in with GitHub open soon.

Related customer support evals

  • τ-bench (tau-bench) — Policy-following support agents with tools, plus consistency across repeated runs.

All Customer support evals · Comparison table