AgentDojo
by ETH Zurich (SPY Lab) & Invariant Labs
What is AgentDojo? AgentDojo is a safety & factuality eval made by ETH Zurich (SPY Lab) & Invariant Labs. An extensible environment for testing whether tool-using agents can be hijacked by prompt injections hidden in the data their tools return. The paper's release has 97 realistic tasks, such as handling email, e-banking and travel bookings, and 629 security test cases, with pluggable attacks and defences.
Editor note
An extensible environment for testing whether tool-using agents can be hijacked by prompt injections hidden in the data their tools return. The paper's release has 97 realistic tasks, such as handling email, e-banking and travel bookings, and 629 security test cases, with pluggable attacks and defences.
- What it tests
- Agent task success and robustness to prompt injection attacks.
- Original source
- github.com/ethz-spylab/agentdojo
- Last updated
- Repo last commit 2026-06-02
- License
- MIT
Reviews
No reviews yet.
Reviews from practitioners who sign in with GitHub open soon.
Related safety & factuality evals
- HarmBench — Robustness to jailbreaks and harmful requests.
- TruthfulQA — Truthfulness versus imitating human misconceptions.
- StrongREJECT — Real effectiveness of jailbreak attacks.
- SimpleQA — Short-form factuality and calibration.