AILuminate (Safety 1.2.1 and Jailbreak 1.1)

Editor note

A consensus-built safety benchmark from MLCommons. The Safety test checks how reliably a text chat system refuses or safely redirects requests across 12 hazard categories, from violent and non-violent crimes and weapons to hate, privacy, defamation and specialized health, legal and financial advice. The Jailbreak test rewrites the same hazardous prompts with known attack techniques, so the gap between the two scores shows how much safety an attacker can strip away. Recent versions grade with a deterministic evaluator trained by MLCommons instead of LLM judges and report percentiles from 0 to 100. Both now run on demand in MLCommons' Candela service; public Safety 1.2.1 results for 27 systems came out on 29 Oct 2026, with Jailbreak results to follow. A 1,200-prompt demo set is public on GitHub.

What it tests

How often a chat model or guarded deployment produces harmful content, and how much of its safety survives jailbreak rewrites.

Original source

mlcommons.org/2026/10/ailuminate-safety-1-2-1-and-jailbreak-1-1/

Last updated

Safety 1.2.1 and Jailbreak 1.1 released 2026-10-29; demo prompt repo last push 2025-06-11

License

Apache-2.0 (demo prompt repo mlcommons/ailuminate)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Related safety & factuality evals

• HarmBench — Robustness to jailbreaks and harmful requests.
• TruthfulQA — Truthfulness versus imitating human misconceptions.
• StrongREJECT — Real effectiveness of jailbreak attacks.
• SimpleQA — Short-form factuality and calibration.

All Safety & factuality evals