AI Professions Prompt Benchmark
by AI Professions
What is AI Professions Prompt Benchmark? AI Professions Prompt Benchmark is a writing & instruction following eval made by AI Professions. An open test of general chat models on everyday professional drafting: 100 prompts for real tasks in professions such as accounting, law, healthcare administration, real estate, engineering and sales, plus Excel and Microsoft 365 work, in English, Spanish, French and Portuguese. Each prompt has a fictional input and five pass/fail criteria locked before any model ran, one of them always a check against invented figures, facts or citations. The results page reports 1,500 runs and 7,500 verdicts across five models, with transcripts public. Grading is by an LLM judge from a different model family, and the publisher reports that swapping judges moved scores by about 17 points, so it says the results are not a model ranking. Plain chat only, no tools. Submitted by the publisher.
Editor note
An open test of general chat models on everyday professional drafting: 100 prompts for real tasks in professions such as accounting, law, healthcare administration, real estate, engineering and sales, plus Excel and Microsoft 365 work, in English, Spanish, French and Portuguese. Each prompt has a fictional input and five pass/fail criteria locked before any model ran, one of them always a check against invented figures, facts or citations. The results page reports 1,500 runs and 7,500 verdicts across five models, with transcripts public. Grading is by an LLM judge from a different model family, and the publisher reports that swapping judges moved scores by about 17 points, so it says the results are not a model ranking. Plain chat only, no tools. Submitted by the publisher.
- What it tests
- Whether chat models follow professional drafting briefs without inventing facts, across four languages.
- Original source
- aiprofessions.ai/models/ai-model-test-results/
- Last updated
- Results page updated 21 Oct 2026; repo last push 2026-10-22 (GitHub API, read 17 Nov 2026)
- License
- CC-BY-4.0 data and code (GitHub API, read 17 Nov 2026)
Related writing & instruction following evals
- EQ-Bench Creative Writing — Creative writing quality judged against a rubric.
- AlpacaEval — General helpfulness of chat responses.
- MT-Bench (LLM-as-a-judge paper) — Multi-turn conversation quality; how reliable LLM judges are.
- IFEval — Following explicit, verifiable instructions.
Discussion
No one has discussed AI Professions Prompt Benchmark yet. Have you run it? Share what it caught, what it missed or how you set it up.