Writing & instruction following

AI Professions Prompt Benchmark

by AI Professions

What is AI Professions Prompt Benchmark? AI Professions Prompt Benchmark is a writing & instruction following eval made by AI Professions. An open test of general chat models on everyday professional drafting: 100 prompts for real tasks in professions such as accounting, law, healthcare administration, real estate, engineering and sales, plus Excel and Microsoft 365 work, in English, Spanish, French and Portuguese. Each prompt has a fictional input and five pass/fail criteria locked before any model ran, one of them always a check against invented figures, facts or citations. The results page reports 1,500 runs and 7,500 verdicts across five models, with transcripts public. Grading is by an LLM judge from a different model family, and the publisher reports that swapping judges moved scores by about 17 points, so it says the results are not a model ranking. Plain chat only, no tools. Submitted by the publisher.

Page last updated 2026-10-18 · Evals Review

Editor note

An open test of general chat models on everyday professional drafting: 100 prompts for real tasks in professions such as accounting, law, healthcare administration, real estate, engineering and sales, plus Excel and Microsoft 365 work, in English, Spanish, French and Portuguese. Each prompt has a fictional input and five pass/fail criteria locked before any model ran, one of them always a check against invented figures, facts or citations. The results page reports 1,500 runs and 7,500 verdicts across five models, with transcripts public. Grading is by an LLM judge from a different model family, and the publisher reports that swapping judges moved scores by about 17 points, so it says the results are not a model ranking. Plain chat only, no tools. Submitted by the publisher.

What it tests
Whether chat models follow professional drafting briefs without inventing facts, across four languages.
Original source
aiprofessions.ai/models/ai-model-test-results/
Last updated
Results page updated 21 Oct 2026; repo last push 2026-10-22 (GitHub API, read 17 Nov 2026)
License
CC-BY-4.0 data and code (GitHub API, read 17 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related writing & instruction following evals

All Writing & instruction following evals

Discussion

No one has discussed AI Professions Prompt Benchmark yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions