Multimodal Knowledge Agent (MKA)
by Google DeepMind (run by Artificial Analysis)
What is Multimodal Knowledge Agent (MKA)? Multimodal Knowledge Agent (MKA) is an agents & tool use eval made by Google DeepMind (run by Artificial Analysis). A benchmark for agents doing professional work where the key facts live in visual files: architectural drawings, electrical schematics, charts, maps and lab plates. Each of its 200 tasks, written with experts across 10 industries and 83 occupations, gives the agent a folder of input files and asks for a finished deliverable such as a report, spreadsheet, slide deck or marked-up drawing; Google estimates each would take a professional three to eight hours. An agentic judge scores the files against an expert rubric of about 70 yes/no checks, weighted 75% on factual accuracy and 25% on visual design. Artificial Analysis keeps the 200-task set private and runs every model in its open-source Stirrup harness; 10 public example tasks are on Hugging Face. On Artificial Analysis' leaderboard (read 11 Nov 2026), Claude Fable 5.2 led the headline Climb score at 57%, with Gemini 4 Carbon at 54%.
Editor note
A benchmark for agents doing professional work where the key facts live in visual files: architectural drawings, electrical schematics, charts, maps and lab plates. Each of its 200 tasks, written with experts across 10 industries and 83 occupations, gives the agent a folder of input files and asks for a finished deliverable such as a report, spreadsheet, slide deck or marked-up drawing; Google estimates each would take a professional three to eight hours. An agentic judge scores the files against an expert rubric of about 70 yes/no checks, weighted 75% on factual accuracy and 25% on visual design. Artificial Analysis keeps the 200-task set private and runs every model in its open-source Stirrup harness; 10 public example tasks are on Hugging Face. On Artificial Analysis' leaderboard (read 11 Nov 2026), Claude Fable 5.2 led the headline Climb score at 57%, with Gemini 4 Carbon at 54%.
- What it tests
- Whether an agent can read dense visual documents, combine them, and produce a correct, well-formatted professional deliverable.
- Original source
- artificialanalysis.ai/evaluations/mka
- Last updated
- Announced 2026-11-10; Hugging Face public set last modified 2026-11-10
- License
- Apache-2.0 (10 public tasks on Hugging Face; 200-task set private)
Related agents & tool use evals
- GAIA — General assistant skills: browsing, file handling, multi-step reasoning.
- WebArena — Autonomous web navigation and task completion.
- OSWorld — Computer-use agents operating real operating systems and apps.
- Terminal-Bench — Agents working in the shell end to end.
Discussion
No one has discussed Multimodal Knowledge Agent (MKA) yet. Have you run it? Share what it caught, what it missed or how you set it up.