Image & vision evals evals
CodingAgents & tool useCustomer supportWriting & instruction followingSafety & factualityHealthcareMultilingual & Indian languagesRAG & retrievalReasoning & knowledgeRubric libraries & frameworksImage & vision evals
Image & vision evals
MMMU
About 11.5K college-level questions across six disciplines and 30 subjects, each pairing text with images such as charts, diagrams, maps and chemical structures.
Image & vision evalsMathVista
6,141 math problems set in visual contexts, drawn from 28 existing multimodal datasets plus three new ones (IQTest, FunctionQA, PaperQA).
Image & vision evalsMMBench
A bilingual (English and Chinese) multiple-choice benchmark that breaks vision-language ability into fine-grained skills, with a scoring method meant to be more robust than single-pass answers.
Image & vision evalsChartQA
9.6K human-written and 23.1K generated questions about real charts that need visual reading plus logical and arithmetic reasoning.