Healthcare evals
CodingAgents & tool useCustomer supportWriting & instruction followingSafety & factualityHealthcareMultilingual & Indian languagesRAG & retrievalReasoning & knowledgeRubric libraries & frameworks
Healthcare
HealthBench
5,000 realistic, multi-turn health conversations, each graded against rubrics written by 262 physicians from 60 countries. Includes harder and consensus-validated subsets.
HealthcareMedQA
Multiple-choice questions from medical licensing exams. Easy to run, but exam-style and widely used, so it reveals little about real clinical conversations.
HealthcareMedHELM
A holistic medical leaderboard covering many real clinical task types, not only exam questions.
HealthcareKARMA-OpenMedEvalKit
An open toolkit for evaluating medical AI with an emphasis on Indian healthcare, covering text, image and audio models and adding new India-focused datasets.
HealthcareSAKHI
A maternal-health benchmark built around a WhatsApp chatbot for Hindi- and Marathi-speaking mothers, with rubric verdicts from Indian doctors to calibrate the LLM judge.