Image & vision

VideoPhy-2

by UCLA and Google Research (Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, Kai-Wei Chang)

What is VideoPhy-2? VideoPhy-2 is an image & vision eval made by UCLA and Google Research (Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Goldenberg, Aditya Grover, Kai-Wei Chang). An action-centric physical-commonsense benchmark for text-to-video models. It curates 200 diverse actions and detailed prompts; human raters score semantic adherence, physical commonsense and specific physical-rule violations. The official page on 14 Nov 2026 reported that the best listed open model, Wan2.1-14B, reached 21.9% joint performance on the hard subset. ICLR 2026 (arXiv:2503.06800). An automatic evaluator, VideoPhy2-eval, is also released. The earlier VideoPhy paper (arXiv:2406.03520, ICLR 2025) and MIT-licensed code are at github.com/Hritikbansal/videophy.

Page last updated 2026-10-18 · Evals Review

Editor note

An action-centric physical-commonsense benchmark for text-to-video models. It curates 200 diverse actions and detailed prompts; human raters score semantic adherence, physical commonsense and specific physical-rule violations. The official page on 14 Nov 2026 reported that the best listed open model, Wan2.1-14B, reached 21.9% joint performance on the hard subset. ICLR 2026 (arXiv:2503.06800). An automatic evaluator, VideoPhy2-eval, is also released. The earlier VideoPhy paper (arXiv:2406.03520, ICLR 2025) and MIT-licensed code are at github.com/Hritikbansal/videophy.

What it tests
Semantic adherence plus physical commonsense of action videos, including a hard subset and rule-level violations.
Original source
videophy2.github.io/
Last updated
ICLR 2026; VideoPhy repo LICENSE re-read 14 Nov 2026 (MIT)
License
MIT (Hritikbansal/videophy LICENSE, re-read 14 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related image & vision evals

  • MMMU — Expert-level multimodal understanding and reasoning.
  • MathVista — Mathematical reasoning over figures, plots and images.
  • MMBench — Broad perception and reasoning skills of vision-language models.
  • ChartQA — Question answering over charts.

All Image & vision evals · Comparison table

Discussion

No one has discussed VideoPhy-2 yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions