Image & vision

Physics-IQ

by INSAIT, Sofia University and Google DeepMind (Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, Robert Geirhos)

What is Physics-IQ? Physics-IQ is an image & vision eval made by INSAIT, Sofia University and Google DeepMind (Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, Robert Geirhos). Tests whether a generative video model understands physical outcomes, not just visual realism. The dataset is 396 real filmed videos (66 scenarios × 3 camera angles × 2 takes) covering solid mechanics, fluids, optics, thermodynamics and magnetism. The model sees a short conditioning clip (or a single frame for image-to-video) and must generate the continuation; the score compares that continuation to a real second take. WACV 2026. Official code: github.com/google-deepmind/physics-IQ-benchmark. The original videos are on a public GCS bucket; the later Verified set is a separate entry.

Page last updated 2026-10-18 · Evals Review

Editor note

Tests whether a generative video model understands physical outcomes, not just visual realism. The dataset is 396 real filmed videos (66 scenarios × 3 camera angles × 2 takes) covering solid mechanics, fluids, optics, thermodynamics and magnetism. The model sees a short conditioning clip (or a single frame for image-to-video) and must generate the continuation; the score compares that continuation to a real second take. WACV 2026. Official code: github.com/google-deepmind/physics-IQ-benchmark. The original videos are on a public GCS bucket; the later Verified set is a separate entry.

What it tests
Physical understanding of generated video continuations against filmed ground truth.
Original source
physics-iq.github.io/
Last updated
Official repo last push 2026-10-20 (GitHub API, read 14 Nov 2026); LICENSE re-read 14 Nov 2026
License
Apache-2.0 for software; CC-BY-4.0 for other materials (repo LICENSE, re-read 14 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related image & vision evals

  • MMMU — Expert-level multimodal understanding and reasoning.
  • MathVista — Mathematical reasoning over figures, plots and images.
  • MMBench — Broad perception and reasoning skills of vision-language models.
  • ChartQA — Question answering over charts.

All Image & vision evals · Comparison table

Discussion

No one has discussed Physics-IQ yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions