Human video Elo vs physics scores
Short answer: On 14 Nov 2026 we compared published leaderboards, not model outputs. The Artificial Analysis text-to-video board (AA-Video-T2V v2.1, 34 models, Wan 3.0 first at Elo 1175.13) did not share four unambiguously matched models with Physics-IQ Verified (53 rows), Physics-IQ Original (32 rows) or VideoPhy-2 hard (7 models). Under our pre-registered rules we therefore computed no rank correlation. Physics-IQ lists image-to-video and video-to-video rows, not text-to-video; VideoPhy-2 lists older text-to-video names that do not appear under the same version on the human board.
Limits: read these before quoting a number
- This is a comparison of published tables on one date, not a new leaderboard and not a run of any model.
- Zero matched pairs means we do not report Spearman or Kendall figures. A missing correlation is not evidence that either board is wrong.
- Input type matters. We did not treat an image-to-video physics score as if it belonged to a text-to-video Elo row, even when the marketing name was the same.
- Two boards could not be read. LMArena Video returned HTTP 403 (a Cloudflare challenge). The official VBench-2.0 totals live in a Hugging Face dataset that returned HTTP 401. We did not invent those numbers.
Method (pre-registered)
The method was committed before the first snapshot: PREREGISTRATION.md (git commit eb28592, 2026-11-14 18:23:11 IST).
| Human board | Artificial Analysis AA-Video-T2V v2.1, text-to-video only, parsed from the official page payload. 34 models. The page also labelled "7 models added in the last 30 days". |
|---|---|
| Physics boards | Official Physics-IQ Verified and Original tables in the google-deepmind/physics-IQ-benchmark README (commit d3f3de1), and the official VideoPhy-2 hard-subset joint scores. |
| A match | Same model family and the same version token, the same input type (text-to-video only against T2V rows), and no best-of-N, reward or rewriter add-on on only one side. Ambiguous names stayed unmatched. |
| Correlation rule | Spearman and Kendall tau-b only if a pair of boards had 4 or more matches; a bootstrap interval only if there were 6 or more. Fewer than 4 matches: publish the tables, do not compute a correlation. |
Results
| Board | Rows we parsed | Input type | Matches with AA-Video-T2V v2.1 |
|---|---|---|---|
| AA-Video-T2V v2.1 | 34 models (human Elo). First: Wan 3.0 at 1175.13, 7,787 appearances. | text-to-video | — |
| Physics-IQ Verified | 53 | 36 image-to-video, 17 multiframe (video-to-video). No text-to-video row. | 0 |
| Physics-IQ Original | 32 | 22 image-to-video, 10 multiframe (video-to-video). No text-to-video row. | 0 |
| VideoPhy-2 hard (joint) | 7 (open and closed tables) | text-to-video action prompts | 0 |
VideoPhy-2 is the physics board that is actually text-to-video. Its listed models (Wan2.1-14B at 21.9 hard joint, plus CogVideoX-5B, Cosmos-Diff-7B, Hunyuan-13B, VideoCrafter-2, Ray2 and Sora) do not appear under those same version names on AA-Video-T2V v2.1, which is led by later names such as Wan 3.0.
Same family, still unmatched
Several human-board names sit near a physics-board name. Under the pre-registered rules they are not matches. The three exact-name pairs below differ by input type; the others also differ by version token.
| Human board (T2V Elo) | Physics-IQ Verified | Why unmatched |
|---|---|---|
| Veo 3.1 Lite (rank 27, Elo 947.88) | Veo 3.1 Lite (rank 46, score 31.83, i2v) | Same version name; text-to-video board vs image-to-video physics row. |
| Veo 3.1 Fast (rank 26, Elo 954.76) | Veo 3.1 Fast (rank 50, score 29.96, i2v) | Same version name; text-to-video board vs image-to-video physics row. |
| MiniMax H3 Max (rank 8, Elo 1149.55) | MiniMax H3 Max (rank 38, score 36.2, i2v) | Same version name; text-to-video board vs image-to-video physics row. |
| Veo 3.1 (rank 25, Elo 963.37) | Veo 3.1 Lite / Veo 3.1 Fast (i2v) | Related names, not the same version token, and input types differ. |
| Wan 3.0 (rank 1, Elo 1175.13) | Wan 2.2 14B / Wan 2.2 5B (i2v) | Same family, different version (3.0 vs 2.2), and input types differ. |
| MiniMax H3 (768p) (rank 3, Elo 1160.91) | MiniMax H3 (FL2VA) / MiniMax H3 Max (i2v) | 768p and FL2VA / Max are not the same version token; input types also differ. |
Boards we could not read
- LMArena Video ( arena.ai/leaderboard/video): HTTP 403 Cloudflare challenge on arena.ai/leaderboard/video and the related paths we tried. No ranks taken.
- VBench-2.0 official totals: Official project page has no numeric table. Official VBench-2.0 results CSV in dataset Vchitect/vbench_leaderboard_submission returned HTTP 401. We did not invent a total.
Credit
The tables belong to their authors. Artificial Analysis publishes the Video Arena Elo boards. Physics-IQ is by INSAIT, Sofia University and Google DeepMind; Physics-IQ Verified is by Anates Labs with University of Technology Nuremberg, Tübingen AI Center, TU Munich / Helmholtz AI, and Google DeepMind. VideoPhy-2 is by UCLA and Google Research. VBench-2.0 is by Shanghai AI Laboratory, NTU S-Lab and collaborators. LMArena publishes the arena.ai video boards we could not read.
Data, scripts and licence
Download the pre-registration, source log with retrieval times, summary, parsed CSVs for AA-Video-T2V v2.1, Physics-IQ Verified, Physics-IQ Original and VideoPhy-2, plus raw HTTP snapshots under /data/tests/2026-11-14-human-votes-vs-physics-scores/raw/ (file index). The analysis script is in our repo under tests/human-votes-vs-physics-scores/analyze.py.
Licence: our method notes, parsed tables and this page text are CC BY 4.0. Attribution: Evals Review editorial team, Human video Elo vs physics scores, 14 Nov 2026, https://evalsreview.com/tests/human-votes-vs-physics-scores. The underlying leaderboards remain the property of their authors and are used here as published snapshots.