Image & vision

VBench-2.0

by Shanghai AI Laboratory, NTU S-Lab, Sun Yat-Sen University and CUHK (Dian Zheng, Ziqi Huang and collaborators)

What is VBench-2.0? VBench-2.0 is an image & vision eval made by Shanghai AI Laboratory, NTU S-Lab, Sun Yat-Sen University and CUHK (Dian Zheng, Ziqi Huang and collaborators). A later suite on the VBench line that scores generated videos on intrinsic faithfulness rather than only visual polish. It uses five categories — Human Fidelity, Controllability, Creativity, Physics and Commonsense — broken into 18 fine-grained dimensions, mixing generalist VLMs/LLMs with specialist anomaly detectors and human-preference checks. Paper: arXiv:2503.21755 (March 2025). Code lives in the Vchitect/VBench repository. The official Space names a Total Score, but the results CSV we requested from the project's Hugging Face dataset returned HTTP 401 on 14 Nov 2026.

Page last updated 2026-10-18 · Evals Review

Editor note

A later suite on the VBench line that scores generated videos on intrinsic faithfulness rather than only visual polish. It uses five categories — Human Fidelity, Controllability, Creativity, Physics and Commonsense — broken into 18 fine-grained dimensions, mixing generalist VLMs/LLMs with specialist anomaly detectors and human-preference checks. Paper: arXiv:2503.21755 (March 2025). Code lives in the Vchitect/VBench repository. The official Space names a Total Score, but the results CSV we requested from the project's Hugging Face dataset returned HTTP 401 on 14 Nov 2026.

What it tests
Whether generated video follows anatomy, control, creativity, physics and commonsense, not just looks coherent.
Original source
vchitect.github.io/VBench-2.0-project/
Last updated
Project page dated Mar 2025; parent repo last push 2026-08-21; LICENSE re-read 14 Nov 2026
License
Apache-2.0 (parent Vchitect/VBench repo LICENSE, re-read 14 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related image & vision evals

  • MMMU — Expert-level multimodal understanding and reasoning.
  • MathVista — Mathematical reasoning over figures, plots and images.
  • MMBench — Broad perception and reasoning skills of vision-language models.
  • ChartQA — Question answering over charts.

All Image & vision evals · Comparison table

Discussion

No one has discussed VBench-2.0 yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions