Code2Video-Compass
by Fudan University and Sun Yat-sen University (Zhenghan Chen, Zekai Shao, Xin Lin, Leting Cheng, Siming Chen)
What is Code2Video-Compass? Code2Video-Compass is an image & vision eval made by Fudan University and Sun Yat-sen University (Zhenghan Chen, Zekai Shao, Xin Lin, Leting Cheng, Siming Chen). A general-purpose benchmark for code-based video generation: a model must write executable programs that produce a complete video, across text-to-video, image-to-video and video-to-video settings. It has 2,104 tasks in 10 categories and 24 subtypes, scored with 12 objective metrics and five subjective metrics (including an MLLM judge). Paper: arXiv:2610.28416 (20 Oct 2026). Code is at github.com/code2video-compass/Code2Video-Compass.
Editor note
A general-purpose benchmark for code-based video generation: a model must write executable programs that produce a complete video, across text-to-video, image-to-video and video-to-video settings. It has 2,104 tasks in 10 categories and 24 subtypes, scored with 12 objective metrics and five subjective metrics (including an MLLM judge). Paper: arXiv:2610.28416 (20 Oct 2026). Code is at github.com/code2video-compass/Code2Video-Compass.
- What it tests
- Whether a model can write code that builds a complete, aligned T2V, I2V or V2V video.
- Original source
- code2video-compass.github.io/
- Last updated
- arXiv:2610.28416 (2026-10-20); GitHub repo last push 2026-11-11; LICENSE read 14 Nov 2026
- License
- Apache-2.0 (GitHub LICENSE, read 14 Nov 2026)
Related image & vision evals
Discussion
No one has discussed Code2Video-Compass yet. Have you run it? Share what it caught, what it missed or how you set it up.