StreamEval
by StreamEval Team (affiliations listed on the paper include Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and Reactor)
What is StreamEval? StreamEval is an image & vision eval made by StreamEval Team (affiliations listed on the paper include Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and Reactor). A benchmark for interactive streaming video generation, not one-shot clips. It covers five tasks — avatar dialogue, avatar performance, world generation, instruction editing and reference editing — with 300 scripted cases lasting 15 to 300 seconds, and ten metrics for task execution, state consistency, visual fidelity and audiovisual coordination. The paper (arXiv:2610.28402, October 2026) says results from 38 models will be updated monthly. Code, data and a leaderboard are public.
Editor note
A benchmark for interactive streaming video generation, not one-shot clips. It covers five tasks — avatar dialogue, avatar performance, world generation, instruction editing and reference editing — with 300 scripted cases lasting 15 to 300 seconds, and ten metrics for task execution, state consistency, visual fidelity and audiovisual coordination. The paper (arXiv:2610.28402, October 2026) says results from 38 models will be updated monthly. Code, data and a leaderboard are public.
- What it tests
- Request execution and state consistency in interactive, long-session streaming video generation and editing.
- Original source
- streameval.ai
- Last updated
- arXiv:2610.28402; repo last push 2026-10-25 (GitHub API, earlier read 14 Nov 2026); LICENSE re-read 14 Nov 2026
- License
- Apache-2.0 (GitHub LICENSE, re-read 14 Nov 2026)
Related image & vision evals
Discussion
No one has discussed StreamEval yet. Have you run it? Share what it caught, what it missed or how you set it up.