Image & vision

StreamEval

by StreamEval Team (affiliations listed on the paper include Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and Reactor)

What is StreamEval? StreamEval is an image & vision eval made by StreamEval Team (affiliations listed on the paper include Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and Reactor). A benchmark for interactive streaming video generation, not one-shot clips. It covers five tasks — avatar dialogue, avatar performance, world generation, instruction editing and reference editing — with 300 scripted cases lasting 15 to 300 seconds, and ten metrics for task execution, state consistency, visual fidelity and audiovisual coordination. The paper (arXiv:2610.28402, October 2026) says results from 38 models will be updated monthly. Code, data and a leaderboard are public.

Page last updated 2026-10-18 · Evals Review

Editor note

A benchmark for interactive streaming video generation, not one-shot clips. It covers five tasks — avatar dialogue, avatar performance, world generation, instruction editing and reference editing — with 300 scripted cases lasting 15 to 300 seconds, and ten metrics for task execution, state consistency, visual fidelity and audiovisual coordination. The paper (arXiv:2610.28402, October 2026) says results from 38 models will be updated monthly. Code, data and a leaderboard are public.

What it tests
Request execution and state consistency in interactive, long-session streaming video generation and editing.
Original source
streameval.ai
Last updated
arXiv:2610.28402; repo last push 2026-10-25 (GitHub API, earlier read 14 Nov 2026); LICENSE re-read 14 Nov 2026
License
Apache-2.0 (GitHub LICENSE, re-read 14 Nov 2026)

Facts come only from the linked source (checked 15–16 Oct 2026). Credit belongs to the creators. We don't score or rank evals: how we review.

Reviews

No reviews yet.

Reviews are written by people who sign in with GitHub, one per account per eval. We can't confirm that each reviewer used the eval; we remove reviews that break our rules, never just for being negative, and we never add, buy or edit reviews. Averages, when shown, come only from these reviews.

Sign in with GitHub to review

Related image & vision evals

  • MMMU — Expert-level multimodal understanding and reasoning.
  • MathVista — Mathematical reasoning over figures, plots and images.
  • MMBench — Broad perception and reasoning skills of vision-language models.
  • ChartQA — Question answering over charts.

All Image & vision evals · Comparison table

Discussion

No one has discussed StreamEval yet. Have you run it? Share what it caught, what it missed or how you set it up.

Sign in with GitHub to comment

Be specific and kind. No promotion or affiliate links. Community rules · All discussions