@rohanpaul_ai: Most video models look better than they understand and Video quality is only the easiest thing to notice. LongCat just …

X AI KOLs Following Papers

Summary

LongCat released WBench, a benchmark for video world models that tests control, memory, instruction-following, and physical plausibility across 289 cases and 20 models, finding that no model excels in all dimensions, highlighting the gap between video quality and true world simulation.

Most video models look better than they understand and Video quality is only the easiest thing to notice. LongCat just released WBench, it turned video world model testing from a beauty contest into a stress test for control, multi-turn memory, instruction-following, and physical plausibility. It exposed the gap between beautiful video generation and controllable world simulation. A pretty clip is not enough, because a usable world model must keep the same scene, obey later actions, move the camera correctly, preserve objects, and avoid impossible cause-and-effect. WBench tests this with 289 cases, 1,058 interaction turns, 20 models, 5 dimensions, and 22 automatic metrics, covering navigation, subject actions, event edits, perspective switches, and both viewpoints. Across all those 20 evaluated models, the paper finds that no model dominates all dimensions, which means current systems have not yet merged high-quality rendering, reliable control, long-horizon memory, and physical rule-following into one stable capability. Its design separates the world setup from the user action, so researchers can identify whether a failure comes from weak rendering, poor scene setup, bad control, lost state, or broken physics. Navigation has near-zero connection with visual quality, consistency, or physics, meaning a model can look strong while still failing to move on command. The key shift: stop asking only “does the video look good?” and start asking “can the model keep a controllable world alive across many turns?” 1.
Original Article
View Cached Full Text

Cached at: 06/03/26, 05:54 PM

Most video models look better than they understand and Video quality is only the easiest thing to notice.

LongCat just released WBench, it turned video world model testing from a beauty contest into a stress test for control, multi-turn memory, instruction-following, and physical plausibility.

It exposed the gap between beautiful video generation and controllable world simulation.

A pretty clip is not enough, because a usable world model must keep the same scene, obey later actions, move the camera correctly, preserve objects, and avoid impossible cause-and-effect.

WBench tests this with 289 cases, 1,058 interaction turns, 20 models, 5 dimensions, and 22 automatic metrics, covering navigation, subject actions, event edits, perspective switches, and both viewpoints.

Across all those 20 evaluated models, the paper finds that no model dominates all dimensions, which means current systems have not yet merged high-quality rendering, reliable control, long-horizon memory, and physical rule-following into one stable capability.

Its design separates the world setup from the user action, so researchers can identify whether a failure comes from weak rendering, poor scene setup, bad control, lost state, or broken physics.

Navigation has near-zero connection with visual quality, consistency, or physics, meaning a model can look strong while still failing to move on command.

The key shift: stop asking only “does the video look good?” and start asking “can the model keep a controllable world alive across many turns?”

Similar Articles

MemoBench: Benchmarking World Modeling in Dynamically Changing Environments

Hugging Face Daily Papers

MemoBench is a diagnostic benchmark for evaluating video generation models' memory consistency in dynamically changing environments, where objects disappear and reappear in updated states. It includes 360 ground-truth clips and an evaluation suite combining automated metrics with VQA-based assessment, revealing insights into memory consistency challenges.

Video-Oasis: Rethinking Evaluation of Video Understanding

Hugging Face Daily Papers

Video-Oasis reveals that 55% of existing video benchmarks can be solved without visual input, exposing significant capability gaps in current video understanding models. State-of-the-art models perform only marginally above random guessing on the remaining video-native challenges.