WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
Abstract
WorldMemBench evaluates world memory in video models by testing static and dynamic retention across repeated revisits and state transitions.
As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requires both preserving previously observed content and ensuring coherent world evolution over time. World memory is a critical capability to evaluate alongside visual fidelity and video duration. We introduce WorldMemBench, a benchmark for systematically evaluating world memory in video world models. Across 260 test cases, it assesses static memory through the retention of human identity and appearance, object appearance and geometry, and environmental content and geometry, and dynamic memory through event progression and state persistence. Using controlled camera departures and multi-revisit trajectories, the benchmark evaluates static content against fixed initial references and dynamic events against expected state transitions. It combines appearance and geometry metrics with event-specific assessments, and reports video quality and camera-following accuracy as complementary diagnostics. Evaluation of nine representative models shows that strong static retention does not imply reliable dynamic state evolution, while repeated revisits reveal memory degradation beyond the first return. Human studies support the alignment between our automatic evaluations and human judgments. WorldMemBench provides a systematic testbed for diagnosing memory failures and guiding the development of persistent, temporally coherent video worlds.
Community
Hi everyone! We are the authors of WorldMemBench. Here are the official resources:
- ๐ Paper: https://arxiv.org/abs/2610.19063
- ๐ GitHub: https://github.com/ZJU-REAL/WorldMemBench
- ๐ Website and leaderboard: https://zju-real.github.io/WorldMemBench/
- ๐ค Dataset: https://huggingface.co/datasets/rescuerz/WorldMemBench
- ๐ฌ Examples: https://huggingface.co/datasets/rescuerz/WorldMemBench-examples
- ๐ฆ Evaluation weights: https://huggingface.co/rescuerz/WorldMemBench-weights
Questions and feedback are welcome!
We introduce WorldMemBench, a benchmark for evaluating whether video world models remember what they have seen and sustain coherent changes when events move out of view. Across 260 cases and nine models, we find that strong static memory does not guarantee reliable dynamic evolution, and repeated revisits expose degradation that a single return can miss. By jointly assessing static memory, dynamic memory, visual quality, and camera following, WorldMemBench helps diagnose failures and evaluate strategies for building more persistent, temporally coherent video worlds.
Get this paper in your agent:
hf papers read 2610.19063 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 2
rescuerz/WorldMemBench
rescuerz/WorldMemBench-examples
Spaces citing this paper 0
No Space linking this paper