---
title: MBench Leaderboard
emoji: 🏆
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 4.36.1
python_version: '3.10'
app_file: app.py
pinned: false
license: mit
---

# MBench Leaderboard

MBench is a benchmark for evaluating the memory capability of video world models. It focuses on whether a model can preserve a coherent world state across long-horizon video continuation and interaction.

The benchmark is organized around three core memory dimensions:

- **Entity Consistency:** persistent object and human identity, geometry, texture, and appearance.
- **Environment Consistency:** stable spatial layout, reprojection behavior, lighting, and style.
- **Causal Consistency:** reliable state evolution and interaction consequences over time.

MBench uses trigger-conditioned scoring: Trigger Coverage measures whether the model actually enters the intended memory challenge, Memory Reliability measures consistency after the challenge is triggered, and M-Score balances both with a harmonic mean.

The bundled seed leaderboard is transcribed from Table 2 of the MBench paper. Aggregate leaderboard columns are derived as unweighted averages over the reported sub-dimensions until official leaderboard totals are released.

## Links

- **Dataset:** `studyOverflow/TempMemoryData`
- **Leaderboard data repo:** `PeanutUp/membench_leaderboard_submission`
- **GitHub repo:** https://github.com/study-overflow/MBench
- **Project page:** https://peanutup.github.io/MBench-project/