README.md

---
license: apache-2.0
library_name: diffusers
pipeline_tag: robotics
tags:
  - robotics
  - world-model
  - video-generation
  - vision-action
  - next-forcing
---

# Next Forcing — Base Checkpoint

Base checkpoint for **Next Forcing: Causal World Modeling with Multi-Chunk
Prediction**. This is the causal video-action backbone used as the
initialization for post-training.

- 📄 Paper: https://arxiv.org/pdf/2606.11187
- 🌐 Project page: https://gangweix.github.io/next-forcing/
- 💻 Code: https://github.com/gangweix/next-forcing
- 🤖 RoboTwin post-trained model: [`gangweix/next-forcing-posttrain-robotwin`](https://huggingface.co/gangweix/next-forcing-posttrain-robotwin)

## Model Description

Next Forcing addresses the myopic supervision problem in autoregressive video
world models: next-chunk denoising tends to learn local appearance shortcuts
instead of long-range dynamics, especially at high frame rates. Lightweight
**Multi-Chunk Prediction (MCP)** modules predict multiple future chunks through
a causal chain during training, providing dense temporal supervision to the
backbone.

This base checkpoint contains the backbone **without** MCP modules. The MCP
modules are created at the start of post-training and initialized from the last
`mcp_blocks_per_depth` backbone blocks (`mcp_init_from_backbone = True`), so the
post-trained checkpoint is correspondingly larger (6.7B vs 5.1B).

| | |
| --- | --- |
| Parameters | 5.1B (BF16) |
| Backbone layers | 30 |
| MCP modules | Not included (added during post-training) |

Built on top of the [LingBot-VA](https://github.com/Robbyant/lingbot-va) codebase.

## Repository Layout

```text
transformer/     Causal video-action backbone
vae/
text_encoder/
tokenizer/
```

## Usage

Clone the code and install the dependencies as described in the
[repository README](https://github.com/gangweix/next-forcing).

```bash
python -m pip install "huggingface_hub[cli]"
hf download gangweix/next-forcing-base \
  --local-dir ./checkpoints/next-forcing-base
```

The training code resolves model subfolders by path, so point
`NEXT_FORCING_PRETRAINED_MODEL_PATH` at the **local directory**, not at the Hub
repository id:

```bash
export NEXT_FORCING_PRETRAINED_MODEL_PATH=$PWD/checkpoints/next-forcing-base
export NEXT_FORCING_DATASET_PATH=/path/to/your/dataset
export NEXT_FORCING_SAVE_ROOT=/path/to/your/output

NGPU=8 CONFIG_NAME=robotwin_train \
bash script/run_va_posttrain.sh --init-worker 1
```

MCP defaults (`wan_va/configs/mcp_train_config.py`): `num_mcp_depths = 3`,
`mcp_blocks_per_depth = 3`, `mcp_hidden_collect_layers = [3, 11, 19, 29]`,
`mcp_loss_weights = [0.5, 0.2, 0.1]`.

## License

Released under the Apache License 2.0. Next Forcing is developed on top of the
LingBot-VA codebase; please retain the upstream attribution and license when
redistributing.

## Citation

```bibtex
@article{xu2026next,
  title={Next Forcing: Causal World Modeling with Multi-Chunk Prediction},
  author={Xu, Gangwei and Zhang, Qihang and Zhou, Jiaming and Zhu, Xing and Shen, Yujun and Yang, Xin and Xu, Yinghao},
  journal={arXiv preprint arXiv:2606.11187},
  year={2026}
}
```