RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO

Yanzuo Lu · Ronglai Zuo · Jiankang Deng — Imperial College London

Project page: https://yanzuo.lu/raven

Overview

Wan2.1-T2V-1.3B release. RAVEN is a causal autoregressive text-to-video model for real-time streaming generation. It extrapolates future chunks from already-generated content, and it is trained with a training-time test framework that aligns training attention with inference-time extrapolation.

CM-GRPO is the reinforcement-learning stage on top of RAVEN. It treats a consistency-model sampling step as a conditional Gaussian transition and applies online Group Relative Policy Optimization directly to that kernel.

This repository hosts weights only. Training, inference, and evaluation code live in the code repository. Other artifacts are listed in the RAVEN collection.

Demo

Files

File	Description
Wan2.1-T2V-1.3B-RAVEN	RAVEN backbone (post-distillation, pre-RL). Full CausalWanModel state dict.
Wan2.1-T2V-1.3B-CMGRPO-Merge	CM-GRPO backbone with the LoRA already folded into RAVEN. Same key set as raven_model.pt.
Wan2.1-T2V-1.3B-CMGRPO-LoRA	CM-GRPO adapter only. Use together with raven_model.pt as the base.
Wan2.1-T2V-1.3B-CMGRPO-Full	PEFT-wrapped bundle of base + adapter, kept as an archival interchange format. The current code cannot load it directly. Convert or merge it offline first.

Model details

Item	Value
Base model	Wan-AI/Wan2.1-T2V-1.3B DiT, causal autoregressive variant
Task / mode	Text-to-video, causal streaming extrapolation
Resolution	480 x 832
Frames	81
Frame rate	16 fps
Sampling	4 NFE with a consistency sampler

Usage

Setup

git clone https://github.com/mvp-ai-lab/RAVEN.git
cd RAVEN
conda env create -f tools/environment.yaml
bash tools/prepare_venv.sh
source venv/bin/activate

Download

hf download mvp-lab/RAVEN --local-dir /path/to/RAVEN-weights

Configure and run

Trial YAML files keep site-specific absolute paths (backbone, VAE, text encoder, tokenizer, prompt lists), so edit them or override the keys on the command line before running. The loading schema depends on which checkpoint you use.

• Base or merged checkpoint → models.backbone.weight.path.
• Adapter-only → raven_model.pt as the base plus models.backbone.adapter.weight, with r, lora_alpha, and target_modules matching the CM-GRPO training trial (r = lora_alpha = 256).
• Legacy lora: {enabled, weight} blocks are not part of the current schema and are ignored.

bash tools/multi_run.sh \
  projects/wan_t2v/trials/dmd/wan2_1_1_3B/causal_wan_t2v_dmd/val_only/raven_sample100.yaml

bash tools/multi_run.sh \
  projects/wan_t2v/trials/dmd/wan2_1_1_3B/causal_wan_t2v_dmd/val_only/cmgrpo_raven_sample100.yaml

License

These weights are released under CC BY-NC 4.0. The upstream Wan2.1 components (DiT initialization, VAE, UMT5-XXL text encoder) remain subject to their own licenses and terms. You are responsible for complying with all applicable licenses for the base model, code, data, and dependencies.

Citation

@article{lu2026raven,
  title = {RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO},
  author = {Lu, Yanzuo and Zuo, Ronglai and Deng, Jiankang},
  year = 2026,
  journal = {arXiv preprint arXiv:2605.15190}
}