tz2026 commited on
Commit
74b4958
Β·
verified Β·
1 Parent(s): 582945a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -18
README.md CHANGED
@@ -1,19 +1,16 @@
1
  ---
2
- license: apache-2.0
3
  tags:
4
- - world-model
5
- - autonomous-driving
6
- - video-generation
7
- - motion-planning
8
- - diffusion
9
  ---
10
 
11
  <div align="center">
12
 
13
- # ReWorld: Representation Learning for World Action Models
14
 
15
- **The first representation learning framework for autonomous-driving World Action Models β€”<br>
16
- explicitly optimizing the latent world-to-action pathway.**
17
 
18
  [![arXiv](https://img.shields.io/badge/arXiv-2606.27504-B31B1B?logo=arxiv)](https://arxiv.org/abs/2606.27504)
19
  [![Project Page](https://img.shields.io/badge/Project-Page-1f6feb?logo=githubpages)](https://xiaomi-research.github.io/ReWorld/)
@@ -33,14 +30,14 @@ explicitly optimizing the latent world-to-action pathway.**
33
 
34
  ---
35
 
36
- ## :sparkles: Highlights
37
 
38
  - **Better futures** β€” FVD **81.3 β†’ 61.9** (βˆ’23.9%) on nuScenes video generation, with self-guided sampling enabled for free by intermediate supervision.
39
  - **Safer plans** β€” closed-loop PDMS **89.1 β†’ 90.4** on NAVSIM *Navtest*, with best-in-class NC / DAC / TTC among world-model planners β€” no RL, no test-time scoring.
40
  - **Stronger representations** β€” frozen linear probe on UCF-101 action recognition **68.3% β†’ 80.2%** (+11.9 pts over the DriveLaW baseline).
41
  - **Nearly free** β€” no external encoders, no teacher models, only **+0.3%** per-step Video DiT training cost, and roughly **2Γ— faster** convergence from scratch.
42
 
43
- ## :bulb: Why ReWorld?
44
 
45
  World Action Models (WAMs) couple future environment prediction with action generation, yet standard training supervises only the output ends of the generation and planning modules. The intermediate representations that carry world knowledge are shaped only indirectly, as byproducts of fitting these outputs. We call this the **representation bottleneck of WAMs**: the world-to-action pathway is not explicitly optimized to be future-predictive, cross-modally grounded, or sensitive to closed-loop behavior quality.
46
 
@@ -58,7 +55,7 @@ All supervision is constructed entirely from the WAM's own generation targets, a
58
  <img src="assets/reworld_self_guidance.png" width="100%" alt="Self-guided sampling and faster convergence">
59
  </div>
60
 
61
- ## :trophy: Results at a Glance
62
 
63
  <details open><summary><b>Video generation on nuScenes</b> (val)</summary>
64
 
@@ -127,7 +124,7 @@ Driving-domain pretraining alone barely moves the probe (+1.5 over LTX-Video); t
127
  <br><i>NAVSIM Navtest rollouts (red: ReWorld prediction, green: expert) β€” straight, left turn, right turn, intersection.</i>
128
  </div>
129
 
130
- ## :package: Checkpoints
131
 
132
  | Folder | File | Role |
133
  |:---|:---|:---|
@@ -148,7 +145,7 @@ hf download tz2026/ReWorld
148
 
149
  Please download the **whole repo** (not a single `.safetensors` file). Hugging Face only increments download stats when `config.json` is requested, which `snapshot_download` / `hf download` always fetch.
150
 
151
- ## :rocket: Getting Started
152
 
153
  ### Installation
154
 
@@ -215,11 +212,11 @@ sh scripts/evaluation/run_videodrive_agent_pdm_score_evaluation.sh
215
 
216
  > Optional base imitation before Stage 2: `scripts/training/run_videodrive_train.sh` with [`video_model_train_base.yaml`](DriveLaW-Act/navsim/agents/videodrive/configs/ltx_model/video_model_train_base.yaml), or start from the πŸ€— DriveLaW checkpoint.
217
 
218
- ## :pray: Acknowledgments
219
 
220
- ReWorld builds on [DriveLaW](https://arxiv.org/abs/2512.23421), [NAVSIM](https://github.com/autonomousvision/navsim), [LTX-Video](https://github.com/Lightricks/LTX-Video), [Diffusers](https://github.com/huggingface/diffusers), and the hard-negative protocol of [BeyondDrive](https://github.com/wjl2244/BeyondDrive). Stage 1 uses intermediate supervision inspired by [Internal Guidance](https://arxiv.org/abs/2512.24176). Thanks to all of them for open-sourcing.
221
 
222
- ## :pencil: Citation
223
 
224
  ```bibtex
225
  @article{xia2026reworld,
@@ -232,4 +229,4 @@ ReWorld builds on [DriveLaW](https://arxiv.org/abs/2512.23421), [NAVSIM](https:/
232
 
233
  ## :mailbox: Contact
234
 
235
- Tianze Xia β€” xiatianze@hust.edu.cn Β· Xinggang Wang β€” xgwang@hust.edu.cn
 
1
  ---
 
2
  tags:
3
+ - autonomous-driving
4
+ - video-generation
5
+ - diffusion
6
+ - World-Action-Models
 
7
  ---
8
 
9
  <div align="center">
10
 
11
+ ReWorld: Representation Learning for World Action Models
12
 
13
+ **The first representation learning framework for autonomous-driving World Action Models**
 
14
 
15
  [![arXiv](https://img.shields.io/badge/arXiv-2606.27504-B31B1B?logo=arxiv)](https://arxiv.org/abs/2606.27504)
16
  [![Project Page](https://img.shields.io/badge/Project-Page-1f6feb?logo=githubpages)](https://xiaomi-research.github.io/ReWorld/)
 
30
 
31
  ---
32
 
33
+ ## Highlights
34
 
35
  - **Better futures** β€” FVD **81.3 β†’ 61.9** (βˆ’23.9%) on nuScenes video generation, with self-guided sampling enabled for free by intermediate supervision.
36
  - **Safer plans** β€” closed-loop PDMS **89.1 β†’ 90.4** on NAVSIM *Navtest*, with best-in-class NC / DAC / TTC among world-model planners β€” no RL, no test-time scoring.
37
  - **Stronger representations** β€” frozen linear probe on UCF-101 action recognition **68.3% β†’ 80.2%** (+11.9 pts over the DriveLaW baseline).
38
  - **Nearly free** β€” no external encoders, no teacher models, only **+0.3%** per-step Video DiT training cost, and roughly **2Γ— faster** convergence from scratch.
39
 
40
+ ## Why ReWorld?
41
 
42
  World Action Models (WAMs) couple future environment prediction with action generation, yet standard training supervises only the output ends of the generation and planning modules. The intermediate representations that carry world knowledge are shaped only indirectly, as byproducts of fitting these outputs. We call this the **representation bottleneck of WAMs**: the world-to-action pathway is not explicitly optimized to be future-predictive, cross-modally grounded, or sensitive to closed-loop behavior quality.
43
 
 
55
  <img src="assets/reworld_self_guidance.png" width="100%" alt="Self-guided sampling and faster convergence">
56
  </div>
57
 
58
+ ## Results at a Glance
59
 
60
  <details open><summary><b>Video generation on nuScenes</b> (val)</summary>
61
 
 
124
  <br><i>NAVSIM Navtest rollouts (red: ReWorld prediction, green: expert) β€” straight, left turn, right turn, intersection.</i>
125
  </div>
126
 
127
+ ## Checkpoints
128
 
129
  | Folder | File | Role |
130
  |:---|:---|:---|
 
145
 
146
  Please download the **whole repo** (not a single `.safetensors` file). Hugging Face only increments download stats when `config.json` is requested, which `snapshot_download` / `hf download` always fetch.
147
 
148
+ ## Getting Started
149
 
150
  ### Installation
151
 
 
212
 
213
  > Optional base imitation before Stage 2: `scripts/training/run_videodrive_train.sh` with [`video_model_train_base.yaml`](DriveLaW-Act/navsim/agents/videodrive/configs/ltx_model/video_model_train_base.yaml), or start from the πŸ€— DriveLaW checkpoint.
214
 
215
+ ## Acknowledgments
216
 
217
+ ReWorld builds on [DriveLaW](https://arxiv.org/abs/2512.23421), [NAVSIM](https://github.com/autonomousvision/navsim), [LTX-Video](https://github.com/Lightricks/LTX-Video), [Diffusers](https://github.com/huggingface/diffusers), and the hard-negative protocol of [BeyondDrive](https://github.com/wjl2244/BeyondDrive). Stage 1 uses intermediate supervision inspired by [Internal Guidance](https://github.com/CVL-UESTC/Internal-Guidance). Thanks to all of them for open-sourcing.
218
 
219
+ ## Citation
220
 
221
  ```bibtex
222
  @article{xia2026reworld,
 
229
 
230
  ## :mailbox: Contact
231
 
232
+ Tianze Xia β€” xiatianze@hust.edu.cn Β· Xinggang Wang β€” xgwang@hust.edu.cn