Update README.md

README.md

@@ -1,4 +1,14 @@
-```markdown
+---
+license: other
+language:
+- en
+base_model:
+- Wan-AI/Wan2.2-TI2V-5B
+tags:
+- Pythorch
+- Autonomous-Driving
+---
+````markdown
 ---
 license: mit
 pipeline_tag: other
@@ -9,89 +19,141 @@ tags:
   - end-to-end-autonomous-driving
   - world-action-model
   - trajectory-planning
+  - diffusion-transformer
+  - flow-matching
+  - reinforcement-learning
   - navsim
 ---
 
-# SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
+<h1 align="center">SimWAM: A Simple World Action Model for End-to-End Autonomous Driving</h1>
+
+<div align="center">
+  <a href="https://github.com/H-EmbodVis/SimWAM"><img src="https://img.shields.io/badge/GitHub-Repository-green?logo=github" alt="GitHub repository"></a>
+  <a href="https://huggingface.co/H-EmbodVis/SimWAM"><img src="https://img.shields.io/badge/Hugging%20Face-Model-yellow?logo=huggingface" alt="Hugging Face model"></a>
+  <a href="https://github.com/H-EmbodVis/SimWAM/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue" alt="MIT License"></a>
+
+  <h5>
+    <em>
+      Zongchuang Zhao<sup>1</sup>,
+      Xin Zhou<sup>1</sup>,
+      Tianyang Xu<sup>1</sup>,
+      Zhengyang Sun<sup>1</sup>,
+      Kaixuan Zhou<sup>2</sup>,
+      Honglin Li<sup>2</sup>,
+      Dingkang Liang<sup>1†</sup>,
+      Xiang Bai<sup>1</sup>
+    </em>
+  </h5>
+
+  <sup>1</sup> Huazhong University of Science and Technology<br>
+  <sup>2</sup> Dongfeng Research & Development Institute<br>
+  <sup>†</sup> Project lead
+</div>
+
+## 🔍 Overview
+
+**SimWAM** is a World-Action Model for end-to-end autonomous driving. It transfers video-dynamics priors to trajectory prediction by jointly training a pretrained video expert and a lightweight action diffusion transformer with flow matching.
+
+An isolated attention mask prevents the action branch from accessing future-frame tokens. As a result, the video-generation branch is used only as a training signal and can be removed at inference time, leaving a self-contained action expert that directly predicts future driving trajectories.
+
+This repository contains the official supervised and reinforcement-learning checkpoints for SimWAM on NAVSIM.
+
+<details>
+  <summary><b>Method Summary</b></summary>
 
-SimWAM is a World-Action Model for end-to-end autonomous driving. It transfers video-dynamics priors to trajectory prediction by jointly training a pretrained video expert and a lightweight action diffusion transformer with flow matching.
+SimWAM jointly trains a pretrained video expert and an action expert through a unified attention interface. Both experts attend to the current visual observation, while future-video tokens and action tokens remain isolated from each other.
 
-An isolated attention design prevents the action branch from depending on future frames. The video-generation branch is therefore not required during deployment, allowing the trained action expert to operate as a direct trajectory planner.
+This design transfers motion and traffic-dynamics knowledge from video prediction to trajectory planning without requiring future-scene generation during deployment. The action expert can subsequently be optimized using FlowGRPO and the compositional NAVSIM PDM reward.
 
-This repository contains the official released checkpoints for the SimWAM project.
+</details>
 
-## Task
+### ✨ Key Ideas
 
-SimWAM performs end-to-end autonomous-driving trajectory planning on NAVSIM.
+- Joint flow-matching training of a pretrained video expert and a lightweight action diffusion transformer.
+- Isolated attention between future-video tokens and action tokens.
+- Direct trajectory prediction without future-video generation at inference time.
+- Action-only reinforcement-learning fine-tuning with FlowGRPO.
+- Independent scaling of the video and action experts without shared model parameters.
 
-The released model uses a single front-facing camera and predicts a future ego trajectory directly from the current observation and conditioning information. The project supports supervised imitation learning and FlowGRPO reinforcement-learning fine-tuning.
+## 🎯 Task, Inputs, and Outputs
 
-Because autonomous-driving trajectory planning does not currently map cleanly to a standard Hugging Face inference pipeline, this repository uses `pipeline_tag: other`.
+SimWAM performs end-to-end ego-trajectory planning from a single front-facing camera observation and vehicle-conditioning information.
 
-## Inputs and Outputs
+Because autonomous-driving trajectory planning does not correspond to a standard Hugging Face inference pipeline, this model repository uses `pipeline_tag: other`.
+
+| Benchmark | Inputs | Output | Prediction horizon | Sensor configuration |
+| --- | --- | --- | ---: | --- |
+| NAVSIM | Front RGB image, scene-context embedding, 8-D ego state | Normalized future ego trajectory with shape `[8, 3]` | 8 steps | One front-facing camera at `384 × 672` |
 
 ### Inputs
 
-The NAVSIM configuration uses:
+The official NAVSIM configuration uses:
 
-- A front-facing RGB camera observation resized to `384 × 672`
+- One front-facing RGB camera observation resized to `384 × 672`
 - Precomputed text or scene-context embeddings
 - An 8-dimensional ego-state representation
-- Optional trajectory-planning configuration supplied by the official codebase
+- Model and evaluation configuration supplied by the official SimWAM codebase
 
 ### Outputs
 
-The model predicts:
+The action expert predicts:
 
-- An eight-step future trajectory
-- Three values per trajectory step in the NAVSIM action representation
-- A tensor with shape `[8, 3]` for one evaluated sample
+- An eight-step future ego trajectory
+- Three trajectory values for each predicted step
+- A tensor with shape `[8, 3]` for each evaluated sample
 
-The released checkpoints use normalized trajectory targets. Dataset statistics and the official NAVSIM evaluation code should be used to convert predictions into the required evaluation representation.
+The released checkpoints predict normalized trajectory values. The official dataset statistics and NAVSIM evaluation pipeline are required to convert predictions into the benchmark action representation.
 
-## Checkpoints
+## 📦 Released Checkpoints
 
-| File | Description |
-| --- | --- |
-| `weights/SimWAM.pt` | Supervised SimWAM checkpoint |
-| `weights/SimWAM-RL.pt` | SimWAM checkpoint fine-tuned with FlowGRPO reinforcement learning |
+| Training stage | Checkpoint | Description |
+| --- | --- | --- |
+| Supervised learning | `weights/SimWAM.pt` | SimWAM checkpoint trained with joint video-action flow matching |
+| Reinforcement learning | `weights/SimWAM-RL.pt` | SimWAM checkpoint fine-tuned with FlowGRPO and the NAVSIM PDM reward |
 
-These files are custom PyTorch checkpoints and are not directly loadable with `transformers.AutoModel`.
+The released files are custom PyTorch checkpoints. They require the official SimWAM implementation and cannot be loaded directly with `transformers.AutoModel.from_pretrained()`.
 
-## Download
+## 🛠️ Usage
 
-Download the complete repository:
+You can download the complete model repository using either the Hugging Face CLI or the Python SDK.
+
+### Option 1: Hugging Face CLI
+
+```bash
+pip install -U huggingface_hub
+
+hf download H-EmbodVis/SimWAM \
+  --repo-type model \
+  --local-dir SimWAM-model
+```
+
+### Option 2: Python SDK
 
 ```python
 from huggingface_hub import snapshot_download
 
-local_path = snapshot_download(
-    repo_id="H-EmbodVis/SimWAM"
+snapshot_download(
+    repo_id="H-EmbodVis/SimWAM",
+    repo_type="model",
+    local_dir="SimWAM-model",
 )
-
-print(local_path)
 ```
 
-Download an individual checkpoint:
+To download only the supervised checkpoint:
 
 ```python
 from huggingface_hub import hf_hub_download
 
 checkpoint_path = hf_hub_download(
     repo_id="H-EmbodVis/SimWAM",
+    repo_type="model",
     filename="weights/SimWAM.pt",
 )
 
 print(checkpoint_path)
 ```
 
-Command-line download:
-
-```bash
-hf download H-EmbodVis/SimWAM --local-dir ./SimWAM-checkpoints
-```
-
-## Usage
+### Installation and Evaluation
 
 Clone and install the official implementation:
 
@@ -107,21 +169,51 @@ python -m pip install -e navsim --no-deps
 python -m pip install -e . --no-deps
 ```
 
-Place the downloaded checkpoint under `weights/`, then run the official NAVSIM evaluation script:
+Place the downloaded checkpoints in the `weights/` directory.
+
+A one-sample evaluation of the supervised checkpoint can be launched with:
 
 ```bash
 CKPT=./weights/SimWAM.pt \
 TASK=navsim_uncond_front_384x672_1e-4 \
 NPROC_PER_NODE=1 \
-bash experiments/navsim/run_eval_navsim.sh
+bash experiments/navsim/run_eval_navsim.sh \
+  EVALUATION.max_samples=1 \
+  EVALUATION.num_inference_steps=2 \
+  EVALUATION.save_videos=false
 ```
 
-NAVSIM, nuPlan maps, sensor blobs, text embeddings, normalization statistics, and other required resources must be prepared according to the official project documentation.
+NAVSIM, nuPlan/OpenScene data, maps, sensor blobs, precomputed embeddings, and normalization statistics must be prepared according to the official GitHub documentation.
+
+## 📈 Performance
+
+Using one front-facing camera at `384 × 672`, SimWAM reports a **91.5 PDMS** score on the NAVSIM `navtest` benchmark after reinforcement-learning fine-tuning.
+
+| Configuration | PDMS |
+| --- | ---: |
+| Action-only baseline | 86.6 |
+| Joint video-action training | 90.3 |
+| Joint training with FlowGRPO | **91.5** |
+
+For complete evaluation settings and comparisons, refer to the official implementation:
+
+https://github.com/H-EmbodVis/SimWAM
+
+## 📂 Repository Contents
+
+| Path | Description |
+| --- | --- |
+| `weights/SimWAM.pt` | Supervised SimWAM checkpoint |
+| `weights/SimWAM-RL.pt` | FlowGRPO reinforcement-learning checkpoint |
+| `config.json` | Minimal Hugging Face model metadata |
+| `README.md` | Hugging Face model card |
+| `.gitattributes` | Hugging Face/Xet large-file configuration |
 
-## Repository Contents
+Expected repository structure:
 
 ```text
 .
+├── .gitattributes
 ├── config.json
 ├── README.md
 └── weights
@@ -129,40 +221,35 @@ NAVSIM, nuPlan maps, sensor blobs, text embeddings, normalization statistics, an
     └── SimWAM-RL.pt
 ```
 
-- `README.md` provides the Hugging Face model card and metadata.
-- `config.json` identifies the repository as a custom PyTorch model repository.
-- `weights/SimWAM.pt` contains the supervised checkpoint.
-- `weights/SimWAM-RL.pt` contains the reinforcement-learning checkpoint.
-
-## Performance
-
-The project reports a PDMS score of `91.5` on the NAVSIM `navtest` benchmark using a single front-facing camera.
+## ⚠️ Limitations and Safety
 
-For complete evaluation settings, comparisons, and reproduction instructions, refer to the official GitHub repository.
-
-## Limitations
-
-- The checkpoints require the official SimWAM implementation.
-- The model is designed for the NAVSIM data format and evaluation pipeline.
+- The checkpoints require the official SimWAM implementation and its pinned dependencies.
+- The model is designed for the NAVSIM data format and evaluation protocol.
 - Performance outside the training and evaluation distributions is not guaranteed.
-- Predictions must not be directly deployed on a physical vehicle without additional validation, safety systems, and human oversight.
-- NAVSIM, nuPlan, OpenScene, Wan2.2, and other upstream resources retain their own licenses and usage terms.
+- The model may produce unsafe or invalid trajectories under unfamiliar road, weather, traffic, or sensor conditions.
+- Predictions must not be deployed directly on a physical vehicle without extensive validation, safety constraints, fallback systems, and human oversight.
+- NAVSIM, nuPlan, OpenScene, Wan2.2, and other upstream resources retain their respective licenses and distribution terms.
 
-## GitHub
+## 🔗 Links
 
-Official implementation:
+- **Hugging Face:** https://huggingface.co/H-EmbodVis/SimWAM
+- **GitHub:** https://github.com/H-EmbodVis/SimWAM
+- **NAVSIM:** https://github.com/autonomousvision/navsim
+- **Wan2.2:** https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B
 
-https://github.com/H-EmbodVis/SimWAM
+## ⚖️ License
 
-## License
+The SimWAM source repository is released under the MIT License.
 
-The SimWAM source repository is released under the MIT License. Third-party code, datasets, pretrained models, and other assets remain subject to their respective licenses.
+Third-party code, datasets, pretrained models, and other upstream assets remain subject to their respective licenses and usage terms.
 
-## Citation
+## 📖 Citation
+
+If you find SimWAM useful in your research, please consider citing:
 
 ```bibtex
 @article{zhao2026simwam,
-  title   = {A Simple World Action Model for End-to-End Autonomous Driving},
+  title   = {SimWAM: A Simple World Action Model for End-to-End Autonomous Driving},
   author  = {Zhao, Zongchuang and Zhou, Xin and Xu, Tianyang and
              Sun, Zhengyang and Zhou, Kaixuan and Li, Honglin and
              Liang, Dingkang and Bai, Xiang},
@@ -170,4 +257,4 @@ The SimWAM source repository is released under the MIT License. Third-party code
   year    = {2026}
 }
 ```
-```
+````