SimWAM: A Simple World Action Model for End-to-End Autonomous Driving

Zongchuang Zhao¹, Xin Zhou¹, Tianyang Xu¹, Zhengyang Sun¹, Kaixuan Zhou², Honglin Li², Dingkang Liang^1†, Xiang Bai¹

¹ Huazhong University of Science and Technology
² Dongfeng Research & Development Institute
^† Project lead

🔍 Overview

SimWAM is a World-Action Model for end-to-end autonomous driving. It transfers video-dynamics priors to trajectory prediction by jointly training a pretrained video expert and a lightweight action diffusion transformer with flow matching.

An isolated attention mask prevents the action branch from accessing future-frame tokens. As a result, the video-generation branch is used only as a training signal and can be removed at inference time, leaving a self-contained action expert that directly predicts future driving trajectories.

This repository contains the official supervised and reinforcement-learning checkpoints for SimWAM on NAVSIM.

Method Summary

SimWAM jointly trains a pretrained video expert and an action expert through a unified attention interface. Both experts attend to the current visual observation, while future-video tokens and action tokens remain isolated from each other.

This design transfers motion and traffic-dynamics knowledge from video prediction to trajectory planning without requiring future-scene generation during deployment. The action expert can subsequently be optimized using FlowGRPO and the compositional NAVSIM PDM reward.

✨ Key Ideas

• Joint flow-matching training of a pretrained video expert and a lightweight action diffusion transformer.
• Isolated attention between future-video tokens and action tokens.
• Direct trajectory prediction without future-video generation at inference time.
• Action-only reinforcement-learning fine-tuning with FlowGRPO.
• Independent scaling of the video and action experts without shared model parameters.

🎯 Task, Inputs, and Outputs

SimWAM performs end-to-end ego-trajectory planning from a single front-facing camera observation and vehicle-conditioning information.

Because autonomous-driving trajectory planning does not correspond to a standard Hugging Face inference pipeline, this model repository uses pipeline_tag: other.

Benchmark	Inputs	Output	Prediction horizon	Sensor configuration
NAVSIM	Front RGB image, scene-context embedding, 8-D ego state	Normalized future ego trajectory with shape [8, 3]	8 steps	One front-facing camera at 384 × 672

Inputs

The official NAVSIM configuration uses:

• One front-facing RGB camera observation resized to 384 × 672
• Precomputed text or scene-context embeddings
• An 8-dimensional ego-state representation
• Model and evaluation configuration supplied by the official SimWAM codebase

Outputs

The action expert predicts:

• An eight-step future ego trajectory
• Three trajectory values for each predicted step
• A tensor with shape [8, 3] for each evaluated sample

The released checkpoints predict normalized trajectory values. The official dataset statistics and NAVSIM evaluation pipeline are required to convert predictions into the benchmark action representation.

📦 Released Checkpoints

Training stage	Checkpoint	Description
Supervised learning	weights/SimWAM.pt	SimWAM checkpoint trained with joint video-action flow matching
Reinforcement learning	weights/SimWAM-RL.pt	SimWAM checkpoint fine-tuned with FlowGRPO and the NAVSIM PDM reward

The released files are custom PyTorch checkpoints. They require the official SimWAM implementation and cannot be loaded directly with transformers.AutoModel.from_pretrained().

🛠️ Usage

You can download the complete model repository using either the Hugging Face CLI or the Python SDK.

Option 1: Hugging Face CLI

pip install -U huggingface_hub

hf download H-EmbodVis/SimWAM \
  --repo-type model \
  --local-dir SimWAM-model

Option 2: Python SDK

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="H-EmbodVis/SimWAM",
    repo_type="model",
    local_dir="SimWAM-model",
)

To download only the supervised checkpoint:

from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="H-EmbodVis/SimWAM",
    repo_type="model",
    filename="weights/SimWAM.pt",
)

print(checkpoint_path)

Installation and Evaluation

Clone and install the official implementation:

git clone https://github.com/H-EmbodVis/SimWAM.git
cd SimWAM

conda create -n simwam python=3.10 -y
conda activate simwam

python -m pip install -r requirements.txt
python -m pip install -e navsim --no-deps
python -m pip install -e . --no-deps

Place the downloaded checkpoints in the weights/ directory.

A one-sample evaluation of the supervised checkpoint can be launched with:

CKPT=./weights/SimWAM.pt \
TASK=navsim_uncond_front_384x672_1e-4 \
NPROC_PER_NODE=1 \
bash experiments/navsim/run_eval_navsim.sh \
  EVALUATION.max_samples=1 \
  EVALUATION.num_inference_steps=2 \
  EVALUATION.save_videos=false

NAVSIM, nuPlan/OpenScene data, maps, sensor blobs, precomputed embeddings, and normalization statistics must be prepared according to the official GitHub documentation.

📈 Performance

Using one front-facing camera at 384 × 672, SimWAM reports a 91.5 PDMS score on the NAVSIM navtest benchmark after reinforcement-learning fine-tuning.

Configuration	PDMS
Action-only baseline	86.6
Joint video-action training	90.3
Joint training with FlowGRPO	91.5

For complete evaluation settings and comparisons, refer to the official implementation:

https://github.com/H-EmbodVis/SimWAM

📂 Repository Contents

Path	Description
weights/SimWAM.pt	Supervised SimWAM checkpoint
weights/SimWAM-RL.pt	FlowGRPO reinforcement-learning checkpoint
config.json	Minimal Hugging Face model metadata
README.md	Hugging Face model card
.gitattributes	Hugging Face/Xet large-file configuration

Expected repository structure:

.
├── .gitattributes
├── config.json
├── README.md
└── weights
    ├── SimWAM.pt
    └── SimWAM-RL.pt

⚠️ Limitations and Safety

• The checkpoints require the official SimWAM implementation and its pinned dependencies.
• The model is designed for the NAVSIM data format and evaluation protocol.
• Performance outside the training and evaluation distributions is not guaranteed.
• The model may produce unsafe or invalid trajectories under unfamiliar road, weather, traffic, or sensor conditions.
• Predictions must not be deployed directly on a physical vehicle without extensive validation, safety constraints, fallback systems, and human oversight.
• NAVSIM, nuPlan, OpenScene, Wan2.2, and other upstream resources retain their respective licenses and distribution terms.

🔗 Links

• Hugging Face: https://huggingface.co/H-EmbodVis/SimWAM
• GitHub: https://github.com/H-EmbodVis/SimWAM
• NAVSIM: https://github.com/autonomousvision/navsim
• Wan2.2: https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B

⚖️ License

The SimWAM source repository is released under the MIT License.

Third-party code, datasets, pretrained models, and other upstream assets remain subject to their respective licenses and usage terms.

📖 Citation

If you find SimWAM useful in your research, please consider citing:

@article{zhao2026simwam,
  title   = {SimWAM: A Simple World Action Model for End-to-End Autonomous Driving},
  author  = {Zhao, Zongchuang and Zhou, Xin and Xu, Tianyang and
             Sun, Zhengyang and Zhou, Kaixuan and Li, Honglin and
             Liang, Dingkang and Bai, Xiang},
  journal = {arXiv preprint},
  year    = {2026}
}