| --- |
| license: other |
| license_name: stabilityai-community |
| license_link: https://stability.ai/community-license-agreement |
| tags: |
| - world-model |
| - autonomous-driving |
| - video-generation |
| - gaussian-splatting |
| - panorama |
| - nuscenes |
| --- |
| |
| # PanDWM: Panoramic 4D Driving World Models |
|
|
| Checkpoints for **PanDWM**, a panoramic driving world model that generates 360° driving video in four denoising steps per window and lifts the generated frames into 3D and 4D Gaussians. |
|
|
| ### Paper (coming soon) | Website (coming soon) | [Code](https://github.com/AIGeeksGroup/PanDWM) |
|
|
| ## Files |
|
|
| | File | Component | Size | SHA-256 | |
| |---|---|---|---| |
| | `video/pandwm_video_v84i4250_ema.pth` | Four-step autoregressive video student (EMA weights, training step 4250) | 13 GB | `4def33264008d987fc3a9053a7a3f88d917cd0b467158f6dd109517e359d46cd` | |
| | `gaussian/pandwm_gs_decoder_G3b_2000.pth` | Feed-forward Gaussian decoder (3D and 4D Gaussians) | 220 MB | `6450e980772adc46e2303960be06a26ab98f24805df2547af5a9ef63ec403da1` | |
| | `refiner/pandwm_render_refiner_v1_4000.pth` | Render refiner for laterally shifted views | 13.6 GB | `824fde5807c5060c1443cabd23a5823d65723c63541be3d9c4ddb606efb3b25d` | |
|
|
| All files are PyTorch state dicts saved with `torch.save`. Verify them after download with `sha256sum -c SHA256SUMS`. |
|
|
| ## Required third-party weights (not included) |
|
|
| | Weights | Used for | Source | |
| |---|---|---| |
| | Stable Diffusion 3.5 Medium | VAE and text encoders of the video model and the refiner | [stabilityai/stable-diffusion-3.5-medium](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | |
| | Depth Anything 3 (`DA3-LARGE-1.1`, `DA3METRIC-LARGE`) | Range relabelling of the Gaussians | [depth-anything/DA3-LARGE-1.1](https://huggingface.co/depth-anything/DA3-LARGE-1.1), [depth-anything/DA3METRIC-LARGE](https://huggingface.co/depth-anything/DA3METRIC-LARGE), code: [Depth-Anything-3](https://github.com/ByteDance-Seed/Depth-Anything-3) | |
| | I3D (`i3d_pretrained_400.pt`) | FVD | [TATS](https://github.com/songweige/TATS) | |
|
|
| Inception (FID), VGG (LPIPS) and the other evaluation backbones are downloaded by `torch`/`torchmetrics` on first use. |
|
|
| ## Usage |
|
|
| See the [code repository](https://github.com/AIGeeksGroup/PanDWM) for environment setup, data preparation and the scripts that reproduce each table. |
|
|
| ## License |
|
|
| The video model and the render refiner are derived from Stable Diffusion 3.5 Medium and are distributed under the [Stability AI Community License](https://stability.ai/community-license-agreement). Powered by Stability AI. |
|
|
| Third-party weights listed above are not redistributed here and remain under their own licenses. Training and evaluation use the [nuScenes](https://www.nuscenes.org/) dataset under its terms of use. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{sun2026pandwm, |
| title={PanDWM: Panoramic 4D Driving World Models}, |
| author={Sun, Jiajun and Zhang, Zeyu and Ren, Zeyu and Jiang, Guangfeng and Zhao, Yang and Tang, Hao}, |
| year={2026} |
| } |
| ``` |
|
|
