Instructions to use zihaosheng/NWM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use zihaosheng/NWM with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("zihaosheng/NWM", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Neutral World Model (NWM)
Breaking Survivorship Bias in Driving World Models
Paper | Project Page | GitHub
Model Description
The Neutral World Model (NWM) is a controllable video diffusion model for autonomous driving, built upon Stable Video Diffusion (SVD-XT) with a ControlNet adapter. Given an initial camera image, an ego trajectory, and optional agent bounding boxes, it generates 25 future frames at 512 ร 320 resolution.
Training Data
NWM is trained on routine driving data from nuScenes and real accident videos from MM-AU. A two-stage training procedure and a bounding-box-guided objective improve the modeling of surrounding agents and collision dynamics.
Usage
Model weights are available on Hugging Face. Installation and inference instructions are provided in the GitHub repository.
License
The source code uses the MIT License. Pretrained SVD components are subject to their upstream licensing terms.
- Downloads last month
- -