Instructions to use hlwang06/WorldDirector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use hlwang06/WorldDirector with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("hlwang06/WorldDirector", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Add model card, pipeline tag, and links to paper/code
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,3 +1,38 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-sa-4.0
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-sa-4.0
|
| 3 |
+
pipeline_tag: image-to-video
|
| 4 |
---
|
| 5 |
+
|
| 6 |
+
# WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory
|
| 7 |
+
|
| 8 |
+
[๐ Paper](https://huggingface.co/papers/2607.02517) | [๐ Project Page](https://worlddirector.github.io/) | [๐ GitHub Repository](https://github.com/pPetrichor/WorldDirector)
|
| 9 |
+
|
| 10 |
+
**WorldDirector** is a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration.
|
| 11 |
+
|
| 12 |
+
Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, this framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, this approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view.
|
| 13 |
+
|
| 14 |
+
## Setup & Inference
|
| 15 |
+
|
| 16 |
+
For instructions on environment setup, preparing latents/conditions, and running the causal video generation pipeline, please refer to the [WorldDirector GitHub repository](https://github.com/pPetrichor/WorldDirector).
|
| 17 |
+
|
| 18 |
+
### Model Download via CLI
|
| 19 |
+
|
| 20 |
+
You can download the model weights using `huggingface-cli` as follows:
|
| 21 |
+
|
| 22 |
+
```shell
|
| 23 |
+
pip install "huggingface_hub[cli]"
|
| 24 |
+
huggingface-cli download hlwang06/WorldDirector --local-dir ./WorldDirector-14B
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
## Citation
|
| 28 |
+
|
| 29 |
+
If you find this work useful, please consider citing our paper:
|
| 30 |
+
|
| 31 |
+
```bibtex
|
| 32 |
+
@article{wang2026worlddirector,
|
| 33 |
+
title={WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory},
|
| 34 |
+
author={Hanlin Wang and Hao Ouyang and Qiuyu Wang and Wen Wang and Qingyan Bai and Ka Leong Cheng and Yue Yu and Yixuan Li and Yihao Meng and Zichen Liu and Yanhong Zeng and Yujun Shen and Qifeng Chen},
|
| 35 |
+
journal={arXiv preprint arXiv:2607.02517},
|
| 36 |
+
year={2026}
|
| 37 |
+
}
|
| 38 |
+
```
|