Image-to-Video
Diffusers
Safetensors
Wan2.2
ti2v
world-model
video-generation
camera-control
long-video-generation
autoregressive
diffusion
transformer
Instructions to use GD-ML/DreamX-World-5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use GD-ML/DreamX-World-5B with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("GD-ML/DreamX-World-5B", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Wan2.2
How to use GD-ML/DreamX-World-5B with Wan2.2:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
| 3 |
tags:
|
| 4 |
- world-model
|
| 5 |
- video-generation
|
|
@@ -95,26 +95,26 @@ In the current inference script, `action_speed_list` is used as the relative dur
|
|
| 95 |
|
| 96 |
### Camera Action Commands
|
| 97 |
|
| 98 |
-
DreamX-World-5B uses `WASD` for camera translation and `IJKL` for
|
| 99 |
|
| 100 |
-
| Action |
|
| 101 |
-
|--------|-------------|
|
| 102 |
-
| `w` |
|
| 103 |
-
| `s` |
|
| 104 |
| `a` | Move left |
|
| 105 |
| `d` | Move right |
|
| 106 |
-
| `i` |
|
| 107 |
-
| `k` |
|
| 108 |
-
| `j` |
|
| 109 |
-
| `l` |
|
| 110 |
|
| 111 |
Actions can be composed in one string:
|
| 112 |
|
| 113 |
-
- `wi`:
|
| 114 |
-
- `wk`:
|
| 115 |
-
- `wj`:
|
| 116 |
-
- `wl`:
|
| 117 |
-
- `dj`: move right while
|
| 118 |
|
| 119 |
### Run Inference
|
| 120 |
|
|
@@ -178,27 +178,19 @@ Because the default causal block size is 3 latent frames, `num_output_frames` sh
|
|
| 178 |
| **Causal Block Size** | 3 latent frames per block by default |
|
| 179 |
| **VAE** | Wan2.2 VAE, temporal compression 4x, spatial compression 16x |
|
| 180 |
| **Text Encoder** | UMT5-XXL |
|
| 181 |
-
| **Denoising Steps** | `[1000, 750, 500, 250]` in the provided AR config |
|
| 182 |
| **Precision** | BFloat16 |
|
| 183 |
|
| 184 |
-
##
|
| 185 |
|
| 186 |
-
|
| 187 |
-
- Camera actions are approximate controls, not guaranteed physical camera trajectories.
|
| 188 |
-
- The provided inference script resizes inputs to 704 x 1280 and uses fixed normalized camera intrinsics.
|
| 189 |
-
- `action_speed_list` controls relative segment duration in the provided script; it does not directly change the per-frame camera velocity.
|
| 190 |
-
- Inference requires a CUDA GPU and substantial VRAM. Longer generations require more time and memory.
|
| 191 |
-
- Generated videos may reflect biases or artifacts from the training data and base model.
|
| 192 |
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
The model should not be used to create deceptive, harmful, illegal, or rights-violating content, or to represent generated videos as real-world footage.
|
| 198 |
|
| 199 |
## License
|
| 200 |
|
| 201 |
-
This model is released under the [
|
| 202 |
|
| 203 |
## Citation
|
| 204 |
|
|
|
|
| 1 |
---
|
| 2 |
+
license: mit
|
| 3 |
tags:
|
| 4 |
- world-model
|
| 5 |
- video-generation
|
|
|
|
| 95 |
|
| 96 |
### Camera Action Commands
|
| 97 |
|
| 98 |
+
DreamX-World-5B uses `WASD` for camera translation and `IJKL` for camera rotation:
|
| 99 |
|
| 100 |
+
| Action | Camera Control |
|
| 101 |
+
|--------|----------------|
|
| 102 |
+
| `w` | Push in |
|
| 103 |
+
| `s` | Pull out |
|
| 104 |
| `a` | Move left |
|
| 105 |
| `d` | Move right |
|
| 106 |
+
| `i` | Tilt up |
|
| 107 |
+
| `k` | Tilt down |
|
| 108 |
+
| `j` | Pan left |
|
| 109 |
+
| `l` | Pan right |
|
| 110 |
|
| 111 |
Actions can be composed in one string:
|
| 112 |
|
| 113 |
+
- `wi`: push in while tilting up
|
| 114 |
+
- `wk`: push in while tilting down
|
| 115 |
+
- `wj`: push in while panning left
|
| 116 |
+
- `wl`: push in while panning right
|
| 117 |
+
- `dj`: move right while panning left
|
| 118 |
|
| 119 |
### Run Inference
|
| 120 |
|
|
|
|
| 178 |
| **Causal Block Size** | 3 latent frames per block by default |
|
| 179 |
| **VAE** | Wan2.2 VAE, temporal compression 4x, spatial compression 16x |
|
| 180 |
| **Text Encoder** | UMT5-XXL |
|
|
|
|
| 181 |
| **Precision** | BFloat16 |
|
| 182 |
|
| 183 |
+
## WeChat Group
|
| 184 |
|
| 185 |
+
Join our WeChat group for discussion:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 186 |
|
| 187 |
+
<div align="center">
|
| 188 |
+
<img src="assets/wechat_group_qrcode.png" width="300" alt="WeChat Group QR Code">
|
| 189 |
+
</div>
|
|
|
|
|
|
|
| 190 |
|
| 191 |
## License
|
| 192 |
|
| 193 |
+
This model is released under the [MIT License](LICENSE).
|
| 194 |
|
| 195 |
## Citation
|
| 196 |
|