DreamX-Creator 1.0: Model Weights
DreamX Team
[](https://arxiv.org/abs/2608.31106)
[](../LICENSE)
-----
This directory holds all model weights for **DreamX-Creator 1.0**, a research
framework for **native joint audio-video generation**. Given a first frame and
a text prompt, the 7B base generator jointly models modality-specialized video
and audio streams; the **Autoregressive 1-Step 2K Refiner** (SR-DiT 5B) then
upgrades the generated video to high-quality 2K output.
Weights are **not stored in the git repository**. They are distributed on
[HuggingFace]() and [ModelScope]() and should be placed under this directory
following the layout below.
## Expected Layout
```
checkpoints/
├── creator/ # DreamX-Creator 1.0 joint generator (7B, LoRA merged)
│ ├── video_model/ # video DiT shards + config
│ ├── audio_model/ # audio DiT + config
│ └── cross_attn_weights.safetensors # gated A2V/V2A cross-modal attention
├── audio_vae/ # CreatorDACVAE audio VAE
├── refiner/ # 2K refiner
│ ├── sr_dit_5b.pt # SR-DiT 5B refiner
│ ├── latent_upsampler_flash.pt # FlashLatentUpsampler (default)
│ ├── latent_upsampler_2d_causal.pt # causal 2D latent upsampler (optional)
│ └── lightvae_nu_scheme3.pt # distilled fast decoder (optional, off by default)
└── wan2.2_ti2v_5b/ # shared Wan2.2-TI2V-5B dependencies
├── Wan2.2_VAE.pth # video VAE
├── models_t5_umt5-xxl-enc-bf16.pth # UMT5-xxl text encoder
└── google/umt5-xxl/ # tokenizer
```
The `wan2.2_ti2v_5b/` directory can also be downloaded directly from
[Wan-AI/Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B); only the
three entries above are needed. It is shared by both pipelines.
## Which Weights Are Used Where
- **Joint audio-video generation** uses `creator/`, `audio_vae/`, and
`wan2.2_ti2v_5b/`. See the
[audio_video_generation README](../audio_video_generation/README.md) for
setup, input overrides, and memory options.
- **2K refinement** uses `refiner/` and `wan2.2_ti2v_5b/`. See the
[video_refiner README](../video_refiner/README.md) for setup and the full
list of inference knobs.
## Quickstart
Once the weights above are in place, from the repository root:
**1. Joint audio-video generation** (first frame + prompt to synchronized
video with audio):
```bash
cd audio_video_generation
pip install -r requirements.txt
./inference.sh # uses assets/case1.png by default
```
**2. 2K refinement** (super-resolve a generated or external video, audio
unchanged):
```bash
cd ../video_refiner
pip install -r requirements.txt
INPUT=/path/to/video.mp4 bash run_inference.sh
```
## Citation
If you find DreamX-Creator useful in your research, please consider citing our technical report:
```bibtex
@misc{zhu2026dreamxcreatordemocratizingnativeaudiovideo,
title={DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution},
author={Jiashu Zhu and Yanhao Zheng and Ruitian Tian and Rujing Dang and Shen Zhang and Bingze Song and Jiachen Lei and Ruimin Lin and Jiahong Wu and Xiangxiang Chu},
year={2026},
eprint={2608.31106},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2608.31106},
}
```
## License
This project is licensed under the Apache License 2.0. See [LICENSE](../LICENSE) for details.
## Acknowledgement
We would like to thank the [Wan Team](https://github.com/Wan-Video/Wan2.2), the [OpenMOSS Team](https://github.com/OpenMOSS/MOVA), and the [VideoX-Fun Team](https://github.com/aigc-apps/VideoX-Fun) for their outstanding open-source work on Wan, MOVA, and VideoX-Fun, respectively.