| --- |
| license: apache-2.0 |
| tags: |
| - text-to-video |
| - image-to-video |
| - neural-materials |
| - wan |
| - siggraph |
| --- |
| |
| # VideoNeuMat — weights |
|
|
| Model weights for **VideoNeuMat: Neural Material Extraction from Generative Video Models** (SIGGRAPH 2026). |
|
|
| - Project page: https://bowenxueai.github.io/VideoNeuMat/ |
| - Code: https://github.com/bowenxueai/videoneumatcode |
|
|
| The pipeline turns a **text prompt (or image)** into an 81-frame 1024×1024 material video with a |
| fine-tuned Wan-2.1 video model, then a **feed-forward LRM** extracts a re-renderable neural material |
| (BRDF + displacement) from that video. |
|
|
| ## Files |
|
|
| | path | what | base model | |
| |---|---|---| |
| | `wan14b/step-10000.safetensors` | **T2V** material generator — full fine-tuned DiT (27 GB) | Wan2.1-T2V-14B | |
| | `wan14b_t2v_lora/step-9000.safetensors` | **T2V** material generator — LoRA (293 MB) | Wan2.1-T2V-14B | |
| | `wan14b_i2v/step-9500.safetensors` | **I2V** material generator — full fine-tuned DiT (31 GB) | Wan2.1-I2V-14B | |
| | `lrm/latent_module.pth` | **LRM** encoder — material → latent (feed-forward) | — | |
| | `lrm/mlp.pth` | shared neural-material **MLP** decoder | — | |
|
|
| All three generators produce the same 81-pose sparse-rig material video that the LRM consumes. |
| The base Wan-2.1 weights (`Wan-AI/Wan2.1-T2V-14B`, `Wan-AI/Wan2.1-I2V-14B`) are downloaded separately |
| from the official Wan-AI repos — see the code repo's README and `release/download_weights.sh`. |
|
|
| The generator weights are fine-tunes of Wan-2.1 (Apache-2.0). The VideoNeuMat code is MIT-licensed. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{xue2026videoneumat, |
| author = {Xue, Bowen and Hadadan, Saeed and Zeng, Zheng and Rousselle, Fabrice and Montazeri, Zahra and Hasan, Milos}, |
| title = {VideoNeuMat: Neural Material Extraction from Generative Video Models}, |
| booktitle = {ACM SIGGRAPH 2026 Conference Papers}, |
| year = {2026}, |
| } |
| ``` |
|
|
