yley commited on
Commit
15c52cf
·
verified ·
1 Parent(s): 24c4f04

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +47 -0
README.md ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - text-to-video
5
+ - image-to-video
6
+ - neural-materials
7
+ - wan
8
+ - siggraph
9
+ ---
10
+
11
+ # VideoNeuMat — weights
12
+
13
+ Model weights for **VideoNeuMat: Neural Material Extraction from Generative Video Models** (SIGGRAPH 2026).
14
+
15
+ - Project page: https://bowenxueai.github.io/VideoNeuMat/
16
+ - Code: https://github.com/bowenxueai/videoneumatcode
17
+
18
+ The pipeline turns a **text prompt (or image)** into an 81-frame 1024×1024 material video with a
19
+ fine-tuned Wan-2.1 video model, then a **feed-forward LRM** extracts a re-renderable neural material
20
+ (BRDF + displacement) from that video.
21
+
22
+ ## Files
23
+
24
+ | path | what | base model |
25
+ |---|---|---|
26
+ | `wan14b/step-10000.safetensors` | **T2V** material generator — full fine-tuned DiT (27 GB) | Wan2.1-T2V-14B |
27
+ | `wan14b_t2v_lora/step-9000.safetensors` | **T2V** material generator — LoRA (293 MB) | Wan2.1-T2V-14B |
28
+ | `wan14b_i2v/step-9500.safetensors` | **I2V** material generator — full fine-tuned DiT (31 GB) | Wan2.1-I2V-14B |
29
+ | `lrm/latent_module.pth` | **LRM** encoder — material → latent (feed-forward) | — |
30
+ | `lrm/mlp.pth` | shared neural-material **MLP** decoder | — |
31
+
32
+ All three generators produce the same 81-pose sparse-rig material video that the LRM consumes.
33
+ The base Wan-2.1 weights (`Wan-AI/Wan2.1-T2V-14B`, `Wan-AI/Wan2.1-I2V-14B`) are downloaded separately
34
+ from the official Wan-AI repos — see the code repo's README and `release/download_weights.sh`.
35
+
36
+ The generator weights are fine-tunes of Wan-2.1 (Apache-2.0). The VideoNeuMat code is MIT-licensed.
37
+
38
+ ## Citation
39
+
40
+ ```bibtex
41
+ @inproceedings{xue2026videoneumat,
42
+ author = {Xue, Bowen and Hadadan, Saeed and Zeng, Zheng and Rousselle, Fabrice and Montazeri, Zahra and Hasan, Milos},
43
+ title = {VideoNeuMat: Neural Material Extraction from Generative Video Models},
44
+ booktitle = {ACM SIGGRAPH 2026 Conference Papers},
45
+ year = {2026},
46
+ }
47
+ ```