PeppX commited on
Commit
5480438
Β·
verified Β·
1 Parent(s): 1e8f51c

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +57 -0
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: diffusers
4
+ tags:
5
+ - video
6
+ - diffusion
7
+ - vae
8
+ - decoder
9
+ - wan
10
+ - pixel-space
11
+ - pid
12
+ pipeline_tag: image-to-video
13
+ ---
14
+
15
+ # Video-PiD: Pixel-Space Decoder for Wan 2.1
16
+
17
+ A small 3D pixel-space diffusion model that runs on top of Wan 2.1's VAE-decoded video frames to fix the "plastic" / waxy look of latent diffusion decoders.
18
+
19
+ **Status: pre-alpha. No weights yet.** This repo will hold the trained checkpoints. The source code lives at [github.com/madxmoron/video-pid](https://github.com/madxmoron/video-pid).
20
+
21
+ ```
22
+ Wan 2.1 1.3B T2V Video-PiD
23
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” Wan-VAE β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” residual β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
24
+ β”‚ text │──▢ decode ─────▢│ pixel │──▢ denoise ──▢│ pixel │──▢ video
25
+ β”‚ latent β”‚ (plastic) β”‚ frames β”‚ (4 steps) β”‚ frames β”‚ (sharp)
26
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
27
+ ```
28
+
29
+ ## Why
30
+
31
+ Latent diffusion decoders (Wan-VAE, SD-VAE, etc.) throw away high-frequency detail and re-introduce a "waxy" smoothness. Video-PiD is a tiny post-pass that re-denoises the decoded frames in pixel space, conditioning on the original latent, and outputs a residual that adds back the detail.
32
+
33
+ Inspired by NVIDIA's [PiD](https://research.nvidia.com/labs/sil/projects/pid/) (image-only). We extend it to video, in 3D, as a plug-in for Wan 2.1.
34
+
35
+ ## Roadmap
36
+
37
+ - [ ] Architecture spec pinned (in progress)
38
+ - [ ] Training run on Panda-70M / HD-VGGT
39
+ - [ ] First checkpoint release (v0.1.0)
40
+ - [ ] ComfyUI node
41
+
42
+ See [github.com/madxmoron/video-pid/blob/main/docs/ROADMAP.md](https://github.com/madxmoron/video-pid/blob/main/docs/ROADMAP.md) for the full plan.
43
+
44
+ ## License
45
+
46
+ Apache 2.0.
47
+
48
+ ## Citation
49
+
50
+ ```bibtex
51
+ @software{video_pid_2026,
52
+ author = {madxmoron},
53
+ title = {Video-PiD: Pixel-Space Decoder for Wan 2.1},
54
+ year = {2026},
55
+ url = {https://github.com/madxmoron/video-pid}
56
+ }
57
+ ```