Wherebot101 commited on
Commit
ab1e957
ยท
verified ยท
1 Parent(s): 920adda

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +67 -0
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen-Image-Edit
4
+ pipeline_tag: image-to-image
5
+ tags:
6
+ - comfyui
7
+ - novel-view-synthesis
8
+ - camera-control
9
+ - fp8
10
+ - qwen-image-edit
11
+ - diffusion-single-file
12
+ ---
13
+
14
+ # MetaView for ComfyUI (fp8-scaled, Lightning 8-step fused)
15
+
16
+ Single-file ComfyUI checkpoint of **[MetaView](https://huggingface.co/Kwai-Kolors/MetaView)** โ€” *Monocular Novel View Synthesis with Scale-Aware Implicit Geometry Priors* (ECCV 2026, [arXiv:2607.12000](https://arxiv.org/abs/2607.12000)) โ€” pre-fused and quantized for fast, low-VRAM inference. Give it one photo and a camera move (yaw / pitch / radius) and it re-renders the scene from the new viewpoint.
17
+
18
+ ## What's inside
19
+
20
+ | Component | Detail |
21
+ |---|---|
22
+ | Base | [Qwen/Qwen-Image-Edit](https://huggingface.co/Qwen/Qwen-Image-Edit) transformer (original release) |
23
+ | Graft | MetaView `model-2500-best` delta: `_3D_in` projection + per-block `prope_attn` parallel 3D attention (PRoPE camera-aware rope), kept bf16 |
24
+ | Speed | [Qwen-Image-Edit-Lightning-8steps-V1.0](https://huggingface.co/lightx2v/Qwen-Image-Lightning) LoRA fused at alpha 0.125 โ†’ **8 steps, CFG 1.0** |
25
+ | Quantization | 841 base layers fp8_e4m3fn, per-tensor scale = amax/448, ComfyUI-native `weight_scale` convention; sensitive layers (img_in / txt_in / proj_out / norms / biases) and all MetaView layers bf16 |
26
+ | Size | 26.5 GB (vs ~41 GB bf16 stack) โ€” runs on a 16 GB GPU via ComfyUI weight streaming |
27
+
28
+ Quantization is effectively lossless in practice (max per-layer relative error ~0.03% typical; outputs match the bf16 reference within run-to-run sampler noise).
29
+
30
+ ## Requirements
31
+
32
+ Load with the **[Enviral Design node pack](https://github.com/EnviralDesign/comfyUI-enviral-design-node-pack)** (`MetaViewModelLoader`, `MetaViewDA3Loader`, `MetaView3DConditioning`) โ€” the grafted architecture is not detected by stock loaders. Everything else is stock ComfyUI.
33
+
34
+ Companion models:
35
+
36
+ | Model | File / dir | Where |
37
+ |---|---|---|
38
+ | Text encoder | `qwen_2.5_vl_7b_fp8_scaled.safetensors` โ†’ `models/text_encoders/` | [Comfy-Org/Qwen-Image_ComfyUI](https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI) |
39
+ | VAE | `qwen_image_vae.safetensors` โ†’ `models/vae/` | [Comfy-Org/Qwen-Image_ComfyUI](https://huggingface.co/Comfy-Org/Qwen-Image_ComfyUI) |
40
+ | 3D features | `DA3-GIANT-1.1/` (whole dir) โ†’ `models/depth_anything_3/` | [depth-anything/DA3-GIANT-1.1](https://huggingface.co/depth-anything/DA3-GIANT-1.1) |
41
+ | Depth | `DA3NESTED-GIANT-LARGE-1.1/` (whole dir) โ†’ `models/depth_anything_3/` | [depth-anything/DA3NESTED-GIANT-LARGE-1.1](https://huggingface.co/depth-anything/DA3NESTED-GIANT-LARGE-1.1) |
42
+
43
+ This file goes in `models/diffusion_models/`.
44
+
45
+ ## Usage
46
+
47
+ Example workflow ships with the node pack (`workflows/`). Wiring summary:
48
+
49
+ ```
50
+ LoadImage โ”€โ”€โ”ฌโ”€โ–ถ MetaView3DConditioning (yaw / pitch / radius, auto_radius) โ”€โ–ถ positive โ”€โ”
51
+ โ””โ”€โ–ถ TextEncodeQwenImageEdit (image, no vae) โ”‚
52
+ MetaViewModelLoader โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ถ KSampler โ”€โ”€โ”€โ”€โ”ดโ”€โ–ถ VAEDecode โ”€โ–ถ Save
53
+ MetaViewDA3Loader โ”€โ–ถ (da3_model) LATENT from MetaView3DConditioning โ”€โ–ถ KSampler
54
+ ```
55
+
56
+ - **Sampler:** 8 steps, **CFG 1.0**, euler / simple (Lightning is fused โ€” do not add another speed LoRA).
57
+ - **Prompt:** the trigger phrase `้•œๅคด่ง†่ง’่ฝฌๅˆฐๆŒ‡ๅฎšไฝ็ฝฎ` ("move the camera to the specified position"). Extra text after it is best-effort โ€” the model's first-class use is promptless scene mutation.
58
+ - **Camera:** yaw/pitch in degrees; `auto_radius` derives orbit radius from scene depth (recommended).
59
+ - Wire the **VAE into MetaView3DConditioning** (not into the text encode node) so the reference latent lands at generation resolution.
60
+ - Tip: launch ComfyUI with `--fast fp8_matrix_mult` to run the fp8 layers on tensor cores.
61
+
62
+ ## License & attribution
63
+
64
+ This checkpoint is a merge of Apache-2.0 licensed weights and is released under **Apache-2.0**:
65
+ [Kwai-Kolors/MetaView](https://huggingface.co/Kwai-Kolors/MetaView) ยฉ Kuaishou (ECCV 2026) ยท [Qwen/Qwen-Image-Edit](https://huggingface.co/Qwen/Qwen-Image-Edit) ยฉ Alibaba Cloud ยท [lightx2v Qwen-Image-Lightning](https://huggingface.co/lightx2v/Qwen-Image-Lightning).
66
+
67
+ Note: the Depth-Anything-3 models used by the conditioning stage (not contained in this file) are **CC-BY-NC-4.0 (non-commercial)** โ€” evaluate accordingly for commercial pipelines.