mack-williams commited on
Commit
3a80acb
Β·
verified Β·
1 Parent(s): a9a5f7b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +126 -1
README.md CHANGED
@@ -8,4 +8,129 @@ tags:
8
  - NVFP4
9
  - Sparse_Attention
10
  - Wan
11
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  - NVFP4
9
  - Sparse_Attention
10
  - Wan
11
+ ---
12
+ # 🎬 Wan2.2-NVFP4-Sparse
13
+
14
+ > **An extremely efficient Wan 2.2 14B variant: NVFP4 Quantization-Aware Step Distillation with Sparse Attention for Blackwell Architecture**
15
+
16
+ [![GitHub](https://img.shields.io/badge/GitHub-ModelTC/LightX2V-blue)](https://github.com/ModelTC/LightX2V)
17
+ [![HuggingFace](https://img.shields.io/badge/HuggingFace-lightx2v-yellow)](https://huggingface.co/lightx2v/)
18
+
19
+ ## πŸ“‹ Table of Contents
20
+
21
+ - [✨ Features](#-features)
22
+ - [πŸš€ Quick Start](#-quick-start)
23
+ - [🎬 Generation Results](#-generation-results)
24
+ - [⚑ Performance Comparison](#-performance-comparison)
25
+ - [⚠️ Notes](#️-notes)
26
+ - [🀝 Community](#-community)
27
+
28
+ ## ✨ Features
29
+
30
+ - **⚑ 4-Step Inference**: Two high-noise expert steps followed by two low-noise expert steps, enabling extremely fast Wan2.2 MoE generation on a single Blackwell GPU.
31
+ - **🎯 NVFP4 Quantization**: Quantization-aware step distillation reduces memory traffic and compute cost while targeting Blackwell architecture.
32
+ - **🧩 Sparse Attention**: Accelerates the costly O(n²) self-attention workload with sparse attention, reducing end-to-end latency for high-resolution video generation.
33
+ - **πŸ”§ LightX2V Integration**: Recommended runtime stack for stable deployment and best performance.
34
+ - **πŸš€ High-Quality Generation**: Preserves the visual quality of Wan2.2-T2V-14B while dramatically improving inference speed.
35
+
36
+ ## πŸš€ Quick Start
37
+
38
+ We strongly recommend using the official LightX2V Docker image for the cleanest environment and best reproducibility.
39
+
40
+ ### Option A: Docker Recommended
41
+
42
+ ```bash
43
+ # 1. Pull LightX2V Docker image
44
+ docker pull lightx2v/lightx2v:26052301-cu130-5090
45
+
46
+ # 2. Run inference
47
+ bash scripts/wan22/distill/run_wan22_moe_t2v_extreme.sh
48
+ ```
49
+
50
+ ### Option B: Manual Installation
51
+
52
+ If Docker is not available, install the environment manually:
53
+
54
+ ```bash
55
+ # 1. Install LightX2V
56
+ git clone https://github.com/ModelTC/LightX2V.git
57
+ cd LightX2V
58
+ uv pip install -v .
59
+
60
+ # 2. Install NVFP4 Kernel
61
+ pip install scikit_build_core uv
62
+ git clone https://github.com/NVIDIA/cutlass.git
63
+ cd lightx2v_kernel
64
+
65
+ MAX_JOBS=$(nproc) CMAKE_BUILD_PARALLEL_LEVEL=$(nproc) \
66
+ uv build --wheel \
67
+ -Cbuild-dir=build . \
68
+ -Ccmake.define.CUTLASS_PATH=/path/to/cutlass \
69
+ --verbose --color=always --no-build-isolation
70
+
71
+ pip install dist/*whl --force-reinstall --no-deps
72
+
73
+ # 3. Run inference
74
+ bash scripts/wan22/distill/run_wan22_moe_t2v_extreme.sh
75
+ ```
76
+
77
+ Script: [run_wan22_moe_t2v_extreme.sh](https://github.com/ModelTC/LightX2V/blob/main/scripts/wan22/distill/run_wan22_moe_t2v_extreme.sh)
78
+
79
+ ## 🎬 Generation Results
80
+
81
+ <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 8px; padding: 16px; margin: 16px 0;">
82
+ <p style="font-style: italic; color: #475569; margin: 0; padding: 12px; background: white; border-radius: 6px; border-left: 4px solid #3b82f6;">
83
+ "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage"
84
+ </p>
85
+ </div>
86
+
87
+
88
+ | Resolution | Wan2.2-T2V-14B | Wan2.2-NVFP4-Sparse |
89
+ | --- | --- | --- |
90
+ | 480p | <video controls style="width: 260px; height: 180px; border-radius: 6px; object-fit: cover;" src="https://cdn-uploads.huggingface.co/production/uploads/658e760cccbc1e2cc78b4258/WTHhrzx7XR4S1Ys_6Kzx4.mp4"></video> | <video controls style="width: 260px; height: 180px; border-radius: 6px; object-fit: cover;" src="https://cdn-uploads.huggingface.co/production/uploads/658e760cccbc1e2cc78b4258/zorpw7gm9At0J2kCmvkDr.mp4"></video> |
91
+ | 720p | <video controls style="width: 260px; height: 180px; border-radius: 6px; object-fit: cover;" src="https://cdn-uploads.huggingface.co/production/uploads/658e760cccbc1e2cc78b4258/vkiyKj7CJA-r0yTz7TEum.mp4"></video> | <video controls style="width: 260px; height: 180px; border-radius: 6px; object-fit: cover;" src="https://cdn-uploads.huggingface.co/production/uploads/658e760cccbc1e2cc78b4258/TuECbzvW5jI9NHG6GLvIR.mp4"></video> |
92
+
93
+
94
+ ## ⚑ Performance Comparison
95
+
96
+ **Test Environment**: RTX 5090 Single GPU | LightX2V Framework | End-to-End Latency
97
+
98
+ | Resolution | Wan2.2-T2V-14B | Wan2.2-NVFP4-Sparse | Speedup |
99
+ | --- | ---: | ---: | ---: |
100
+ | 480p | 734s | 14.15s | 51.9x |
101
+ | 720p | 2668s | 45s | 59.3x |
102
+
103
+ ## ⚠️ Notes
104
+
105
+ ### System Requirements
106
+
107
+ - **Required Hardware**: NVIDIA RTX 50-series GPUs or other Blackwell architecture GPUs.
108
+ - **Recommended Runtime**: `lightx2v/lightx2v:26052301-cu130-5090`.
109
+
110
+ ### Dependencies
111
+
112
+ - Prepare Wan2.2 T5 / VAE components following the standard LightX2V Wan2.2 model structure.
113
+ - Use Blackwell + NVFP4 kernels for optimal speed and memory efficiency.
114
+
115
+ ### Performance Tips
116
+
117
+ - Use the provided extreme inference script for the 4-step high-noise / low-noise expert schedule.
118
+ - Sparse attention is most beneficial at higher resolutions where self-attention dominates latency.
119
+ - Enable CPU offload only when GPU memory is limited, since offload can reduce throughput.
120
+
121
+ ## 🀝 Community
122
+
123
+ - **πŸ› Issues**: [GitHub Issues](https://github.com/ModelTC/LightX2V/issues)
124
+ - **πŸ€— Models**: [HuggingFace Hub](https://huggingface.co/lightx2v/)
125
+ - **πŸ“– Documentation**: [LightX2V Docs](https://github.com/ModelTC/LightX2V)
126
+
127
+ ---
128
+
129
+ <div align="center">
130
+
131
+ **If you find this project helpful, please give us a ⭐ on [GitHub](https://github.com/ModelTC/LightX2V)**
132
+
133
+ For questions or issues, please open an issue on [LightX2V](https://github.com/ModelTC/LightX2V/issues) or contact lvchengtao0319@gmail.com.
134
+
135
+ </div>
136
+