README.md

metadata

base_model:
  - Wan-AI/Wan2.2-I2V-A14B
  - Wan-AI/Wan2.2-TI2V-5B
  - Wan-AI/Wan2.1-I2V-14B-720P
library_name: diffusers
license: apache-2.0
tags:
  - diffusion-single-file
  - comfyui
  - distillation
  - LoRA
  - video
  - video generation
  - sparse-attention
pipeline_tag: text-to-video

Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention

This repository contains the weights and artifacts for Light Forcing, the first sparse attention solution tailored for autoregressive (AR) video generation models.

Light Forcing introduces a Chunk-Aware Growth mechanism and Hierarchical Sparse Attention to capture informative historical and local context. It enables significant end-to-end speedups (e.g., up to 3.0× on an RTX 5090) for models like Wan2.1 and Wan2.2 while maintaining high visual quality.

🚀 Quick Start

Fast Inference

To use Light Forcing for video generation, please refer to the official GitHub repository for environment setup and model weights.

For short-video generation (e.g., 5s):

python inference.py \
  --config_path configs/light_forcing_short.yaml \
  --output_folder videos/light_forcing_short \
  --checkpoint_path path/to/short_video_gen.pt \
  --data_path prompts/MovieGenVideoBench_extended.txt \
  --use_ema

For long-video generation (e.g., 15s):

python inference.py \
  --config_path configs/light_forcing_long.yaml \
  --output_folder videos/light_forcing_long \
  --checkpoint_path path/to/long_video_gen.pt \
  --data_path prompts/MovieGenVideoBench_extended.txt \
  --use_ema \
  --num_output_frames 63

---

🎨 LightVAE

⚡ Efficient Video Autoencoder (VAE) Model Collection

From Official Models to Lightx2v Distilled Optimized Versions - Balancing Quality, Speed and Memory

---

---

For VAE, the LightX2V team has conducted a series of deep optimizations, deriving two major series: LightVAE and LightTAE, which significantly reduce memory consumption and improve inference speed while maintaining high quality.

💡 Core Advantages

📊 Official VAE Features: Highest Quality ⭐⭐⭐⭐⭐ ✅ Best reconstruction accuracy ✅ Complete detail preservation ❌ Large memory usage (~8-12 GB) ❌ Slow inference speed	🚀 Open Source TAE Series Features: Fastest Speed ⚡⚡⚡⚡⚡ ✅ Minimal memory usage (~0.4 GB) ✅ Extremely fast inference ❌ Average quality ⭐⭐⭐ ❌ Potential detail loss
🎯 LightVAE Series (Our Optimization) Features: Best Balanced Solution ⚖️ ✅ Uses Causal 3D Conv (same as official) ✅ Quality close to official ⭐⭐⭐⭐ ✅ Memory reduced by ~50% (~4-5 GB) ✅ Speed increased by 2-3x ✅ Balances quality, speed, and memory 🏆	⚡ LightTAE Series (Our Optimization) Features: Fast Speed + Good Quality 🏆 ✅ Minimal memory usage (~0.4 GB) ✅ Extremely fast inference ✅ Quality close to official ⭐⭐⭐⭐ ✅ Significantly surpasses open source TAE

---

📦 Available Models

🎯 Wan2.1 Series VAE

Model Name	Type	Architecture	Description
Wan2.1_VAE	Official VAE	Causal Conv3D	Wan2.1 official video VAE model Highest quality, large memory, slow speed
taew2_1	Open Source Small AE	Conv2D	Open source model based on taeHV Small memory, fast speed, average quality
lighttaew2_1	LightTAE Series	Conv2D	Our distilled optimized version based on taew2_1 Small memory, fast speed, quality close to official ✨
lightvaew2_1	LightVAE Series	Causal Conv3D	Our pruned 75% on WanVAE2.1 architecture then trained+distilled Best balance: high quality + low memory + fast speed 🏆

🎯 Wan2.2 Series VAE

Model Name	Type	Architecture	Description
Wan2.2_VAE	Official VAE	Causal Conv3D	Wan2.2 official video VAE model Highest quality, large memory, slow speed
taew2_2	Open Source Small AE	Conv2D	Open source model based on taeHV Small memory, fast speed, average quality
lighttaew2_2	LightTAE Series	Conv2D	Our distilled optimized version based on taew2_2 Small memory, fast speed, quality close to official ✨

---

📊 Performance Comparison

Video Reconstruction (Wan2.1 Series, 5s 81-frame video)

• Precision: BF16 | Hardware: NVIDIA H100

Speed	Wan2.1_VAE	taew2_1	lighttaew2_1	lightvaew2_1
Encode Speed	4.1721 s	0.3956 s	0.3956 s	1.5014s
Decode Speed	5.4649 s	0.2463 s	0.2463 s	2.0697s

GPU Memory	Wan2.1_VAE	taew2_1	lighttaew2_1	lightvaew2_1
Encode Memory	8.4954 GB	0.00858 GB	0.00858 GB	4.7631 GB
Decode Memory	10.1287 GB	0.41199 GB	0.41199 GB	5.5673 GB

🧪 VAE Reconstruction Test

You can test the VAE models independently using the standalone script provided in the repository:

# Test LightVAE (Wan2.1)
python -m lightx2v.models.video_encoders.hf.vid_recon \
    input_video.mp4 \
    --checkpoint ./models/vae/lightvaew2_1.pth \
    --model_type vaew2_1 \
    --device cuda \
    --dtype bfloat16 \
    --use_lightvae

📑 Citation

@article{lv2026light,
  title={Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention},
  author={Lv, Chengtao and Shi, Yumeng and Huang, Yushi and Gong, Ruihao and Ren, Shen and Wang, Wenya},
  journal={arXiv preprint arXiv:2602.04789},
  year={2026}
}

🤝 Community & Support

• GitHub Issues: ModelTC/LightX2V
• LightX2V Homepage: https://github.com/ModelTC/LightX2V