README.md

---
base_model:
- Wan-AI/Wan2.2-I2V-A14B
- Wan-AI/Wan2.2-TI2V-5B
- Wan-AI/Wan2.1-I2V-14B-720P
library_name: diffusers
license: apache-2.0
tags:
- diffusion-single-file
- comfyui
- distillation
- LoRA
- video
- video generation
- sparse-attention
pipeline_tag: text-to-video
---

# Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention

This repository contains the weights and artifacts for **Light Forcing**, the first sparse attention solution tailored for autoregressive (AR) video generation models.

[![arXiv](https://img.shields.io/badge/arXiv-2602.04789-b31b1b)](https://huggingface.co/papers/2602.04789)
[![GitHub](https://img.shields.io/badge/GitHub-LightForcing-blue?logo=github)](https://github.com/chengtao-lv/LightForcing)

Light Forcing introduces a *Chunk-Aware Growth* mechanism and *Hierarchical Sparse Attention* to capture informative historical and local context. It enables significant end-to-end speedups (e.g., up to 3.0× on an RTX 5090) for models like Wan2.1 and Wan2.2 while maintaining high visual quality.

## 🚀 Quick Start

### Fast Inference

To use Light Forcing for video generation, please refer to the official [GitHub repository](https://github.com/chengtao-lv/LightForcing) for environment setup and model weights.

**For short-video generation (e.g., 5s):**

```shell
python inference.py \
  --config_path configs/light_forcing_short.yaml \
  --output_folder videos/light_forcing_short \
  --checkpoint_path path/to/short_video_gen.pt \
  --data_path prompts/MovieGenVideoBench_extended.txt \
  --use_ema
```

**For long-video generation (e.g., 15s):**

```shell
python inference.py \
  --config_path configs/light_forcing_long.yaml \
  --output_folder videos/light_forcing_long \
  --checkpoint_path path/to/long_video_gen.pt \
  --data_path prompts/MovieGenVideoBench_extended.txt \
  --use_ema \
  --num_output_frames 63
```

---

# 🎨 LightVAE

## ⚡ Efficient Video Autoencoder (VAE) Model Collection

*From Official Models to Lightx2v Distilled Optimized Versions - Balancing Quality, Speed and Memory*
![img_lightx2v](https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/tTnp8-ARpj3wGxfo5P55c.png)

---

[![🤗 HuggingFace](https://img.shields.io/badge/🤗-HuggingFace-yellow)](https://huggingface.co/lightx2v)
[![GitHub](https://img.shields.io/badge/GitHub-LightX2V-blue?logo=github)](https://github.com/ModelTC/LightX2V)
[![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE)

---

For VAE, the LightX2V team has conducted a series of deep optimizations, deriving two major series: **LightVAE** and **LightTAE**, which significantly reduce memory consumption and improve inference speed while maintaining high quality.

## 💡 Core Advantages

<table>
<tr>
<td width="50%">

### 📊 Official VAE
**Features**: Highest Quality ⭐⭐⭐⭐⭐

✅ Best reconstruction accuracy  
✅ Complete detail preservation  
❌ Large memory usage (~8-12 GB)  
❌ Slow inference speed

</td>
<td width="50%">

### 🚀 Open Source TAE Series
**Features**: Fastest Speed ⚡⚡⚡⚡⚡

✅ Minimal memory usage (~0.4 GB)  
✅ Extremely fast inference  
❌ Average quality ⭐⭐⭐  
❌ Potential detail loss

</td>
</tr>
<tr>
<td width="50%">

### 🎯 **LightVAE Series** (Our Optimization)
**Features**: Best Balanced Solution ⚖️

✅ Uses **Causal 3D Conv** (same as official)  
✅ **Quality close to official** ⭐⭐⭐⭐  
✅ Memory reduced by **~50%** (~4-5 GB)  
✅ Speed increased by **2-3x**  
✅ Balances quality, speed, and memory 🏆

</td>
<td width="50%">

### ⚡ **LightTAE Series** (Our Optimization)
**Features**: Fast Speed + Good Quality 🏆

✅ Minimal memory usage (~0.4 GB)  
✅ Extremely fast inference  
✅ **Quality close to official** ⭐⭐⭐⭐  
✅ **Significantly surpasses open source TAE**

</td>
</tr>
</table>

---

## 📦 Available Models

### 🎯 Wan2.1 Series VAE

| Model Name | Type | Architecture | Description | 
|:--------|:-----|:-----|:-----|
| `Wan2.1_VAE` | Official VAE | Causal Conv3D | Wan2.1 official video VAE model<br>**Highest quality, large memory, slow speed** |
| `taew2_1` | Open Source Small AE | Conv2D | Open source model based on [taeHV](https://github.com/madebyollin/taeHV)<br>**Small memory, fast speed, average quality** |
| **`lighttaew2_1`** | **LightTAE Series** | Conv2D | **Our distilled optimized version based on `taew2_1`**<br>**Small memory, fast speed, quality close to official** ✨ |
| **`lightvaew2_1`** | **LightVAE Series** | Causal Conv3D | **Our pruned 75% on WanVAE2.1 architecture then trained+distilled**<br>**Best balance: high quality + low memory + fast speed** 🏆 |

### 🎯 Wan2.2 Series VAE

| Model Name | Type | Architecture | Description | 
|:--------|:-----|:-----|:-----|
| `Wan2.2_VAE` | Official VAE | Causal Conv3D | Wan2.2 official video VAE model<br>**Highest quality, large memory, slow speed** |
| `taew2_2` | Open Source Small AE | Conv2D | Open source model based on [taeHV](https://github.com/madebyollin/taeHV)<br>**Small memory, fast speed, average quality** |
| **`lighttaew2_2`** | **LightTAE Series** | Conv2D | **Our distilled optimized version based on `taew2_2`**<br>**Small memory, fast speed, quality close to official** ✨ |

---

## 📊 Performance Comparison

### Video Reconstruction (Wan2.1 Series, 5s 81-frame video)
- **Precision**: BF16 | **Hardware**: NVIDIA H100

| Speed | Wan2.1_VAE | taew2_1 | lighttaew2_1 | lightvaew2_1 |
|:-----|:--------------|:------------|:---------------------|:-------------|
| **Encode Speed** | 4.1721 s | 0.3956 s | 0.3956 s | 1.5014s |
| **Decode Speed** | 5.4649 s | 0.2463 s | 0.2463 s | 2.0697s |

| GPU Memory | Wan2.1_VAE | taew2_1 | lighttaew2_1 | lightvaew2_1 |
|:-----|:--------------|:------------|:---------------------|:-------------|
| **Encode Memory** | 8.4954 GB | 0.00858 GB | 0.00858 GB | 4.7631 GB |
| **Decode Memory** | 10.1287 GB | 0.41199 GB | 0.41199 GB | 5.5673 GB |

## 🧪 VAE Reconstruction Test

You can test the VAE models independently using the standalone script provided in the repository:

```bash
# Test LightVAE (Wan2.1)
python -m lightx2v.models.video_encoders.hf.vid_recon \
    input_video.mp4 \
    --checkpoint ./models/vae/lightvaew2_1.pth \
    --model_type vaew2_1 \
    --device cuda \
    --dtype bfloat16 \
    --use_lightvae
```

## 📑 Citation

```bibtex
@article{lv2026light,
  title={Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention},
  author={Lv, Chengtao and Shi, Yumeng and Huang, Yushi and Gong, Ruihao and Ren, Shen and Wang, Wenya},
  journal={arXiv preprint arXiv:2602.04789},
  year={2026}
}
```

## 🤝 Community & Support

- **GitHub Issues**: [ModelTC/LightX2V](https://github.com/ModelTC/LightX2V/issues)
- **LightX2V Homepage**: [https://github.com/ModelTC/LightX2V](https://github.com/ModelTC/LightX2V)