--- base_model: - Wan-AI/Wan2.2-I2V-A14B - Wan-AI/Wan2.2-TI2V-5B - Wan-AI/Wan2.1-I2V-14B-720P library_name: diffusers license: apache-2.0 tags: - diffusion-single-file - comfyui - distillation - LoRA - video - video generation - sparse-attention pipeline_tag: text-to-video --- # Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention This repository contains the weights and artifacts for **Light Forcing**, the first sparse attention solution tailored for autoregressive (AR) video generation models. [![arXiv](https://img.shields.io/badge/arXiv-2602.04789-b31b1b)](https://huggingface.co/papers/2602.04789) [![GitHub](https://img.shields.io/badge/GitHub-LightForcing-blue?logo=github)](https://github.com/chengtao-lv/LightForcing) Light Forcing introduces a *Chunk-Aware Growth* mechanism and *Hierarchical Sparse Attention* to capture informative historical and local context. It enables significant end-to-end speedups (e.g., up to 3.0ร— on an RTX 5090) for models like Wan2.1 and Wan2.2 while maintaining high visual quality. ## ๐Ÿš€ Quick Start ### Fast Inference To use Light Forcing for video generation, please refer to the official [GitHub repository](https://github.com/chengtao-lv/LightForcing) for environment setup and model weights. **For short-video generation (e.g., 5s):** ```shell python inference.py \ --config_path configs/light_forcing_short.yaml \ --output_folder videos/light_forcing_short \ --checkpoint_path path/to/short_video_gen.pt \ --data_path prompts/MovieGenVideoBench_extended.txt \ --use_ema ``` **For long-video generation (e.g., 15s):** ```shell python inference.py \ --config_path configs/light_forcing_long.yaml \ --output_folder videos/light_forcing_long \ --checkpoint_path path/to/long_video_gen.pt \ --data_path prompts/MovieGenVideoBench_extended.txt \ --use_ema \ --num_output_frames 63 ``` --- # ๐ŸŽจ LightVAE ## โšก Efficient Video Autoencoder (VAE) Model Collection *From Official Models to Lightx2v Distilled Optimized Versions - Balancing Quality, Speed and Memory* ![img_lightx2v](https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/tTnp8-ARpj3wGxfo5P55c.png) --- [![๐Ÿค— HuggingFace](https://img.shields.io/badge/๐Ÿค—-HuggingFace-yellow)](https://huggingface.co/lightx2v) [![GitHub](https://img.shields.io/badge/GitHub-LightX2V-blue?logo=github)](https://github.com/ModelTC/LightX2V) [![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE) --- For VAE, the LightX2V team has conducted a series of deep optimizations, deriving two major series: **LightVAE** and **LightTAE**, which significantly reduce memory consumption and improve inference speed while maintaining high quality. ## ๐Ÿ’ก Core Advantages
### ๐Ÿ“Š Official VAE **Features**: Highest Quality โญโญโญโญโญ โœ… Best reconstruction accuracy โœ… Complete detail preservation โŒ Large memory usage (~8-12 GB) โŒ Slow inference speed ### ๐Ÿš€ Open Source TAE Series **Features**: Fastest Speed โšกโšกโšกโšกโšก โœ… Minimal memory usage (~0.4 GB) โœ… Extremely fast inference โŒ Average quality โญโญโญ โŒ Potential detail loss
### ๐ŸŽฏ **LightVAE Series** (Our Optimization) **Features**: Best Balanced Solution โš–๏ธ โœ… Uses **Causal 3D Conv** (same as official) โœ… **Quality close to official** โญโญโญโญ โœ… Memory reduced by **~50%** (~4-5 GB) โœ… Speed increased by **2-3x** โœ… Balances quality, speed, and memory ๐Ÿ† ### โšก **LightTAE Series** (Our Optimization) **Features**: Fast Speed + Good Quality ๐Ÿ† โœ… Minimal memory usage (~0.4 GB) โœ… Extremely fast inference โœ… **Quality close to official** โญโญโญโญ โœ… **Significantly surpasses open source TAE**
--- ## ๐Ÿ“ฆ Available Models ### ๐ŸŽฏ Wan2.1 Series VAE | Model Name | Type | Architecture | Description | |:--------|:-----|:-----|:-----| | `Wan2.1_VAE` | Official VAE | Causal Conv3D | Wan2.1 official video VAE model
**Highest quality, large memory, slow speed** | | `taew2_1` | Open Source Small AE | Conv2D | Open source model based on [taeHV](https://github.com/madebyollin/taeHV)
**Small memory, fast speed, average quality** | | **`lighttaew2_1`** | **LightTAE Series** | Conv2D | **Our distilled optimized version based on `taew2_1`**
**Small memory, fast speed, quality close to official** โœจ | | **`lightvaew2_1`** | **LightVAE Series** | Causal Conv3D | **Our pruned 75% on WanVAE2.1 architecture then trained+distilled**
**Best balance: high quality + low memory + fast speed** ๐Ÿ† | ### ๐ŸŽฏ Wan2.2 Series VAE | Model Name | Type | Architecture | Description | |:--------|:-----|:-----|:-----| | `Wan2.2_VAE` | Official VAE | Causal Conv3D | Wan2.2 official video VAE model
**Highest quality, large memory, slow speed** | | `taew2_2` | Open Source Small AE | Conv2D | Open source model based on [taeHV](https://github.com/madebyollin/taeHV)
**Small memory, fast speed, average quality** | | **`lighttaew2_2`** | **LightTAE Series** | Conv2D | **Our distilled optimized version based on `taew2_2`**
**Small memory, fast speed, quality close to official** โœจ | --- ## ๐Ÿ“Š Performance Comparison ### Video Reconstruction (Wan2.1 Series, 5s 81-frame video) - **Precision**: BF16 | **Hardware**: NVIDIA H100 | Speed | Wan2.1_VAE | taew2_1 | lighttaew2_1 | lightvaew2_1 | |:-----|:--------------|:------------|:---------------------|:-------------| | **Encode Speed** | 4.1721 s | 0.3956 s | 0.3956 s | 1.5014s | | **Decode Speed** | 5.4649 s | 0.2463 s | 0.2463 s | 2.0697s | | GPU Memory | Wan2.1_VAE | taew2_1 | lighttaew2_1 | lightvaew2_1 | |:-----|:--------------|:------------|:---------------------|:-------------| | **Encode Memory** | 8.4954 GB | 0.00858 GB | 0.00858 GB | 4.7631 GB | | **Decode Memory** | 10.1287 GB | 0.41199 GB | 0.41199 GB | 5.5673 GB | ## ๐Ÿงช VAE Reconstruction Test You can test the VAE models independently using the standalone script provided in the repository: ```bash # Test LightVAE (Wan2.1) python -m lightx2v.models.video_encoders.hf.vid_recon \ input_video.mp4 \ --checkpoint ./models/vae/lightvaew2_1.pth \ --model_type vaew2_1 \ --device cuda \ --dtype bfloat16 \ --use_lightvae ``` ## ๐Ÿ“‘ Citation ```bibtex @article{lv2026light, title={Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention}, author={Lv, Chengtao and Shi, Yumeng and Huang, Yushi and Gong, Ruihao and Ren, Shen and Wang, Wenya}, journal={arXiv preprint arXiv:2602.04789}, year={2026} } ``` ## ๐Ÿค Community & Support - **GitHub Issues**: [ModelTC/LightX2V](https://github.com/ModelTC/LightX2V/issues) - **LightX2V Homepage**: [https://github.com/ModelTC/LightX2V](https://github.com/ModelTC/LightX2V)