README.md

metadata

base_model:
  - Wan-AI/Wan2.2-S2V-14B
language:
  - en
license: apache-2.0
pipeline_tag: image-to-video
tags:
  - lora
  - talking-head
  - audio-driven
  - avatar-generation

🎬 Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length

Yubo Huang^1,2 · Hailong Guo^2,3 · Fangtai Wu^2,4 · Shifeng Zhang² · Shijie Huang² · Qijun Gan⁴ · Lin Liu¹ · Sirui Zhao^1,* · Enhong Chen^1,* · Jiaming Liu^2,‡ · Steven Hoi²

¹ University of Science and Technology of China    ² Alibaba Group    ³ Beijing University of Posts and Telecommunications    ⁴ Zhejiang University

^* Corresponding authors.    ^‡ Project leader.

This repository contains the weights for the paper Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length.

TL;DR: Live Avatar is an algorithm–system co-designed framework that enables real-time, streaming, infinite-length interactive avatar video generation. Powered by a 14B-parameter diffusion model, it achieves 45 FPS on multi-card H800 GPUs with 4-step sampling and supports Block-wise Autoregressive processing for 10,000+ second streaming videos.

👀 More Demos:
🤖 Human-AI Conversation  |  ♾️ Infinite Video  |  🎭 Diverse Characters  |  🎬 Animated Tech Explanation
👉 Click Here to Visit Project Page! 🌐

---

✨ Highlights

• ⚡ ​​Real-time Streaming Interaction​​ - Achieve 45 FPS real-time streaming with low latency
• ♾️ ​​​​Infinite-length Autoregressive Generation​​​​ - Support 10,000+ second continuous video generation
• 🎨 ​​​​Generalization Performances​​​​ - Strong generalization across cartoon characters, singing, and diverse scenarios

---

📰 News

• [2026.1.20] 🚀 Major performance breakthrough (v1.1)! FP8 quantization enables inference on 48GB GPUs, while advanced compilation and cuDNN attention boost speed to ~2.5x peak and 3x average FPS. Achieving stable 45+ FPS on multi-H800.
• [2025.12.16] 🎉 LiveAvatar has reached 1,000+ stars on GitHub!
• [2025.12.12] 🚀 We released single-gpu inference Code — a single 80GB VRAM GPU is enough to enjoy.
• [2025.12.08] 🚀 We released real-time inference Code and the model Weight.
• [2025.12.08] 🎉 LiveAvatar won the Hugging Face #1 Paper of the day!
• [2025.12.04] 🔥 We released Paper and demo page Website.

---

📑 Todo List

🌟 Early December (core code release)

• ✅ Release the paper
• ✅ Release the demo website
• ✅ Release checkpoints on Hugging Face
• ✅ Release Gradio Web UI
• ✅ Experimental real-time streaming inference on at least H800 GPUs
  • ✅ Distribution-matching distillation to 4 steps
  • ✅ Timestep-forcing pipeline parallelism

⚙️ Later updates

• ✅ Inference code supporting single GPU (offline generation)
• ✅ Multi-character support
• ✅ Inference Acceleration Stage1 (RoPE optimization, compilation, LoRA merge)
• ✅ Streaming-VAE intergration
• ✅ Inference Acceleration Stage2 (further compilation, fp8, cudnn attn)
• ⬜ UI integration for easily streaming interaction
• ⬜ TTS integration
• ⬜ Training code
• ⬜ LiveAvatar v1.2

🛠️ Installation

Please follow the steps below to set up the environment.

1. Create Environment

conda create -n liveavatar python=3.10 -y
conda activate liveavatar

2. Install CUDA Dependencies (optional)

conda install nvidia/label/cuda-12.4.1::cuda -y
conda install -c nvidia/label/cuda-12.4.1 cudatoolkit -y

3. Install PyTorch & Flash Attention

pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128

# For H800/H200 setups:
pip install flash_attn_3 --find-links https://windreamer.github.io/flash-attention3-wheels/cu128_torch280 --extra-index-url https://download.pytorch.org/whl/cu128

# Otherwise:
pip install flash-attn==2.8.3 --no-build-isolation

4. Install Python Requirements

pip install -r requirements.txt

5. Install FFMPEG

apt-get update && apt-get install -y ffmpeg                 

---

📥 Download Models

Please download the pretrained checkpoints and place them in the ./ckpt/ directory.

Model Component	Description	Link
WanS2V-14B	base model	🤗 Huggingface
liveAvatar	our lora model	🤗 Huggingface

pip install "huggingface_hub[cli]"
huggingface-cli download Wan-AI/Wan2.2-S2V-14B --local-dir ./ckpt/Wan2.2-S2V-14B
huggingface-cli download Quark-Vision/Live-Avatar --local-dir ./ckpt/LiveAvatar

🚀 Inference

Real-time Inference with TPP

💡 Requires multi-GPU setup with at least 80GB VRAM.

# CLI Inference
bash infinite_inference_multi_gpu.sh
# Gradio Web UI
bash gradio_multi_gpu.sh

Single-GPU Inference

💡 Can run on a single GPU with at least 80GB VRAM.

# CLI Inference
bash infinite_inference_single_gpu.sh
# Gradio Web UI
bash gradio_single_gpu.sh

📝 Citation

If you find this project useful for your research, please consider citing our paper:

@misc{huang2025liveavatarstreamingrealtime,
      title={Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length}, 
      author={Yubo Huang and Hailong Guo and Fangtai Wu and Shifeng Zhang and Shijie Huang and Qijun Gan and Lin Liu and Sirui Zhao and Enhong Chen and Jiaming Liu and Steven Hoi},
      year={2025},
      eprint={2512.04677},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2512.04677}, 
}

📜 License Agreement

• The majority of this project is released under the Apache 2.0 license.
• The Wan model (base model) is also released under the Apache 2.0 license.