yibolu96 commited on
Commit
ec51ec0
·
verified ·
1 Parent(s): bbfef35

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -2
README.md CHANGED
@@ -1,14 +1,42 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
 
5
 
6
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6461b412846a6c8c8305319d/41_dvtx232Kzu8MkY6qEx.png)
7
 
8
- `lyraDiff-Flux.1-dev` converted from the standard [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) model weights using this [script](https://github.com/TMElyralab/lyraDiff/blob/main/lyradiff/convert_model_scripts/quantize.py) to be compatiable with [lyraDiff](https://github.com/TMElyralab/lyraDiff)
9
 
10
  It contains both `FP8` and `FP16` version of converted Flux.1-dev
11
 
12
  ## Usage
13
 
14
- We provide a reference implementation of lyraDiff version of Flux.1-dev, as well as sampling code, in a dedicated [github repository](https://github.com/TMElyralab/lyraDiff).
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ <h2 align="center">lyraDiff: An Out-of-box Acceleration Engine for Diffusion and DiT Models</h2>
5
+ <div align="center">
6
+ Sa Xiao<sup>*</sup>, Yibo Lu<sup>*</sup>, Kangjian Wu<sup>*</sup>, Bin Wu<sup>†</sup>, Haoxiong Su, Mian Peng, Qiwen Mao, Wenjiang Zhou</br>(*co-first author), (†Corresponding Author, benbinwu@tencent.com)</br>
7
+ Lyra Lab, Tencent Music Entertainment</br>
8
+ <p>[<a href="https://github.com/TMElyralab/lyraDiff">github</a>] [<a href="https://huggingface.co/TMElyralab/lyraDiff">huggingface</a>] </p>
9
+ </div>
10
+
11
+ ## Introduction
12
+
13
+ 🌈`lyraDiff` is currently the **Fastest Diffusion Acceleration Engine** that **doesn't need recompilation with dynamic input shapes**.
14
+
15
+ The core features include:
16
+ - 🚀 **State-of-the-art Inference Speed**: `lyraDiff` utilizes multiple techniques to achieve up to 2x speedup of the model inference, including **Quantization**, **Fused GEMM Kernels**, **Flash Attention**, and **NHWC & Fused GroupNorm**.
17
+ - 🔥 **Memory Efficiency**: `lyraDiff` utilizes buffer-based DRAM reuse strategy and multiple types of quantizations (FP8/INT8/INT4) to save **10-40%** of DRAM usage.
18
+ - 🔥 **Extensive Model Support**: `lyraDiff` supports a wide range of Generative/SR models such as **SD1.5, SDXL, FLUX, S3Diff, SUPIR, etc.**, and those most commonly used plugins such as **LoRA, ControlNet and Ip-Adapter**.
19
+ - 🔥 **Zero Compilation Deployment**: Unlike **TensorRT** or **AITemplate**, which takes minutes to compile, `lyraDiff` eliminates runtime recompilation overhead even with model inputs of dynamic shapes.
20
+ - 🔥 **Image Gen Consistency**: The outputs of `lyraDiff` are aligned with the ones of [HF diffusers](https://github.com/huggingface/diffusers) at the pixel level, even under LoRA switch in quantization mode.
21
+ - 🚀 **Fast Plugin Hot-swap**: `lyraDiff` provides **Super Fast Model Hot-swap for ControlNet and LoRA** which can hugely benefit a real-time image gen service.
22
 
23
 
24
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6461b412846a6c8c8305319d/41_dvtx232Kzu8MkY6qEx.png)
25
 
26
+ `lyraDiff-Flux.1-dev` is converted from the standard [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) model weights using this [script](https://github.com/TMElyralab/lyraDiff/blob/main/lyradiff/convert_model_scripts/quantize.py) to be compatiable with [lyraDiff](https://github.com/TMElyralab/lyraDiff)
27
 
28
  It contains both `FP8` and `FP16` version of converted Flux.1-dev
29
 
30
  ## Usage
31
 
32
+ We provide a reference implementation of lyraDiff version of Flux.1-dev, as well as sampling code, in a dedicated [github repository](https://github.com/TMElyralab/lyraDiff).
33
+
34
+ ## Citation
35
+ ``` bibtex
36
+ @Misc{lyraDiff_2025,
37
+ author = {Kangjian Wu, Zhengtao Wang, Yibo Lu, Haoxiong Su, Sa Xiao, Qiwen Mao, Mian Peng, Bin Wu, Wenjiang Zhou},
38
+ title = {lyraDiff: Accelerating Diffusion Models with best flexibility},
39
+ howpublished = {\url{https://github.com/TMElyralab/lyraDiff}},
40
+ year = {2025}
41
+ }
42
+ ```