Update README.md
Browse files
README.md
CHANGED
|
@@ -1,14 +1,42 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
|
| 6 |

|
| 7 |
|
| 8 |
-
`lyraDiff-Flux.1-dev` converted from the standard [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) model weights using this [script](https://github.com/TMElyralab/lyraDiff/blob/main/lyradiff/convert_model_scripts/quantize.py) to be compatiable with [lyraDiff](https://github.com/TMElyralab/lyraDiff)
|
| 9 |
|
| 10 |
It contains both `FP8` and `FP16` version of converted Flux.1-dev
|
| 11 |
|
| 12 |
## Usage
|
| 13 |
|
| 14 |
-
We provide a reference implementation of lyraDiff version of Flux.1-dev, as well as sampling code, in a dedicated [github repository](https://github.com/TMElyralab/lyraDiff).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
<h2 align="center">lyraDiff: An Out-of-box Acceleration Engine for Diffusion and DiT Models</h2>
|
| 5 |
+
<div align="center">
|
| 6 |
+
Sa Xiao<sup>*</sup>, Yibo Lu<sup>*</sup>, Kangjian Wu<sup>*</sup>, Bin Wu<sup>†</sup>, Haoxiong Su, Mian Peng, Qiwen Mao, Wenjiang Zhou</br>(*co-first author), (†Corresponding Author, benbinwu@tencent.com)</br>
|
| 7 |
+
Lyra Lab, Tencent Music Entertainment</br>
|
| 8 |
+
<p>[<a href="https://github.com/TMElyralab/lyraDiff">github</a>] [<a href="https://huggingface.co/TMElyralab/lyraDiff">huggingface</a>] </p>
|
| 9 |
+
</div>
|
| 10 |
+
|
| 11 |
+
## Introduction
|
| 12 |
+
|
| 13 |
+
🌈`lyraDiff` is currently the **Fastest Diffusion Acceleration Engine** that **doesn't need recompilation with dynamic input shapes**.
|
| 14 |
+
|
| 15 |
+
The core features include:
|
| 16 |
+
- 🚀 **State-of-the-art Inference Speed**: `lyraDiff` utilizes multiple techniques to achieve up to 2x speedup of the model inference, including **Quantization**, **Fused GEMM Kernels**, **Flash Attention**, and **NHWC & Fused GroupNorm**.
|
| 17 |
+
- 🔥 **Memory Efficiency**: `lyraDiff` utilizes buffer-based DRAM reuse strategy and multiple types of quantizations (FP8/INT8/INT4) to save **10-40%** of DRAM usage.
|
| 18 |
+
- 🔥 **Extensive Model Support**: `lyraDiff` supports a wide range of Generative/SR models such as **SD1.5, SDXL, FLUX, S3Diff, SUPIR, etc.**, and those most commonly used plugins such as **LoRA, ControlNet and Ip-Adapter**.
|
| 19 |
+
- 🔥 **Zero Compilation Deployment**: Unlike **TensorRT** or **AITemplate**, which takes minutes to compile, `lyraDiff` eliminates runtime recompilation overhead even with model inputs of dynamic shapes.
|
| 20 |
+
- 🔥 **Image Gen Consistency**: The outputs of `lyraDiff` are aligned with the ones of [HF diffusers](https://github.com/huggingface/diffusers) at the pixel level, even under LoRA switch in quantization mode.
|
| 21 |
+
- 🚀 **Fast Plugin Hot-swap**: `lyraDiff` provides **Super Fast Model Hot-swap for ControlNet and LoRA** which can hugely benefit a real-time image gen service.
|
| 22 |
|
| 23 |
|
| 24 |

|
| 25 |
|
| 26 |
+
`lyraDiff-Flux.1-dev` is converted from the standard [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) model weights using this [script](https://github.com/TMElyralab/lyraDiff/blob/main/lyradiff/convert_model_scripts/quantize.py) to be compatiable with [lyraDiff](https://github.com/TMElyralab/lyraDiff)
|
| 27 |
|
| 28 |
It contains both `FP8` and `FP16` version of converted Flux.1-dev
|
| 29 |
|
| 30 |
## Usage
|
| 31 |
|
| 32 |
+
We provide a reference implementation of lyraDiff version of Flux.1-dev, as well as sampling code, in a dedicated [github repository](https://github.com/TMElyralab/lyraDiff).
|
| 33 |
+
|
| 34 |
+
## Citation
|
| 35 |
+
``` bibtex
|
| 36 |
+
@Misc{lyraDiff_2025,
|
| 37 |
+
author = {Kangjian Wu, Zhengtao Wang, Yibo Lu, Haoxiong Su, Sa Xiao, Qiwen Mao, Mian Peng, Bin Wu, Wenjiang Zhou},
|
| 38 |
+
title = {lyraDiff: Accelerating Diffusion Models with best flexibility},
|
| 39 |
+
howpublished = {\url{https://github.com/TMElyralab/lyraDiff}},
|
| 40 |
+
year = {2025}
|
| 41 |
+
}
|
| 42 |
+
```
|