Duplicate from stabilityai/stable-diffusion-xl-1.0-tensorrt

Co-authored-by: pharmapsychotic <pharmapsychotic@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+lcm/unetxl.opt/dbf91c42-985c-11ee-9041-0242ac110002 filter=lfs diff=lfs merge=lfs -text
+lcmlora/unetxl-8c8ce9e8b00b259425e5f3eaa4b1d705-1.00.opt/1376e228-9608-11ee-9b07-0242ac110002 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,139 @@
+---
+library_name: tensorrt
+license: openrail++
+base_model: stabilityai/stable-diffusion-xl-base-1.0
+language:
+  - en
+tags:
+  - stable-diffusion
+  - stable-diffusion-xl
+  - stable-diffusion-xl-lcm
+  - stable-diffusion-xl-lcmlora
+  - tensorrt
+  - text-to-image
+---
+
+# Stable Diffusion XL 1.0 TensorRT
+
+## Introduction
+
+This repository hosts the TensorRT versions(sdxl, sdxl-lcm, sdxl-lcmlora) of **Stable Diffusion XL 1.0** created in collaboration with [NVIDIA](https://huggingface.co/nvidia). The optimized versions give substantial improvements in speed and efficiency.
+
+See the [usage instructions](#usage-example) for how to run the SDXL pipeline with the ONNX files hosted in this repository. 
+
+
+![examples](./examples.jpg)
+
+## Model Description
+
+- **Developed by:** Stability AI
+- **Model type:** Diffusion-based text-to-image generative model
+- **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/blob/main/LICENSE.md)
+- **Model Description:** This is a conversion of the [SDXL base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) and [SDXL refiner 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0) models for [NVIDIA TensorRT](https://developer.nvidia.com/tensorrt) optimized inference
+
+
+## Performance Comparison
+
+#### Timings for 30 steps at 1024x1024
+
+| Accelerator | Baseline (non-optimized) | NVIDIA TensorRT (optimized) | Percentage improvement |
+|-------------|--------------------------|-----------------------------|------------------------|
+| A10         | 9399 ms                  | 8160 ms                     | ~13%                   |
+| A100        | 3704 ms                  | 2742 ms                     | ~26%                   |
+| H100        | 2496 ms                  | 1471 ms                     | ~41%                   |
+
+#### Image throughput for 30 steps at 1024x1024
+
+| Accelerator | Baseline (non-optimized) | NVIDIA TensorRT (optimized) | Percentage improvement |
+|-------------|--------------------------|-----------------------------|------------------------|
+| A10         | 0.10 images/sec          | 0.12 images/sec             | ~20%                   |
+| A100        | 0.27 images/sec          | 0.36 images/sec             | ~33%                   |
+| H100        | 0.40 images/sec          | 0.68 images/sec             | ~70%                   |
+
+#### Timings for Latent Consistency Model(LCM) version for 4 steps at 1024x1024
+
+| Accelerator | CLIP                     | Unet                        | VAE                    |Total                   |
+|-------------|--------------------------|-----------------------------|------------------------|------------------------|
+| A100        | 1.08 ms                  | 192.02 ms                   | 228.34 ms              | 426.16 ms              |
+| H100        | 0.78 ms                  | 102.8 ms                    | 126.95 ms              | 234.22 ms              |
+
+
+## Usage Example
+
+1. Following the [setup instructions](https://github.com/rajeevsrao/TensorRT/blob/release/9.2/demo/Diffusion/README.md) on launching a TensorRT NGC container.
+```shell
+git clone https://github.com/rajeevsrao/TensorRT.git
+cd TensorRT
+git checkout release/9.2
+docker run --rm -it --gpus all -v $PWD:/workspace nvcr.io/nvidia/pytorch:23.11-py3 /bin/bash
+```
+
+2. Download the SDXL TensorRT files from this repo
+```shell
+git lfs install 
+git clone https://huggingface.co/stabilityai/stable-diffusion-xl-1.0-tensorrt
+cd stable-diffusion-xl-1.0-tensorrt
+git lfs pull
+cd ..
+```
+
+3. Install libraries and requirements
+```shell
+cd demo/Diffusion
+python3 -m pip install --upgrade pip
+pip3 install -r requirements.txt
+python3 -m pip install --pre --upgrade --extra-index-url https://pypi.nvidia.com tensorrt
+```
+
+4. Perform TensorRT optimized inference:
+
+  - **SDXL**
+    
+    The first invocation produces plan files in `engine_xl_base` and `engine_xl_refiner` specific to the accelerator being run on and are reused for later invocations. 
+
+    ```
+    python3 demo_txt2img_xl.py \
+      "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" \
+      --build-static-batch \
+      --use-cuda-graph \
+      --num-warmup-runs 1 \
+      --width 1024 \
+      --height 1024 \
+      --denoising-steps 30 \
+      --onnx-base-dir /workspace/stable-diffusion-xl-1.0-tensorrt/sdxl-1.0-base \
+      --onnx-refiner-dir /workspace/stable-diffusion-xl-1.0-tensorrt/sdxl-1.0-refiner
+    ```
+
+  - **SDXL-LCM**
+    
+    The first invocation produces plan files in --engine-dir specific to the accelerator being run on and are reused for later invocations. 
+    ```
+    python3 demo_txt2img_xl.py \
+      ""Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"" \
+      --version=xl-1.0 \
+      --onnx-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcm \
+      --engine-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcm/engine-sdxl-lcm-nocfg \
+      --scheduler LCM \
+      --denoising-steps 4 \
+      --guidance-scale 0.0 \
+      --seed 42
+    
+    ```
+  - **SDXL-LCMLORA**
+
+    The first invocation produces plan files in --engine-dir specific to the accelerator being run on and are reused for later invocations. 
+
+    ```
+    python3 demo_txt2img_xl.py \
+      ""Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"" \
+      --version=xl-1.0 \
+      --onnx-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcmlora \
+      --engine-dir /workspace/stable-diffusion-xl-1.0-tensorrt/lcm/engine-sdxl-lcmlora-nocfg \
+      --scheduler LCM \
+      --lora-path latent-consistency/lcm-lora-sdxl \
+      --lora-scale 1.0 \
+      --denoising-steps 4 \
+      --guidance-scale 0.0 \
+      --seed 42
+    
+    ```

lcm/clip.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c342a572b89967ec14697c57655f2b81d27172eed6de07b6f7ee91e3b914514
+size 322531134

lcm/clip2.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d85ae80d928b8a02b56572374a0faad41d5f4b82da473d973ebda9fbd89d970
+size 1517189726

lcm/unetxl.opt/dbf91c42-985c-11ee-9041-0242ac110002

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b146e7cda628219dfaf6c9924716e1cb94dc6c74bbf964761da7da7929a615f9
+size 5136090880

lcm/unetxl.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6470c2fbe084e33c7672401c269469d06330fc51a419c8c5e24bac44d78a0ef
+size 3369087

lcm/vae.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7045d54982c1bb7c8898d38a71e6fa9bbd0aaac5222fefafe49842ccb016507
+size 99186612

lcmlora/clip.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c342a572b89967ec14697c57655f2b81d27172eed6de07b6f7ee91e3b914514
+size 322531134

lcmlora/clip2.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d85ae80d928b8a02b56572374a0faad41d5f4b82da473d973ebda9fbd89d970
+size 1517189726

lcmlora/unetxl-8c8ce9e8b00b259425e5f3eaa4b1d705-1.00.opt/1376e228-9608-11ee-9b07-0242ac110002

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e096cfd4fa4f0eac4b554b0d3f80356f413533c4af657fc9f9d532493813271b
+size 5136090880

lcmlora/unetxl-8c8ce9e8b00b259425e5f3eaa4b1d705-1.00.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44abca57a01d64c79a7fb5d14e9e043b83d1358f8116defe3147e5241a3d3936
+size 3369087

lcmlora/vae.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7045d54982c1bb7c8898d38a71e6fa9bbd0aaac5222fefafe49842ccb016507
+size 99186612

sdxl-1.0-base/clip.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bcd2a625e64a43bd8d78168178c1383891c540a022d7984d86974a2b4661aba
+size 322531134

sdxl-1.0-base/clip2.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb3e48f933c5dfe6cf8ae9d2121818d37f239215b113df3405242254bae732a2
+size 1517189726

sdxl-1.0-base/unetxl.opt/435d4c0a-2d32-11ee-8476-0242c0a80101

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbdd1938d37406e9ea9889dbffdcd38f74da588fda7eb63b9351c491fd573853
+size 5136090880

sdxl-1.0-base/unetxl.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbd5a42d8c38934068eccc8ec08f5a63aca8eba7cda06b717dde6f3b665829bf
+size 6136637

sdxl-1.0-refiner/clip2.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb3e48f933c5dfe6cf8ae9d2121818d37f239215b113df3405242254bae732a2
+size 1517189726

sdxl-1.0-refiner/unetxl.opt/6e186582-2d74-11ee-8aa7-0242c0a80102

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b127bd75f1b4ae08c8c4915948ba4bb76ea489fbb611102e46d9470656900d5
+size 4519958016

sdxl-1.0-refiner/unetxl.opt/6ed855ee-2d70-11ee-af8e-0242c0a80101

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63cb1dfb3ccb10ef89ab20dd0edfaaae26635737586ab8ad21d97f195a8cc12b
+size 847120896

sdxl-1.0-refiner/unetxl.opt/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0448a6ee66e6a46bd396f11003ff7075a53fda7bfeef43854cf2acdc894d3ba1
+size 4040948