File size: 16,501 Bytes
e5a9386 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 | ---
library_name: pytorch
license: other
tags:
- generative_ai
- android
pipeline_tag: unconditional-image-generation
---

# Stable-Diffusion-v2.1: Optimized for Qualcomm Devices
Generates high resolution images from text prompts using a latent diffusion model. This model uses CLIP ViT-L/14 as text encoder, U-Net based latent denoising, and VAE based decoder to generate the final image.
This is based on the implementation of Stable-Diffusion-v2.1 found [here](https://github.com/CompVis/stable-diffusion/tree/main).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/stable_diffusion_v2_1) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
## Deploy to Snapdragon X Elite NPU
Please follow the [Stable Diffusion Windows App](https://github.com/quic/ai-hub-apps/tree/main/apps/windows/python/StableDiffusion) tutorial to quantize model with custom weights.
## Quantize and Deploy Your Own Fine-Tuned Stable Diffusion
Please follow the [Quantize Stable Diffusion]({REPOSITORY_URL}/tutorials/stable_diffusion/quantize_stable_diffusion.md) tutorial to quantize model with custom weights.
## Getting Started
There are two ways to deploy this model on your device:
### Option 1: Download Pre-Exported Models
Below are pre-exported model assets ready for deployment.
| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X2 Elite | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_x2_elite.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_x_elite.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_8gen3.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_qcs8550_proxy.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_8_elite_for_galaxy.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_7gen4.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_8_elite_gen5.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_qcm6690.zip)
| PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-precompiled_qnn_onnx-w8a16-qualcomm_qcs9075.zip)
| QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X2 Elite | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_snapdragon_x2_elite.zip)
| QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_snapdragon_x_elite.zip)
| QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_snapdragon_8gen3.zip)
| QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_qcs8550_proxy.zip)
| QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_sa8775p.zip)
| QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_snapdragon_8_elite_for_galaxy.zip)
| QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_snapdragon_7gen4.zip)
| QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_snapdragon_8_elite_gen5.zip)
| QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_sa7255p.zip)
| QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_qcm6690.zip)
| QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/stable_diffusion_v2_1/releases/v0.48.0/stable_diffusion_v2_1-qnn_context_binary-w8a16-qualcomm_qcs9075.zip)
For more device-specific assets and performance metrics, visit **[Stable-Diffusion-v2.1 on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/stable_diffusion_v2_1)**.
### Option 2: Export with Custom Configurations
Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/stable_diffusion_v2_1) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations
This option is ideal if you need to customize the model beyond the default configuration provided here.
See our repository for [Stable-Diffusion-v2.1 on GitHub](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/stable_diffusion_v2_1) for usage instructions.
## Model Details
**Model Type:** Model_use_case.image_generation
**Model Stats:**
- Input: Text prompt to generate image
## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X2 Elite | 3.513 ms | 378 - 378 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | 8.344 ms | 378 - 378 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 5.642 ms | 0 - 11 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 7.965 ms | 0 - 386 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | 7.986 ms | 0 - 3 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | 74.111 ms | 0 - 10 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 4.626 ms | 0 - 7 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 11.572 ms | 0 - 7 MB | NPU
| text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4.294 ms | 0 - 10 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X2 Elite | 3.815 ms | 0 - 0 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | 8.258 ms | 0 - 0 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | 5.719 ms | 0 - 7 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | 18.684 ms | 0 - 8 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | 8.007 ms | 0 - 1 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | 8.362 ms | 0 - 8 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | 8.054 ms | 0 - 2 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | 72.011 ms | 0 - 6 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | 18.684 ms | 0 - 8 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 4.696 ms | 0 - 13 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | 11.593 ms | 0 - 6 MB | NPU
| text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 4.301 ms | 0 - 9 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X2 Elite | 40.18 ms | 845 - 845 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | 96.06 ms | 843 - 843 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 69.984 ms | 0 - 11 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 96.024 ms | 0 - 898 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | 89.962 ms | 0 - 3 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | 2207.671 ms | 0 - 11 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 55.839 ms | 0 - 11 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 177.521 ms | 0 - 11 MB | NPU
| unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 38.417 ms | 0 - 9 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X2 Elite | 40.433 ms | 0 - 0 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | 96.801 ms | 0 - 0 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | 68.974 ms | 0 - 7 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | 233.267 ms | 0 - 7 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | 96.543 ms | 0 - 2 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | 90.188 ms | 0 - 7 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | 89.54 ms | 0 - 2 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | 2232.46 ms | 0 - 6 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | 233.267 ms | 0 - 7 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 53.753 ms | 1 - 9 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | 179.017 ms | 0 - 7 MB | NPU
| unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 39.84 ms | 0 - 9 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X2 Elite | 125.686 ms | 62 - 62 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | 264.42 ms | 62 - 62 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 202.83 ms | 3 - 10 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 266.85 ms | 0 - 66 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | 249.575 ms | 3 - 6 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | 6283.129 ms | 1 - 7 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 178.279 ms | 3 - 10 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 514.308 ms | 2 - 8 MB | NPU
| vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 123.483 ms | 3 - 13 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X2 Elite | 133.374 ms | 0 - 0 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | 260.818 ms | 0 - 0 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | 202.09 ms | 0 - 8 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | 718.975 ms | 0 - 8 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | 264.333 ms | 0 - 2 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | 250.394 ms | 0 - 9 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | 247.568 ms | 0 - 3 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | 6352.675 ms | 0 - 6 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | 718.975 ms | 0 - 8 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 175.481 ms | 0 - 14 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | 514.429 ms | 0 - 6 MB | NPU
| vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 127.588 ms | 0 - 10 MB | NPU
## License
* The license for the original implementation of Stable-Diffusion-v2.1 can be found
[here](https://github.com/CompVis/stable-diffusion/blob/main/LICENSE).
## References
* [High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752)
* [Source Model Implementation](https://github.com/CompVis/stable-diffusion/tree/main)
## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
## Usage and Limitations
This model may not be used for or in connection with any of the following applications:
- Accessing essential private and public services and benefits;
- Administration of justice and democratic processes;
- Assessing or recognizing the emotional state of a person;
- Biometric and biometrics-based systems, including categorization of persons based on sensitive characteristics;
- Education and vocational training;
- Employment and workers management;
- Exploitation of the vulnerabilities of persons resulting in harmful behavior;
- General purpose social scoring;
- Law enforcement;
- Management and operation of critical infrastructure;
- Migration, asylum and border control management;
- Predictive policing;
- Real-time remote biometric identification in public spaces;
- Recommender systems of social media platforms;
- Scraping of facial images (from the internet or otherwise); and/or
- Subliminal manipulation
|