| | --- |
| | license: openrail++ |
| | tags: |
| | - text-to-image |
| | - stable-diffusion |
| | --- |
| | # SD-XL 1.0-base Model Card |
| |
|
| | > **Note:** This repository is a **mirror** and **not** the [original upstream source](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0). |
| | > The original model, weights, and documentation are developed and maintained by **Stability AI**. |
| | > |
| | > The model weights hosted here are **unmodified** and redistributed **as-is**. |
| | > Only minor editorial changes to this README (e.g. formatting or clarification) have been made and do **not** affect the model, its behavior, or its licensing. |
| | > |
| | > The model is released under the **CreativeML Open RAIL++-M License**, which permits use and redistribution **subject to explicit use-based restrictions** (see *Attachment A*). |
| | > A full copy of the license is included in this repository and applies to all distributions of the model and its derivatives. |
| | > |
| | > Users of this mirror are responsible for complying with all terms of the CreativeML Open RAIL++-M License. |
| | > |
| | > This repository is **not affiliated with or endorsed by Stability AI**. |
| | > The maintainer is willing to cooperate in good faith with the original rights holder regarding reasonable requests. |
| |
|
| | ## Model |
| |
|
| | [SDXL](https://arxiv.org/abs/2307.01952) consists of an [ensemble of experts](https://arxiv.org/abs/2211.01324) pipeline for latent diffusion: |
| | In a first step, the base model is used to generate (noisy) latents, |
| | which are then further processed with a refinement model (available here: https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/) specialized for the final denoising steps. |
| | Note that the base model can be used as a standalone module. |
| |
|
| | Alternatively, we can use a two-stage pipeline as follows: |
| | First, the base model is used to generate latents of the desired output size. |
| | In the second step, we use a specialized high-resolution model and apply a technique called SDEdit (https://arxiv.org/abs/2108.01073, also known as "img2img") |
| | to the latents generated in the first step, using the same prompt. This technique is slightly slower than the first one, as it requires more function evaluations. |
| |
|
| | Source code is available at https://github.com/Stability-AI/generative-models . |
| |
|
| | ### Model Description |
| |
|
| | - **Developed by:** Stability AI |
| | - **Model type:** Diffusion-based text-to-image generative model |
| | - **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/LICENSE.md) |
| | - **Model Description:** This is a model that can be used to generate and modify images based on text prompts. It is a [Latent Diffusion Model](https://arxiv.org/abs/2112.10752) that uses two fixed, pretrained text encoders ([OpenCLIP-ViT/G](https://github.com/mlfoundations/open_clip) and [CLIP-ViT/L](https://github.com/openai/CLIP/tree/main)). |
| | - **Resources for more information:** Check out our [GitHub Repository](https://github.com/Stability-AI/generative-models) and the [SDXL report on arXiv](https://arxiv.org/abs/2307.01952). |
| |
|
| | ### Model Sources |
| |
|
| | For research purposes, we recommend our `generative-models` Github repository (https://github.com/Stability-AI/generative-models), which implements the most popular diffusion frameworks (both training and inference) and for which new functionalities like distillation will be added over time. |
| | [Clipdrop](https://clipdrop.co/stable-diffusion) provides free SDXL inference. |
| |
|
| | - **Repository:** https://github.com/Stability-AI/generative-models |
| | - **Demo:** https://clipdrop.co/stable-diffusion |
| |
|
| | ## Uses |
| |
|
| | ### Direct Use |
| |
|
| | The model is intended for research purposes only. Possible research areas and tasks include |
| |
|
| | - Generation of artworks and use in design and other artistic processes. |
| | - Applications in educational or creative tools. |
| | - Research on generative models. |
| | - Safe deployment of models which have the potential to generate harmful content. |
| | - Probing and understanding the limitations and biases of generative models. |
| |
|
| | Excluded uses are described below. |
| |
|
| | ### Out-of-Scope Use |
| |
|
| | The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model. |
| |
|
| | ## Limitations and Bias |
| |
|
| | ### Limitations |
| |
|
| | - The model does not achieve perfect photorealism |
| | - The model cannot render legible text |
| | - The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere” |
| | - Faces and people in general may not be generated properly. |
| | - The autoencoding part of the model is lossy. |
| |
|
| | ### Bias |
| | While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases. |
| |
|
