image to image interface

README.md

@@ -2,14 +2,18 @@
 license: openrail++
 tags:
 - stable-diffusion
-- text-to-image
+- image-to-image
+widget:
+  - src: >-
+      https://huggingface.co/datasets/mishig/sample_images/resolve/main/canny-edge.jpg
+    prompt: Girl with Pearl Earring
 duplicated_from: stabilityai/stable-diffusion-2-1-base
 ---
 
 # Stable Diffusion v2-1-base Model Card
 This model card focuses on the model associated with the Stable Diffusion v2-1-base model.
 
-This `stable-diffusion-2-1-base` model fine-tunes [stable-diffusion-2-base](https://huggingface.co/stabilityai/stable-diffusion-2-base) (`512-base-ema.ckpt`) with 220k extra steps taken, with `punsafe=0.98` on the same dataset. 
+This `stable-diffusion-2-1-base` model fine-tunes [stable-diffusion-2-base](https://huggingface.co/stabilityai/stable-diffusion-2-base) (`512-base-ema.ckpt`) with 220k extra steps taken, with `punsafe=0.98` on the same dataset.
 
 - Use it with the [`stablediffusion`](https://github.com/Stability-AI/stablediffusion) repository: download the `v2-1_512-ema-pruned.ckpt` [here](https://huggingface.co/stabilityai/stable-diffusion-2-1-base/resolve/main/v2-1_512-ema-pruned.ckpt).
 - Use it with 🧨 [`diffusers`](#examples)
@@ -31,7 +35,7 @@ This `stable-diffusion-2-1-base` model fine-tunes [stable-diffusion-2-base](http
           year      = {2022},
           pages     = {10684-10695}
       }
-      
+
 
 ## Examples
 
@@ -53,8 +57,8 @@ pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, to
 pipe = pipe.to("cuda")
 
 prompt = "a photo of an astronaut riding a horse on mars"
-image = pipe(prompt).images[0]  
-    
+image = pipe(prompt).images[0]
+
 image.save("astronaut_rides_horse.png")
 ```
 
@@ -64,7 +68,7 @@ image.save("astronaut_rides_horse.png")
 
 # Uses
 
-## Direct Use 
+## Direct Use
 The model is intended for research purposes only. Possible research areas and tasks include
 
 - Safe deployment of models which have the potential to generate harmful content.
@@ -109,11 +113,11 @@ Using the model to generate content that is cruel to individuals is a misuse of
   [LAION-5B](https://laion.ai/blog/laion-5b/), which contains adult, violent and sexual content. To partially mitigate this, we have filtered the dataset using LAION's NFSW detector (see Training section).
 
 ### Bias
-While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases. 
-Stable Diffusion vw was primarily trained on subsets of [LAION-2B(en)](https://laion.ai/blog/laion-5b/), 
-which consists of images that are limited to English descriptions. 
-Texts and images from communities and cultures that use other languages are likely to be insufficiently accounted for. 
-This affects the overall output of the model, as white and western cultures are often set as the default. Further, the 
+While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
+Stable Diffusion vw was primarily trained on subsets of [LAION-2B(en)](https://laion.ai/blog/laion-5b/),
+which consists of images that are limited to English descriptions.
+Texts and images from communities and cultures that use other languages are likely to be insufficiently accounted for.
+This affects the overall output of the model, as white and western cultures are often set as the default. Further, the
 ability of the model to generate content with non-English prompts is significantly worse than with English-language prompts.
 Stable Diffusion v2 mirrors and exacerbates biases to such a degree that viewer discretion must be advised irrespective of the input or its intent.
 
@@ -126,7 +130,7 @@ The model developers used the following dataset for training the model:
 - LAION-5B and subsets (details below). The training data is further filtered using LAION's NSFW detector, with a "p_unsafe" score of 0.1 (conservative). For more details, please refer to LAION-5B's [NeurIPS 2022](https://openreview.net/forum?id=M3Y74vmsMcY) paper and reviewer discussions on the topic.
 
 **Training Procedure**
-Stable Diffusion v2 is a latent diffusion model which combines an autoencoder with a diffusion model that is trained in the latent space of the autoencoder. During training, 
+Stable Diffusion v2 is a latent diffusion model which combines an autoencoder with a diffusion model that is trained in the latent space of the autoencoder. During training,
 
 - Images are encoded through an encoder, which turns images into latent representations. The autoencoder uses a relative downsampling factor of 8 and maps images of shape H x W x 3 to latents of shape H/f x W/f x 4
 - Text prompts are encoded through the OpenCLIP-ViT/H text-encoder.
@@ -148,7 +152,7 @@ The additional input channels of the U-Net which process this extra information
 - `512-inpainting-ema.ckpt`: Resumed from `512-base-ema.ckpt` and trained for another 200k steps. Follows the mask-generation strategy presented in [LAMA](https://github.com/saic-mdal/lama) which, in combination with the latent VAE representations of the masked image, are used as an additional conditioning.
 The additional input channels of the U-Net which process this extra information were zero-initialized. The same strategy was used to train the [1.5-inpainting checkpoint](https://github.com/saic-mdal/lama).
 - `x4-upscaling-ema.ckpt`: Trained for 1.25M steps on a 10M subset of LAION containing images `>2048x2048`. The model was trained on crops of size `512x512` and is a text-guided [latent upscaling diffusion model](https://arxiv.org/abs/2112.10752).
-In addition to the textual input, it receives a `noise_level` as an input parameter, which can be used to add noise to the low-resolution input according to a [predefined diffusion schedule](configs/stable-diffusion/x4-upscaling.yaml). 
+In addition to the textual input, it receives a `noise_level` as an input parameter, which can be used to add noise to the low-resolution input according to a [predefined diffusion schedule](configs/stable-diffusion/x4-upscaling.yaml).
 
 - **Hardware:** 32 x 8 x A100 GPUs
 - **Optimizer:** AdamW
@@ -156,11 +160,11 @@ In addition to the textual input, it receives a `noise_level` as an input parame
 - **Batch:** 32 x 8 x 2 x 4 = 2048
 - **Learning rate:** warmup to 0.0001 for 10,000 steps and then kept constant
 
-## Evaluation Results 
+## Evaluation Results
 Evaluations with different classifier-free guidance scales (1.5, 2.0, 3.0, 4.0,
 5.0, 6.0, 7.0, 8.0) and 50 steps DDIM sampling steps show the relative improvements of the checkpoints:
 
-![pareto](https://huggingface.co/stabilityai/stable-diffusion-2/resolve/main/model-variants.jpg) 
+![pareto](https://huggingface.co/stabilityai/stable-diffusion-2/resolve/main/model-variants.jpg)
 
 Evaluated using 50 DDIM steps and 10000 random prompts from the COCO2017 validation set, evaluated at 512x512 resolution.  Not optimized for FID scores.
 
@@ -185,4 +189,4 @@ Based on that information, we estimate the following CO2 emissions using the [Ma
         pages     = {10684-10695}
     }
 
-*This model card was written by: Robin Rombach, Patrick Esser and David Ha and is based on the [Stable Diffusion v1](https://github.com/CompVis/stable-diffusion/blob/main/Stable_Diffusion_v1_Model_Card.md) and [DALL-E Mini model card](https://huggingface.co/dalle-mini/dalle-mini).*
+*This model card was written by: Robin Rombach, Patrick Esser and David Ha and is based on the [Stable Diffusion v1](https://github.com/CompVis/stable-diffusion/blob/main/Stable_Diffusion_v1_Model_Card.md) and [DALL-E Mini model card](https://huggingface.co/dalle-mini/dalle-mini).*