Instructions to use stabilityai/sd-x2-latent-upscaler with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use stabilityai/sd-x2-latent-upscaler with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("stabilityai/sd-x2-latent-upscaler", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Why the upscaled image is noisy?
Your upscaler is nice! It's very fast.
My model, Cool Japan Diffusion, is the finetuned model of Stable Diffusion 2.1 .
I generated a image by my model.
However, the image upscaled by your model is noisy.
Why the upscaled image is noisy?
This is because the VAE of my model is difference from your model?
Or this is because the U-Net of my model is difference from your model?
Do I need to finetune the your model?
@alfredplpl I love your space! Just had quite some fun playing with it:)
I've only tested it on 512 x 512, but It should work with your fine-tuned model!
you can swap out the VAE in upscaler, and use the VAE in your model instead to see if it improves:)
Thanks! I will try to swap the VAEs.
Any resources or pointers (or ideally code) on fine tuning the VAE for a custom dataset?
the same issue when upscale images generated by finetuned Stable diffusion Unet. I think this latent upscaler model has a lot of limiitations.



