ermu2001 commited on
Commit
d4dd95e
·
1 Parent(s): 4be5b1a

Delete MODELS/sdxl-vae-fp16-fix

Browse files
MODELS/sdxl-vae-fp16-fix/.gitattributes DELETED
@@ -1,38 +0,0 @@
1
- *.7z filter=lfs diff=lfs merge=lfs -text
2
- *.arrow filter=lfs diff=lfs merge=lfs -text
3
- *.bin filter=lfs diff=lfs merge=lfs -text
4
- *.bz2 filter=lfs diff=lfs merge=lfs -text
5
- *.ckpt filter=lfs diff=lfs merge=lfs -text
6
- *.ftz filter=lfs diff=lfs merge=lfs -text
7
- *.gz filter=lfs diff=lfs merge=lfs -text
8
- *.h5 filter=lfs diff=lfs merge=lfs -text
9
- *.joblib filter=lfs diff=lfs merge=lfs -text
10
- *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
- *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
- *.model filter=lfs diff=lfs merge=lfs -text
13
- *.msgpack filter=lfs diff=lfs merge=lfs -text
14
- *.npy filter=lfs diff=lfs merge=lfs -text
15
- *.npz filter=lfs diff=lfs merge=lfs -text
16
- *.onnx filter=lfs diff=lfs merge=lfs -text
17
- *.ot filter=lfs diff=lfs merge=lfs -text
18
- *.parquet filter=lfs diff=lfs merge=lfs -text
19
- *.pb filter=lfs diff=lfs merge=lfs -text
20
- *.pickle filter=lfs diff=lfs merge=lfs -text
21
- *.pkl filter=lfs diff=lfs merge=lfs -text
22
- *.pt filter=lfs diff=lfs merge=lfs -text
23
- *.pth filter=lfs diff=lfs merge=lfs -text
24
- *.rar filter=lfs diff=lfs merge=lfs -text
25
- *.safetensors filter=lfs diff=lfs merge=lfs -text
26
- saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
- *.tar.* filter=lfs diff=lfs merge=lfs -text
28
- *.tar filter=lfs diff=lfs merge=lfs -text
29
- *.tflite filter=lfs diff=lfs merge=lfs -text
30
- *.tgz filter=lfs diff=lfs merge=lfs -text
31
- *.wasm filter=lfs diff=lfs merge=lfs -text
32
- *.xz filter=lfs diff=lfs merge=lfs -text
33
- *.zip filter=lfs diff=lfs merge=lfs -text
34
- *.zst filter=lfs diff=lfs merge=lfs -text
35
- *tfevents* filter=lfs diff=lfs merge=lfs -text
36
- images/fix-fp16.png filter=lfs diff=lfs merge=lfs -text
37
- images/fix-fp32.png filter=lfs diff=lfs merge=lfs -text
38
- images/orig-fp32.png filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
MODELS/sdxl-vae-fp16-fix/README.md DELETED
@@ -1,58 +0,0 @@
1
- ---
2
- license: mit
3
- tags:
4
- - stable-diffusion
5
- - stable-diffusion-diffusers
6
- inference: false
7
- ---
8
- # SDXL-VAE-FP16-Fix
9
-
10
- SDXL-VAE-FP16-Fix is the [SDXL VAE](https://huggingface.co/stabilityai/sdxl-vae)*, but modified to run in fp16 precision without generating NaNs.
11
-
12
- | VAE | Decoding in `float32` / `bfloat16` precision | Decoding in `float16` precision |
13
- | --------------------- | -------------------------------------------- | ------------------------------- |
14
- | SDXL-VAE | ✅ ![](./images/orig-fp32.png) | ⚠️ ![](./images/orig-fp16.png) |
15
- | SDXL-VAE-FP16-Fix | ✅ ![](./images/fix-fp32.png) | ✅ ![](./images/fix-fp16.png) |
16
-
17
- ## 🧨 Diffusers Usage
18
-
19
- Just load this checkpoint via `AutoencoderKL`:
20
-
21
- ```py
22
- import torch
23
- from diffusers import DiffusionPipeline, AutoencoderKL
24
-
25
- vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
26
- pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-0.9", vae=vae, torch_dtype=torch.float16, variant="fp16", use_safetensors=True)
27
- pipe.to("cuda")
28
-
29
- refiner = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-refiner-0.9", vae=vae, torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
30
- refiner.to("cuda")
31
-
32
- n_steps = 40
33
- high_noise_frac = 0.7
34
-
35
- prompt = "A majestic lion jumping from a big stone at night"
36
-
37
- image = pipe(prompt=prompt, num_inference_steps=n_steps, denoising_end=high_noise_frac, output_type="latent").images
38
- image = refiner(prompt=prompt, num_inference_steps=n_steps, denoising_start=high_noise_frac, image=image).images[0]
39
- image
40
- ```
41
-
42
- ![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/lion_refined.png)
43
-
44
- ## Details
45
-
46
- SDXL-VAE generates NaNs in fp16 because the internal activation values are too big:
47
- ![](./images/activation-magnitudes.jpg)
48
-
49
- SDXL-VAE-FP16-Fix was created by finetuning the SDXL-VAE to:
50
- 1. keep the final output the same, but
51
- 2. make the internal activation values smaller, by
52
- 3. scaling down weights and biases within the network
53
-
54
- There are slight discrepancies between the output of SDXL-VAE-FP16-Fix and SDXL-VAE, but the decoded images should be [close enough for most purposes](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7#64c5c0f8e2e5c94bd04eaa80).
55
-
56
- ---
57
-
58
- \* `sdxl-vae-fp16-fix` is specifically based on [SDXL-VAE (0.9)](https://huggingface.co/stabilityai/sdxl-vae/discussions/6#64acea3f7ac35b7de0554490), but it works with SDXL 1.0 too
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
MODELS/sdxl-vae-fp16-fix/config.json DELETED
@@ -1,32 +0,0 @@
1
- {
2
- "_class_name": "AutoencoderKL",
3
- "_diffusers_version": "0.18.0.dev0",
4
- "_name_or_path": ".",
5
- "act_fn": "silu",
6
- "block_out_channels": [
7
- 128,
8
- 256,
9
- 512,
10
- 512
11
- ],
12
- "down_block_types": [
13
- "DownEncoderBlock2D",
14
- "DownEncoderBlock2D",
15
- "DownEncoderBlock2D",
16
- "DownEncoderBlock2D"
17
- ],
18
- "in_channels": 3,
19
- "latent_channels": 4,
20
- "layers_per_block": 2,
21
- "norm_num_groups": 32,
22
- "out_channels": 3,
23
- "sample_size": 512,
24
- "scaling_factor": 0.13025,
25
- "up_block_types": [
26
- "UpDecoderBlock2D",
27
- "UpDecoderBlock2D",
28
- "UpDecoderBlock2D",
29
- "UpDecoderBlock2D"
30
- ],
31
- "force_upcast": false
32
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
MODELS/sdxl-vae-fp16-fix/diffusion_pytorch_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:1b909373b28f2137098b0fd9dbc6f97f8410854f31f84ddc9fa04b077b0ace2c
3
- size 334643238
 
 
 
 
MODELS/sdxl-vae-fp16-fix/images/activation-magnitudes.jpg DELETED
Binary file (269 kB)
 
MODELS/sdxl-vae-fp16-fix/images/fix-fp16.png DELETED

Git LFS Details

  • SHA256: 1f2d50273ee5768f54890f16f76de24009339c86675213616f6a4ca49379d60c
  • Pointer size: 132 Bytes
  • Size of remote file: 1.57 MB
MODELS/sdxl-vae-fp16-fix/images/fix-fp32.png DELETED

Git LFS Details

  • SHA256: a3bc509afe985095cafaf8cb3202edf35f74b5885869a1cc3ee331d65f1b6316
  • Pointer size: 132 Bytes
  • Size of remote file: 1.57 MB
MODELS/sdxl-vae-fp16-fix/images/orig-fp16.png DELETED

Git LFS Details

  • SHA256: 79238174f0b3e2441720c46339b8cce2c8be2c19a1507831e726b51b8bbe3b82
  • Pointer size: 129 Bytes
  • Size of remote file: 3.13 kB
MODELS/sdxl-vae-fp16-fix/images/orig-fp32.png DELETED

Git LFS Details

  • SHA256: 1770748dd6618631a2b5ecf469b80f67c917c747c90ad1ef02adc65406927e62
  • Pointer size: 132 Bytes
  • Size of remote file: 1.6 MB
MODELS/sdxl-vae-fp16-fix/sdxl_vae.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:235745af8d86bf4a4c1b5b4f529868b37019a10f7c0b2e79ad0abca3a22bc6e1
3
- size 334641162