File size: 2,790 Bytes
5e6405f e769093 5e6405f b5ad3a5 331e778 b5ad3a5 c945287 9648ad6 9e5bfab 5e6405f 9e5bfab e769093 f90b634 5e6405f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
---
license: mit
base_model: runwayml/stable-diffusion-v1-5
tags:
- stable-diffusion
- diffusion
- distillation
- flow-matching
- geometric-deep-learning
- research
library_name: diffusers
pipeline_tag: text-to-image
---
# The finetune process for dreambooth
I'll be gathering SD15 latents en-masse using massive batches of sd15 images for common classes.
This time I'll be directly sampling from laion flavors, roughly 500,000 or so 512x512 3 channel images are required to exist as latents.
The synthetic caption system is biased already so that CAN be used, but the biases don't necessarily line up with SD15 so they must be handled in a lesser extent.
Essentially only gap fillers for commonly used tokens that aren't being filled by the common laion flavors will be targeted and synthesized.
Once the balanced dataset is created, then and only then, can we cook.
The dataset will be available to all and the code will be in the repo for replication.
Most likely it will include it's prompt because running clip_l can have a really high batch size so I can feed CLIP_L a ton of prompts with the correct seeds to match and be fine
This will additionally allow me to shuffle tokens for better sd15 generaliation, but I don't know if I'll enable that. I kind of need it be sd15 before I start jiggering with it's insides.
I'll try to get the latent system synthesizing and preparing latents by the end of the day, hopefully they will be done in the next couple days or perhaps rapidly. In any case, the plan stands.
Teacher's latents go into the student for noise learning. Student's output image is compared to the teacher's. Rinse and repeat. I'll create a new trainer colab without david since he's not going to be required for this one.
Additionally, the subsystems will be based directly on known working systems with solid and concise objectives that make sense to the observer.
Lets do this so it works this time.
# I've decided to name this model
* SD15 Lune - the twin sister of SD15 Sol
# Epoch late 20s are showing detail
# Trainer 2 with correct shift
This is the second train currently running alongside the first. It's using the same trainer currently established.
# Epoch 17 samples
This one is starting over with the new trainer, lets see if it fries early or converges properly. |