---
license: mit
---

Code for this can be found at
[https://github.com/gmongaras/Stable-Diffusion-3-From-Scratch](https://github.com/gmongaras/Stable-Diffusion-3-From-Scratch)

Data can be found at
[https://huggingface.co/datasets/gmongaras/CC12M_and_Imagenet21K_Recap_Highqual](https://huggingface.co/datasets/gmongaras/CC12M_and_Imagenet21K_Recap_Highqual)

High res checkpoints can be found at [https://huggingface.co/gmongaras/datav3_attempt4_8GPU_SoftFlash_RoPE2dV2_2AccSteps_stage2/blob/main/README.md](https://huggingface.co/gmongaras/datav3_attempt4_8GPU_SoftFlash_RoPE2dV2_2AccSteps_stage2/blob/main/README.md)

Checkpoints from training a model with:
- 8 GPUs
- a batch size of 70 (I think) with 2 accumulation steps
- using flash attention
- RoPEV2 (something I was experimenting with)
- 256x256 max resolution (but any multiple of 16 up to 256x256 works)