File size: 9,510 Bytes
edc70cd a2bbbfa 0b741fd a2bbbfa 4a4bdc2 0b741fd f136189 16bd206 a2bbbfa 0b741fd a2bbbfa 0b741fd a2bbbfa 0b741fd a2bbbfa 0b741fd a2bbbfa 0b741fd a2bbbfa 0b741fd 5e2cbee 0b741fd 5e2cbee 0b741fd 5e2cbee 0b741fd 611986c 0b741fd 5e2cbee 0b741fd 5e2cbee a9867b7 0b741fd a2bbbfa a9867b7 d25f7f8 f66675c a9867b7 0b741fd d8ab29c aa55c21 d8ab29c a2bbbfa 0b741fd a2bbbfa 0b741fd c57e82e 0b741fd c57e82e 0b741fd 9af3626 0b741fd 9af3626 5e3d330 c7f9d99 5e3d330 3173176 5e3d330 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 |
<table style="border-collapse: collapse; border: 2px solid #ddd; width: 100%;">
<tr>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://discord.com/servers/software-engineering-courses-secourses-772774097734074388"><img src="https://img.shields.io/discord/772774097734074388?label=Discord&logo=discord" alt="Discord"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://hits.seeyoufarm.com"><img src="https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fhuggingface.co%2FMonsterMMORPG%2FModel_Training_Experiments_As_A_Baseline&count_bg=%2379C83D&title_bg=%239E0F0F&icon=apachespark.svg&icon_color=%23E7E7E7&title=views&edge_flat=false" alt="Hits"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://www.patreon.com/SECourses"><img src="https://img.shields.io/badge/Patreon-Support%20Me-F2EB0E?style=for-the-badge&logo=patreon" alt="Patreon"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://www.buymeacoffee.com/DrFurkan"><img src="https://img.shields.io/badge/Buy%20Me%20a%20Coffee-ffdd00?style=for-the-badge&logo=buy-me-a-coffee&logoColor=black" alt="BuyMeACoffee"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://medium.com/@furkangozukara"><img src="https://img.shields.io/badge/Medium-Follow%20Me-800080?style=for-the-badge&logo=medium&logoColor=white" alt="Furkan Gözükara Medium"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://civitai.com/user/SECourses/articles"><img src="https://img.shields.io/static/v1?style=for-the-badge&message=Articles&color=4574E0&logo=Codio&logoColor=FFFFFF&label=CivitAI" alt="Codio"></a></td>
</tr>
<tr>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://www.deviantart.com/monstermmorpg"><img src="https://img.shields.io/badge/DeviantArt-Follow%20Me-990000?style=for-the-badge&logo=deviantart&logoColor=white" alt="Furkan Gözükara DeviantArt"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://www.youtube.com/SECourses"><img src="https://img.shields.io/badge/YouTube-SECourses-C50C0C?style=for-the-badge&logo=youtube" alt="YouTube Channel"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://www.linkedin.com/in/furkangozukara/"><img src="https://img.shields.io/badge/LinkedIn-Follow%20Me-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" alt="Furkan Gözükara LinkedIn"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://www.udemy.com/course/stable-diffusion-dreambooth-lora-zero-to-hero/?referralCode=E327407C9BDF0CEA8156"><img src="https://img.shields.io/static/v1?style=for-the-badge&message=Stable%20Diffusion%20Course&color=A435F0&logo=Udemy&logoColor=FFFFFF&label=Udemy" alt="Udemy"></a></td>
<td style="border: 1px solid #ddd; padding: 3px; text-align: center; vertical-align: middle;"><a href="https://twitter.com/GozukaraFurkan"><img src="https://img.shields.io/badge/Twitter-Follow%20Me-1DA1F2?style=for-the-badge&logo=twitter&logoColor=white" alt="Twitter Follow Furkan Gözükara"></a></td>
</tr>
</table>
# Model Training Experiments and Full Guide and Tutorial: Fine-Tuning vs LoRA Comparison
This repository contains experimental results comparing Fine-Tuning/DreamBooth and LoRA training approaches.
I am sharing how I trained this model with full details and even the dataset: please read entire post very carefully.
This model is purely trained for educational and research purposes only for SFW and ethical image generation.
The workflow and the config used in this tutorial can be used to train clothing, items, animals, pets, objects, styles, simply anything.
The uploaded images have SwarmUI metadata and can be re-generated exactly. For generations FP16 model used but FP8 should yield almost same quality. Don't forget to have used yolo face masking model in prompts.
## How To Use
Download model into diffusion_models of the SwarmUI. Then you need to use Clip-L and T5-XXL models as well. I recommend T5-XXL FP16 or Scaled FP8 version.
A newest fully public tutorial here for how to use : https://youtu.be/-zOKhoO9a5s
## Additional Resources
- Installers and Config Files : https://www.patreon.com/posts/112099700
- FLUX Fine-Tuning / DreamBooth Zero-to-Hero Tutorial : https://youtu.be/FvpWy1x5etM
- FLUX LoRA Training Zero-to-Hero Tutorial : https://youtu.be/nySGu12Y05k)
- Complete Dataset, Training Config Json Files and Testing Prompts : https://www.patreon.com/posts/114972274
- Click below link to download all trained LoRA and Fine-Tuning / DreamBooth checkpoints for free
- https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/tree/main
## Environment Setup
- Kohya GUI Version: `021c6f5ae3055320a56967284e759620c349aa56`
- Torch: 2.5.1
- xFormers: 0.0.28.post3
## Dataset Information
- Resolution: 1024x1024
- Dataset Size: 28 images
- Captions: "ohwx man" (nothing else)
- Activation Token/Trigger Word: "ohwx man"
## Fine-Tuning / DreamBooth Experiment
### Configuration
- Config File: `48GB_GPU_28200MB_6.4_second_it_Tier_1.json`
- Training: Up to 200 epochs with consistent config
- Optimal Result: Epoch 170 (subjective assessment)
### Results
- [Realism Test Part 1](https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/blob/main/Dwayne_Fine_Tune_Realism_Test_Part1.jpg)
- [Realism Test Part 2](https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/blob/main/Dwayne_Fine_Tune_Realism_Test_Part2.jpg)
## LoRA Experiment
### Configuration
- Config File: `Rank_1_29500MB_8_85_Second_IT.json`
- Training: Up to 200 epochs
- Optimal Result: Epoch 160 (subjective assessment)
### Results
- [LoRA Realism Test Part 1](https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/blob/main/Dwayne_LoRA_Realism_Test_Part1.jpg)
- [LoRA Realism Test Part 2](https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/blob/main/Dwayne_LoRA_Realism_Test_Part2.jpg)
## Comparison Results
### LoRA Epochs Comparison
- [LoRA 90 vs 160 vs Fine-Tuning 170 Comparison](https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/blob/main/LoRA_90_Epoch_vs_LoRA_160_Epoch_vs_Fine_Tuning_170_Epoch.jpg)
### Precision Testing
Compared different precision formats in LoRA training:
- FP8 vs FP16 vs FP32 LoRA configurations
- [View Precision Comparison Grid](https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/blob/main/LoRA_Precision_FP8_vs_FP16_vs_FP32_Grid.jpg)
### Model Variant Analysis
Tested various model variants with LoRA (FP32 Version):
- FP8 FLUX DEV Base
- FP8 Scaled
- GGUF 8
- FLUX DEV
- [View Model Variants Comparison Grid](https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline/blob/main/Model_Variants_Tests_Grid.jpg)
- Works best with FP16 DEV base model, then GGUF 8 base model and then FP8 raw base model and FP8 scaled model sometimes works better sometimes worse
### Key Observations
- LoRA demonstrates excellent realism but shows more obvious overfitting when generating stylized images.
- Fine-Tuning / DreamBooth is better than LoRA as expected.
- FP8 almost yields perfect quality as FP32 with LoRA
- I have used Kohya GUI to convert FP32 saved LoRAs into FP16 and FP8
- Here full public article : https://www.patreon.com/posts/115376830
## Model Naming Convention
### Fine-Tuning Models
- `Dwayne_Johnson_FLUX_Fine_Tuning-000010.safetensors`
- 10 epochs
- 280 steps (28 images × 10 epochs)
- Batch size: 1
- Resolution: 1024x1024
- `Dwayne_Johnson_FLUX_Fine_Tuning-000020.safetensors`
- 20 epochs
- 560 steps (28 images × 20 epochs)
- Batch size: 1
- Resolution: 1024x1024
### LoRA Models
- `Dwayne_Johnson_FLUX_LoRA-000010.safetensors`
- 10 epochs
- 280 steps (28 images × 10 epochs)
- Batch size: 1
- Resolution: 1024x1024
- `Dwayne_Johnson_FLUX_LoRA-000020.safetensors`
- 20 epochs
- 560 steps (28 images × 20 epochs)
- Batch size: 1
- Resolution: 1024x1024
## Some Example Images - They have MetaData on CivitAI
CivitAI : https://civitai.com/models/911087
|