Update README.md
Browse files
README.md
CHANGED
|
@@ -3,7 +3,7 @@ license: other
|
|
| 3 |
license_name: fair-ai-public-license-1.0-sd
|
| 4 |
license_link: https://freedevproject.org/faipl-1.0-sd/
|
| 5 |
base_model:
|
| 6 |
-
- CabalResearch/NoobAI-Flux2VAE-RectifiedFlow
|
| 7 |
library_name: diffusers
|
| 8 |
---
|
| 9 |
|
|
@@ -12,16 +12,34 @@ library_name: diffusers
|
|
| 12 |
|
| 13 |
## Model Description
|
| 14 |
|
| 15 |
-
Mugen is a continuation of
|
| 16 |
|
| 17 |
-
It has been trained for 7 additional epochs.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
|
| 20 |
- **Developed by:** Cabal Research (Bluvoll, Anzhc)
|
| 21 |
- **Funded by:** Community
|
| 22 |
- **License:** [fair-ai-public-license-1.0-sd](https://freedevproject.org/faipl-1.0-sd/)
|
| 23 |
-
- **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
## Bias and Limitations
|
| 27 |
|
|
@@ -29,7 +47,6 @@ General data biases from Danbooru might apply.
|
|
| 29 |
|
| 30 |
Flux 2 VAE seem to have brown bias overall, which can be alleviated by adding `sepia` or `brown theme` to negative.
|
| 31 |
|
| 32 |
-
|
| 33 |
## Model Output Examples
|
| 34 |
|
| 35 |
|
|
@@ -41,13 +58,14 @@ Flux 2 VAE seem to have brown bias overall, which can be alleviated by adding `s
|
|
| 41 |
|
| 42 |
#### Comfy
|
| 43 |
|
| 44 |
-
 as is.
|
|
@@ -57,12 +75,15 @@ Same as your normal inference, but with addition of SD3 sampling node, as this m
|
|
| 57 |
Recommended Parameters:
|
| 58 |
**Sampler**: Euler A, Euler, DPM++ SDE, etc.
|
| 59 |
**Steps**: 20-28
|
| 60 |
-
**CFG**: 4-
|
| 61 |
**Shift**: 8-12
|
| 62 |
**Schedule**: Normal/Simple/SGM Uniform
|
| 63 |
**Positive Quality Tags**: `masterpiece, best quality`
|
| 64 |
**Negative Tags**: `worst quality, normal quality, bad anatomy, sepia`
|
| 65 |
|
|
|
|
|
|
|
|
|
|
| 66 |
|
| 67 |
#### A1111 WebUI
|
| 68 |
|
|
@@ -91,12 +112,15 @@ Flux2VAE does not currently have an appropriate high quality preview method, ple
|
|
| 91 |
Recommended Parameters:
|
| 92 |
**Sampler**: Euler A Comfy RF, Euler A2, Euler, DPM++ SDE Comfy, etc. **ALL VARIANTS MUST BE RF OR COMFY, IF AVAILABLE. In ComfyUI routing is automatic, but not in the case of WebUI.**
|
| 93 |
**Steps**: 20-28
|
| 94 |
-
**CFG**: 4-
|
| 95 |
**Shift**: 8-12
|
| 96 |
**Schedule**: Normal/Simple/SGM Uniform
|
| 97 |
**Positive Quality Tags**: `masterpiece, best quality`
|
| 98 |
**Negative Tags**: `worst quality, normal quality, bad anatomy, sepia`
|
| 99 |
|
|
|
|
|
|
|
|
|
|
| 100 |
**ADETAILER FIX FOR RF**:
|
| 101 |
By default, Adetailer discards Advanced Model Sampling extension, which breaks RF. You need to add AMS to this part of settings:
|
| 102 |
|
|
@@ -120,17 +144,10 @@ _builtin_script = (
|
|
| 120 |
|
| 121 |
Or use my fork of Adetailer - https://github.com/Anzhc/aadetailer-reforge
|
| 122 |
|
| 123 |
-
## Training
|
| 124 |
-
|
| 125 |
-
VAE: [Flux2 VAE](https://huggingface.co/black-forest-labs/FLUX.2-dev/tree/main/vae)
|
| 126 |
-
|
| 127 |
-
**VAE Shift**: 0.0760
|
| 128 |
-
**VAE Scale**: 0.6043
|
| 129 |
-
|
| 130 |
|
| 131 |
### LoRA Training
|
| 132 |
|
| 133 |
-
|
| 134 |
|
| 135 |
### Hardware
|
| 136 |
|
|
|
|
| 3 |
license_name: fair-ai-public-license-1.0-sd
|
| 4 |
license_link: https://freedevproject.org/faipl-1.0-sd/
|
| 5 |
base_model:
|
| 6 |
+
- CabalResearch/NoobAI-Flux2VAE-RectifiedFlow-0.3
|
| 7 |
library_name: diffusers
|
| 8 |
---
|
| 9 |
|
|
|
|
| 12 |
|
| 13 |
## Model Description
|
| 14 |
|
| 15 |
+
Mugen is a continuation of our SDXL to Flux 2 VAE conversion, renamed to signify a substantial divergence from the original NoobAI models.
|
| 16 |
|
| 17 |
+
It has been trained for 7 additional epochs, totalling under **8000$** for a full latent space conversion, while preserving and improving upon model anime knowledge.
|
| 18 |
+
|
| 19 |
+
In particular, we have paid attention to character performance in this iteration, and developed in-house approach for benchmarking their performance, about which you can read below.
|
| 20 |
+
|
| 21 |
+
Overall, model performs particularly well with textures and patterns that were previously simply impossible due to SDXL VAE. We prioritized keeping our training as standard-friendly as possible, so local community can easily train on it like on a new Base Model, which it practically is.
|
| 22 |
|
| 23 |
|
| 24 |
- **Developed by:** Cabal Research (Bluvoll, Anzhc)
|
| 25 |
- **Funded by:** Community
|
| 26 |
- **License:** [fair-ai-public-license-1.0-sd](https://freedevproject.org/faipl-1.0-sd/)
|
| 27 |
+
- **Resumed from:** [NoobAI Flux2 VAE v0.3](https://huggingface.co/CabalResearch/NoobAI-Flux2VAE-RectifiedFlow)
|
| 28 |
+
|
| 29 |
+
### Character Knowledge Benchmark
|
| 30 |
+
|
| 31 |
+

|
| 32 |
+
|
| 33 |
+
|
| 34 |
+

|
| 35 |
+
|
| 36 |
+
This benchmark measures character similarity across 1815 characters in this iteration of it.
|
| 37 |
+
|
| 38 |
+
We utilize reference(non-generated) set of images, and measure character features against ai-generated data - this is the similarity score. Our custom in-house model for character discrimination trained on ~1.2kk images is used.
|
| 39 |
|
| 40 |
+
Due to compute constraint, we selected only single model to compare against - not yet released latest version of [Chenkin](https://huggingface.co/ChenkinNoob/ChenkinNoob-XL-V0.3-BETA) model, which is the most trained SDXL-based anime model currently.
|
| 41 |
+
|
| 42 |
+
Fuure benchmark iterations might include different arches and more models.
|
| 43 |
|
| 44 |
## Bias and Limitations
|
| 45 |
|
|
|
|
| 47 |
|
| 48 |
Flux 2 VAE seem to have brown bias overall, which can be alleviated by adding `sepia` or `brown theme` to negative.
|
| 49 |
|
|
|
|
| 50 |
## Model Output Examples
|
| 51 |
|
| 52 |
|
|
|
|
| 58 |
|
| 59 |
#### Comfy
|
| 60 |
|
| 61 |
+

|
| 62 |
+
|
| 63 |
+
(Workflow is available alongside model in repo)
|
| 64 |
|
|
|
|
| 65 |
We will provide a Node, and hope it will be adapted natively in main repo eventually:
|
| 66 |
**https://github.com/Anzhc/SDXL-Flux2VAE-ComfyUI-Node**
|
| 67 |
|
| 68 |
+
Just install it, and it will patch the model config, no mode changes required.
|
| 69 |
|
| 70 |
|
| 71 |
Apparently works in [SwarmUI](https://github.com/mcmonkeyprojects/SwarmUI) as is.
|
|
|
|
| 75 |
Recommended Parameters:
|
| 76 |
**Sampler**: Euler A, Euler, DPM++ SDE, etc.
|
| 77 |
**Steps**: 20-28
|
| 78 |
+
**CFG**: 4-7
|
| 79 |
**Shift**: 8-12
|
| 80 |
**Schedule**: Normal/Simple/SGM Uniform
|
| 81 |
**Positive Quality Tags**: `masterpiece, best quality`
|
| 82 |
**Negative Tags**: `worst quality, normal quality, bad anatomy, sepia`
|
| 83 |
|
| 84 |
+
**Alternative Extended Negative**: `(worst quality:1.1), normal quality, (bad anatomy:1.1), (blurry:1.1), watermark, sepia, (adversarial noise:1.1), jpeg artifacts`
|
| 85 |
+
(Some of our testers pointed out that they prefer longer negative)
|
| 86 |
+
|
| 87 |
|
| 88 |
#### A1111 WebUI
|
| 89 |
|
|
|
|
| 112 |
Recommended Parameters:
|
| 113 |
**Sampler**: Euler A Comfy RF, Euler A2, Euler, DPM++ SDE Comfy, etc. **ALL VARIANTS MUST BE RF OR COMFY, IF AVAILABLE. In ComfyUI routing is automatic, but not in the case of WebUI.**
|
| 114 |
**Steps**: 20-28
|
| 115 |
+
**CFG**: 4-7(or 7-15, if it appears to be weak/bugged)
|
| 116 |
**Shift**: 8-12
|
| 117 |
**Schedule**: Normal/Simple/SGM Uniform
|
| 118 |
**Positive Quality Tags**: `masterpiece, best quality`
|
| 119 |
**Negative Tags**: `worst quality, normal quality, bad anatomy, sepia`
|
| 120 |
|
| 121 |
+
**Alternative Extended Negative**: `(worst quality:1.1), normal quality, (bad anatomy:1.1), (blurry:1.1), watermark, sepia, (adversarial noise:1.1), jpeg artifacts`
|
| 122 |
+
(Some of our testers pointed out that they prefer longer negative)
|
| 123 |
+
|
| 124 |
**ADETAILER FIX FOR RF**:
|
| 125 |
By default, Adetailer discards Advanced Model Sampling extension, which breaks RF. You need to add AMS to this part of settings:
|
| 126 |
|
|
|
|
| 144 |
|
| 145 |
Or use my fork of Adetailer - https://github.com/Anzhc/aadetailer-reforge
|
| 146 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
|
| 148 |
### LoRA Training
|
| 149 |
|
| 150 |
+
You can directly reference config with all parameters: Download
|
| 151 |
|
| 152 |
### Hardware
|
| 153 |
|