Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,10 @@ tags:
|
|
| 9 |
|
| 10 |
This is just a small demonstration to free me from constant arguments with 'experts' telling me I can't do what I've been doing, even though I've been doing it. This is a cross-architeture fusion/graft/merge/mix whatever you want to call it. I'm also essentially applying everything used here while working on my H3 model and was doing it on my LTX 2.3 models before realizing my magnitude attn blend method had bigger and wider reaching applications.
|
| 11 |
|
| 12 |
-
This
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
python graft_krea_to_zimage.py --krea-donor "...10Krea-turbo.safetensors" (just krea2 turbo with the TE defeaters merged in) --zimage-base "...z_image_turbo_bf16.safetensors" --output "krea2z_test10.safetensors" --target main --blocks 0:30 --strength 1.0 --kv --kv-strength 0.25 --mode linear-mag --orthogonal --remap shift --donor-shift 0 --out-proj --out-proj-strength 1.0 --mlp --mlp-strength 0.2 --include-gate --hidden-reduce svd --source-head-selection all48 --target-head-selection all --device cuda
|
| 15 |
|
|
|
|
| 9 |
|
| 10 |
This is just a small demonstration to free me from constant arguments with 'experts' telling me I can't do what I've been doing, even though I've been doing it. This is a cross-architeture fusion/graft/merge/mix whatever you want to call it. I'm also essentially applying everything used here while working on my H3 model and was doing it on my LTX 2.3 models before realizing my magnitude attn blend method had bigger and wider reaching applications.
|
| 11 |
|
| 12 |
+
This can be applied to ANY diffusion transformer model, image, video, or multimodal. And mixed and matched between any and all of them. It won't work on older u-net models or LLM's. You'll need an AI assistant or an ML degree because it's too complex to fully decipher structure targets, match layers and components, code, convert and approximate coverage, and it would just take too long to iterate on manually. It's only worthwhile when you can figure it, test, mix, and output something usable in one sitting. So far the effects are mild but have gotten stronger as I've iterated on strategy. But definitely strong enough considering it's fine tune level change without training anything and is done in seconds once set-up is complete.
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
This simple one-sitting demo graft uses Krea2 turbo as an attention block donor that is blended on to Zimage base turbo's blocks. This is a best case cross-model attn fusion; Krea2 is a better model and Zimage is distilled and untrainable. This method of attn component grafting injects below reinforcement and distillation in a way that training can not. They both have 30 blocks and Krea2's block heads fully cover Zimage's. I ran through a bunch of mixing configs until I felt like the strength was basically at "I told you so." The script used is also uploaded, commandline used was:
|
| 16 |
|
| 17 |
python graft_krea_to_zimage.py --krea-donor "...10Krea-turbo.safetensors" (just krea2 turbo with the TE defeaters merged in) --zimage-base "...z_image_turbo_bf16.safetensors" --output "krea2z_test10.safetensors" --target main --blocks 0:30 --strength 1.0 --kv --kv-strength 0.25 --mode linear-mag --orthogonal --remap shift --donor-shift 0 --out-proj --out-proj-strength 1.0 --mlp --mlp-strength 0.2 --include-gate --hidden-reduce svd --source-head-selection all48 --target-head-selection all --device cuda
|
| 18 |
|