Refresh README: link GitHub repo; add L20 directional-hub / RAW=Turbo / refiner-diffuse / rebalance-lever findings; clarify solo probes are diagnostic (attention-mixed slot, not a clean layer)
Browse files
README.md
CHANGED
|
@@ -15,43 +15,57 @@ tags:
|
|
| 15 |
|
| 16 |
# Krea 2 Projector Explorations
|
| 17 |
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
| layer | L2 | L5 | L8 | L11 | L14 | L17 | L20 | L23 | L26 | L29 | L32 | L35 |
|
| 25 |
|-------|----|----|----|----|----|----|----|----|----|----|----|----|
|
| 26 |
| w | -0.05 | -0.16 | +0.37 | +0.50 | +0.71 | +0.39 | +0.40 | **-1.44** | -0.51 | -0.89 | -0.61 | +0.11 |
|
| 27 |
|
| 28 |
-
|
| 29 |
-
"taps".)
|
| 30 |
|
| 31 |
-
##
|
| 32 |
|
| 33 |
-
|
| 34 |
-
-
|
| 35 |
-
hidden-state layer (zeroing the other 11) so you can see what that individual layer contributes to the
|
| 36 |
-
image.
|
| 37 |
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
-
|
|
|
|
|
|
|
| 42 |
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
-
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
Deterministic edits of the model's existing weights — no training.
|
| 52 |
|
| 53 |
## License
|
| 54 |
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
[krea/Krea-2-Raw](https://huggingface.co/krea/Krea-2-Raw)).
|
|
|
|
| 15 |
|
| 16 |
# Krea 2 Projector Explorations
|
| 17 |
|
| 18 |
+
Small, Krea-derived interpretability artifacts for Krea 2's text conditioning — the learned layer-mix
|
| 19 |
+
("multilayer feature aggregation") plus single-layer probes. **Full toolkit, methods, figures, and write-up:
|
| 20 |
+
[github.com/fblissjr/krea-explorations](https://github.com/fblissjr/krea-explorations).**
|
| 21 |
+
|
| 22 |
+
Krea 2's text encoder is a frozen Qwen3-VL-4B; the DiT takes **12 selected encoder hidden-state layers**
|
| 23 |
+
`[2,5,8,11,14,17,20,23,26,29,32,35]` (`select_layers`), combines them with cross-layer attention, then a
|
| 24 |
+
learned `Linear(12 → 1)` projector (`txtfusion.projector`). That matrix is the model's own per-layer
|
| 25 |
+
weighting — **identical in Raw and Turbo** (cosine 1.0):
|
| 26 |
|
| 27 |
| layer | L2 | L5 | L8 | L11 | L14 | L17 | L20 | L23 | L26 | L29 | L32 | L35 |
|
| 28 |
|-------|----|----|----|----|----|----|----|----|----|----|----|----|
|
| 29 |
| w | -0.05 | -0.16 | +0.37 | +0.50 | +0.71 | +0.39 | +0.40 | **-1.44** | -0.51 | -0.89 | -0.61 | +0.11 |
|
| 30 |
|
| 31 |
+
It combines **contrastively** ("mid plus, deep minus"), not as an average.
|
|
|
|
| 32 |
|
| 33 |
+
## What we measured
|
| 34 |
|
| 35 |
+
These are characterizations of an open model's learned behavior (not architecture — the architecture is
|
| 36 |
+
public); most are low-effort to reproduce. Full method + confidence levels in the GitHub repo.
|
|
|
|
|
|
|
| 37 |
|
| 38 |
+
- **L20 is a learned *directional* attention hub.** In the cross-layer attention, ~91–95% of content tokens
|
| 39 |
+
route to layer 20 — content-driven (not a padding artifact) and a *directional* effect (not a magnitude
|
| 40 |
+
sink). Holds across 5 prompts and on **both Raw and Turbo**. The token-side "refiner" blocks, by contrast,
|
| 41 |
+
are diffuse (no hub).
|
| 42 |
+
- **The projector-rebalance lever is a detail/intensity knob, not an attribute gate.** Benign attributes
|
| 43 |
+
(expression, "wet", blush) come through the aggregation and render with or without rebalancing; boosting
|
| 44 |
+
the deep layers mainly shifts detail / contrast / intensity — consistent with the deep layers carrying
|
| 45 |
+
fine detail.
|
| 46 |
|
| 47 |
+
Per-layer reweighting of Krea 2's conditioning was introduced by
|
| 48 |
+
[nova452/ComfyUI-ConditioningKrea2Rebalance](https://github.com/nova452/ComfyUI-ConditioningKrea2Rebalance)
|
| 49 |
+
and refined by [huwhitememes/comfyui-krea2-conditioning](https://github.com/huwhitememes/comfyui-krea2-conditioning).
|
| 50 |
|
| 51 |
+
## Files
|
| 52 |
+
|
| 53 |
+
- `krea2_projector_original_weights.safetensors` — a **reference copy** of the 12 learned projector weights
|
| 54 |
+
above (the `[1,12]` tensor itself). Read-only reference, not a LoRA to apply.
|
| 55 |
+
- `solo/projector_solo_bNN_Lxx.safetensors` — 12 **diagnostic probes**. Each is a projector `.diff` that, at
|
| 56 |
+
strength 1, keeps one of the projector's 12 inputs and zeroes the other 11, so the DiT conditions on a
|
| 57 |
+
single slot — useful to *see* what that slot contributes (deep slots render coherent images, shallow are
|
| 58 |
+
noise, L14 carries text/structure, L35 alone is unusable).
|
| 59 |
|
| 60 |
+
**Important:** the projector's 12 inputs are the **attention-mixed slots** (output of the 2 layerwise blocks),
|
| 61 |
+
not pristine encoder layers — and because the cross-layer attention routes through L20, every slot already
|
| 62 |
+
carries L20 content. So a "solo Lx" isolates the slot *indexed by* layer x, not a clean layer x. These are
|
| 63 |
+
**interpretability probes, not generation LoRAs** (keeping one input by design gives a partial/degraded image).
|
| 64 |
|
| 65 |
+
Each `solo/` file is a `diffusion_model.txtfusion.projector.diff` patch (one `[1,12]` tensor, ~300 bytes),
|
| 66 |
+
loadable via the stock `LoraLoaderModelOnly` — no custom node. (ComfyUI calls the selected layers "taps".)
|
|
|
|
| 67 |
|
| 68 |
## License
|
| 69 |
|
| 70 |
+
These artifacts derive from Krea 2 and are covered by the **Krea 2 Community License** (see the base model
|
| 71 |
+
linked above).
|
|
|