Promote the condition_proj-weighted matrices to the root, move the unweighted ones to obsolete/
Browse files- README.md +18 -12
- h3_control_identity.safetensors +3 -0
- h3_control_zero.safetensors +3 -0
- h3_qwen3vl_4b_CONDPROJ_tap24.safetensors +3 -0
- h3_qwen3vl_8b_CONDPROJ_tap24.safetensors +3 -0
- obsolete/h3_qwen3vl_4b_int8convrot_tap24.safetensors +3 -0
- obsolete/h3_qwen3vl_4b_tap24.safetensors +3 -0
- obsolete/h3_qwen3vl_4b_tap24_200prompts.safetensors +3 -0
- obsolete/h3_qwen3vl_8b_tap24.safetensors +3 -0
README.md
CHANGED
|
@@ -23,7 +23,13 @@ library_name: comfyui
|
|
| 23 |
These files are useless on their own. They require the custom node:
|
| 24 |
**[github.com/nicolab28/ComfyUI-ClipProj](https://github.com/nicolab28/ComfyUI-ClipProj)**
|
| 25 |
|
| 26 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
**The first release was missing the attention-sink vector, and short prompts suffered for it.** The first token of a sequence is an attention sink: its direction is constant from one prompt to the next (cosine 1.0000 measured over 1966 prompts) and it carries nothing from the text, yet its norm reaches 16 500 against 291 for a text token. Calibration excluded it, rightly — its extreme values would wreck the statistics — but the node projected it anyway, through a matrix that had never seen one, producing an arbitrary vector of enormous norm. Invisible on a 200-token prompt where it is 0.5 % of the positions, ruinous on a 7-token one where it is 14 %.
|
| 29 |
|
|
@@ -49,10 +55,10 @@ Put them in `ComfyUI/models/clip_projections/`.
|
|
| 49 |
|
| 50 |
| File | Use it for | Encoder | Corpus | Test cosine |
|
| 51 |
|---|---|---|---|---|
|
| 52 |
-
| `h3_qwen3vl_4b_CONDPROJ_tap24.
|
| 53 |
-
| `h3_qwen3vl_8b_CONDPROJ_tap24.
|
| 54 |
-
| `h3_control_identity.
|
| 55 |
-
| `h3_control_zero.
|
| 56 |
|
| 57 |
\* measured in a different space from the rows below — **the two sets of numbers cannot be compared**. See the next section.
|
| 58 |
|
|
@@ -61,10 +67,10 @@ Put them in `ComfyUI/models/clip_projections/`.
|
|
| 61 |
|
| 62 |
| File | Encoder | Corpus | Test cosine |
|
| 63 |
|---|---|---|---|
|
| 64 |
-
| `obsolete/h3_qwen3vl_4b_tap24.
|
| 65 |
-
| `obsolete/h3_qwen3vl_4b_tap24_200prompts.
|
| 66 |
-
| `obsolete/h3_qwen3vl_4b_int8convrot_tap24.
|
| 67 |
-
| `obsolete/h3_qwen3vl_8b_tap24.
|
| 68 |
|
| 69 |
These were the v0.1.1 release. They work, and the 2 000-prompt one is calibrated on eight times more data than anything at the root — but they minimise the wrong error, and the root matrices are the ones that generation testing actually ran on. Keep them if you want to A/B the weighting yourself.
|
| 70 |
|
|
@@ -105,9 +111,9 @@ The two control matrices exist to prove the learned matrix is doing the work rat
|
|
| 105 |
|
| 106 |
| Matrix | Output for *"a red ball on a wood table"* |
|
| 107 |
|---|---|
|
| 108 |
-
| `h3_control_zero.
|
| 109 |
-
| `h3_control_identity.
|
| 110 |
-
| `h3_qwen3vl_4b_tap24.
|
| 111 |
|
| 112 |
`‖W_identity‖ = 50.6` against `‖W_learned‖ = 52.4` — near-identical energy, so the difference is structural, not a matter of scale.
|
| 113 |
|
|
|
|
| 23 |
These files are useless on their own. They require the custom node:
|
| 24 |
**[github.com/nicolab28/ComfyUI-ClipProj](https://github.com/nicolab28/ComfyUI-ClipProj)**
|
| 25 |
|
| 26 |
+
## Now in safetensors
|
| 27 |
+
|
| 28 |
+
Every matrix is a `.safetensors` file, and the `.pt` versions have been removed. A `.pt` goes through pickle, which can execute arbitrary code the moment it is opened — an absurd risk for files that hold nothing but tensors and a handful of scalars. The scalars now live in the safetensors header and the node converts them back on load.
|
| 29 |
+
|
| 30 |
+
Update the node to **v0.1.2** to read them. *Requested by @lucas-ai26, who was right.*
|
| 31 |
+
|
| 32 |
+
## Re-download if you got the first release
|
| 33 |
|
| 34 |
**The first release was missing the attention-sink vector, and short prompts suffered for it.** The first token of a sequence is an attention sink: its direction is constant from one prompt to the next (cosine 1.0000 measured over 1966 prompts) and it carries nothing from the text, yet its norm reaches 16 500 against 291 for a text token. Calibration excluded it, rightly — its extreme values would wreck the statistics — but the node projected it anyway, through a matrix that had never seen one, producing an arbitrary vector of enormous norm. Invisible on a 200-token prompt where it is 0.5 % of the positions, ruinous on a 7-token one where it is 14 %.
|
| 35 |
|
|
|
|
| 55 |
|
| 56 |
| File | Use it for | Encoder | Corpus | Test cosine |
|
| 57 |
|---|---|---|---|---|
|
| 58 |
+
| `h3_qwen3vl_4b_CONDPROJ_tap24.safetensors` | **text prompts, the default** | any Qwen3-VL-4B | 240 prompts | 0.845 * |
|
| 59 |
+
| `h3_qwen3vl_8b_CONDPROJ_tap24.safetensors` | **ref2va and image references** | any Qwen3-VL-8B | 203 prompts | 0.860 * |
|
| 60 |
+
| `h3_control_identity.safetensors` | control — run it once | — | — | — |
|
| 61 |
+
| `h3_control_zero.safetensors` | control — run it once | — | — | — |
|
| 62 |
|
| 63 |
\* measured in a different space from the rows below — **the two sets of numbers cannot be compared**. See the next section.
|
| 64 |
|
|
|
|
| 67 |
|
| 68 |
| File | Encoder | Corpus | Test cosine |
|
| 69 |
|---|---|---|---|
|
| 70 |
+
| `obsolete/h3_qwen3vl_4b_tap24.safetensors` | `qwen3vl_4b_bf16` | 2 000 prompts | 0.712 |
|
| 71 |
+
| `obsolete/h3_qwen3vl_4b_tap24_200prompts.safetensors` | `qwen3vl_4b_bf16` | 200 prompts | 0.699 |
|
| 72 |
+
| `obsolete/h3_qwen3vl_4b_int8convrot_tap24.safetensors` | `qwen3vl_4b_int8_convrot` | 240 prompts | 0.697 |
|
| 73 |
+
| `obsolete/h3_qwen3vl_8b_tap24.safetensors` | `qwen3vl_8b_nvfp4` | 203 prompts | 0.731 |
|
| 74 |
|
| 75 |
These were the v0.1.1 release. They work, and the 2 000-prompt one is calibrated on eight times more data than anything at the root — but they minimise the wrong error, and the root matrices are the ones that generation testing actually ran on. Keep them if you want to A/B the weighting yourself.
|
| 76 |
|
|
|
|
| 111 |
|
| 112 |
| Matrix | Output for *"a red ball on a wood table"* |
|
| 113 |
|---|---|
|
| 114 |
+
| `h3_control_zero.safetensors` | a countryside landscape — the prompt is entirely ignored |
|
| 115 |
+
| `h3_control_identity.safetensors` | a golden object in flames — unusable |
|
| 116 |
+
| `h3_qwen3vl_4b_tap24.safetensors` | the red ball on a wood table |
|
| 117 |
|
| 118 |
`‖W_identity‖ = 50.6` against `‖W_learned‖ = 52.4` — near-identical energy, so the difference is structural, not a matter of scale.
|
| 119 |
|
h3_control_identity.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6bd90824280210c53a5a2de4ddfb6c61a6f921ea3ab5c45fbb55e977629947b9
|
| 3 |
+
size 52511408
|
h3_control_zero.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:73842bed5a64979e639cff06e0e9fab00e124f908b8a19b85b83130ffa54e3cb
|
| 3 |
+
size 52511400
|
h3_qwen3vl_4b_CONDPROJ_tap24.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f3cf71cc9ebdcf2b092039eb7243a91b6739c782a497622c20869ed81217948d
|
| 3 |
+
size 52511512
|
h3_qwen3vl_8b_CONDPROJ_tap24.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:add65889e33549dd613b5e05eaa66ad73b10331861c6158cf3970f17fea6b273
|
| 3 |
+
size 83981080
|
obsolete/h3_qwen3vl_4b_int8convrot_tap24.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:21080e95bbe2bee7211fe853935646b0d18c69154fd6cf96263801866ba16b39
|
| 3 |
+
size 52511488
|
obsolete/h3_qwen3vl_4b_tap24.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f0f7ecaf4f431f62055888cbd214c607bf4215feb002ef4aa43a3ba6205ac325
|
| 3 |
+
size 52511480
|
obsolete/h3_qwen3vl_4b_tap24_200prompts.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f73417573fc34474058d0f1f15e8903970898682d9308e93342aa93105b6966f
|
| 3 |
+
size 52511320
|
obsolete/h3_qwen3vl_8b_tap24.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c589f977406ec2587809e7846193db769224b8bed0a2df62334351c01322e03a
|
| 3 |
+
size 83981048
|