NicoLab28 commited on
Commit
7d788f7
·
verified ·
1 Parent(s): 42dafd0

Promote the condition_proj-weighted matrices to the root, move the unweighted ones to obsolete/

Browse files
README.md CHANGED
@@ -23,7 +23,13 @@ library_name: comfyui
23
  These files are useless on their own. They require the custom node:
24
  **[github.com/nicolab28/ComfyUI-ClipProj](https://github.com/nicolab28/ComfyUI-ClipProj)**
25
 
26
- ## v0.1.1 — re-download if you got the first release
 
 
 
 
 
 
27
 
28
  **The first release was missing the attention-sink vector, and short prompts suffered for it.** The first token of a sequence is an attention sink: its direction is constant from one prompt to the next (cosine 1.0000 measured over 1966 prompts) and it carries nothing from the text, yet its norm reaches 16 500 against 291 for a text token. Calibration excluded it, rightly — its extreme values would wreck the statistics — but the node projected it anyway, through a matrix that had never seen one, producing an arbitrary vector of enormous norm. Invisible on a 200-token prompt where it is 0.5 % of the positions, ruinous on a 7-token one where it is 14 %.
29
 
@@ -49,10 +55,10 @@ Put them in `ComfyUI/models/clip_projections/`.
49
 
50
  | File | Use it for | Encoder | Corpus | Test cosine |
51
  |---|---|---|---|---|
52
- | `h3_qwen3vl_4b_CONDPROJ_tap24.pt` | **text prompts, the default** | any Qwen3-VL-4B | 240 prompts | 0.845 * |
53
- | `h3_qwen3vl_8b_CONDPROJ_tap24.pt` | **ref2va and image references** | any Qwen3-VL-8B | 203 prompts | 0.860 * |
54
- | `h3_control_identity.pt` | control — run it once | — | — | — |
55
- | `h3_control_zero.pt` | control — run it once | — | — | — |
56
 
57
  \* measured in a different space from the rows below — **the two sets of numbers cannot be compared**. See the next section.
58
 
@@ -61,10 +67,10 @@ Put them in `ComfyUI/models/clip_projections/`.
61
 
62
  | File | Encoder | Corpus | Test cosine |
63
  |---|---|---|---|
64
- | `obsolete/h3_qwen3vl_4b_tap24.pt` | `qwen3vl_4b_bf16` | 2 000 prompts | 0.712 |
65
- | `obsolete/h3_qwen3vl_4b_tap24_200prompts.pt` | `qwen3vl_4b_bf16` | 200 prompts | 0.699 |
66
- | `obsolete/h3_qwen3vl_4b_int8convrot_tap24.pt` | `qwen3vl_4b_int8_convrot` | 240 prompts | 0.697 |
67
- | `obsolete/h3_qwen3vl_8b_tap24.pt` | `qwen3vl_8b_nvfp4` | 203 prompts | 0.731 |
68
 
69
  These were the v0.1.1 release. They work, and the 2 000-prompt one is calibrated on eight times more data than anything at the root — but they minimise the wrong error, and the root matrices are the ones that generation testing actually ran on. Keep them if you want to A/B the weighting yourself.
70
 
@@ -105,9 +111,9 @@ The two control matrices exist to prove the learned matrix is doing the work rat
105
 
106
  | Matrix | Output for *"a red ball on a wood table"* |
107
  |---|---|
108
- | `h3_control_zero.pt` | a countryside landscape — the prompt is entirely ignored |
109
- | `h3_control_identity.pt` | a golden object in flames — unusable |
110
- | `h3_qwen3vl_4b_tap24.pt` | the red ball on a wood table |
111
 
112
  `‖W_identity‖ = 50.6` against `‖W_learned‖ = 52.4` — near-identical energy, so the difference is structural, not a matter of scale.
113
 
 
23
  These files are useless on their own. They require the custom node:
24
  **[github.com/nicolab28/ComfyUI-ClipProj](https://github.com/nicolab28/ComfyUI-ClipProj)**
25
 
26
+ ## Now in safetensors
27
+
28
+ Every matrix is a `.safetensors` file, and the `.pt` versions have been removed. A `.pt` goes through pickle, which can execute arbitrary code the moment it is opened — an absurd risk for files that hold nothing but tensors and a handful of scalars. The scalars now live in the safetensors header and the node converts them back on load.
29
+
30
+ Update the node to **v0.1.2** to read them. *Requested by @lucas-ai26, who was right.*
31
+
32
+ ## Re-download if you got the first release
33
 
34
  **The first release was missing the attention-sink vector, and short prompts suffered for it.** The first token of a sequence is an attention sink: its direction is constant from one prompt to the next (cosine 1.0000 measured over 1966 prompts) and it carries nothing from the text, yet its norm reaches 16 500 against 291 for a text token. Calibration excluded it, rightly — its extreme values would wreck the statistics — but the node projected it anyway, through a matrix that had never seen one, producing an arbitrary vector of enormous norm. Invisible on a 200-token prompt where it is 0.5 % of the positions, ruinous on a 7-token one where it is 14 %.
35
 
 
55
 
56
  | File | Use it for | Encoder | Corpus | Test cosine |
57
  |---|---|---|---|---|
58
+ | `h3_qwen3vl_4b_CONDPROJ_tap24.safetensors` | **text prompts, the default** | any Qwen3-VL-4B | 240 prompts | 0.845 * |
59
+ | `h3_qwen3vl_8b_CONDPROJ_tap24.safetensors` | **ref2va and image references** | any Qwen3-VL-8B | 203 prompts | 0.860 * |
60
+ | `h3_control_identity.safetensors` | control — run it once | — | — | — |
61
+ | `h3_control_zero.safetensors` | control — run it once | — | — | — |
62
 
63
  \* measured in a different space from the rows below — **the two sets of numbers cannot be compared**. See the next section.
64
 
 
67
 
68
  | File | Encoder | Corpus | Test cosine |
69
  |---|---|---|---|
70
+ | `obsolete/h3_qwen3vl_4b_tap24.safetensors` | `qwen3vl_4b_bf16` | 2 000 prompts | 0.712 |
71
+ | `obsolete/h3_qwen3vl_4b_tap24_200prompts.safetensors` | `qwen3vl_4b_bf16` | 200 prompts | 0.699 |
72
+ | `obsolete/h3_qwen3vl_4b_int8convrot_tap24.safetensors` | `qwen3vl_4b_int8_convrot` | 240 prompts | 0.697 |
73
+ | `obsolete/h3_qwen3vl_8b_tap24.safetensors` | `qwen3vl_8b_nvfp4` | 203 prompts | 0.731 |
74
 
75
  These were the v0.1.1 release. They work, and the 2 000-prompt one is calibrated on eight times more data than anything at the root — but they minimise the wrong error, and the root matrices are the ones that generation testing actually ran on. Keep them if you want to A/B the weighting yourself.
76
 
 
111
 
112
  | Matrix | Output for *"a red ball on a wood table"* |
113
  |---|---|
114
+ | `h3_control_zero.safetensors` | a countryside landscape — the prompt is entirely ignored |
115
+ | `h3_control_identity.safetensors` | a golden object in flames — unusable |
116
+ | `h3_qwen3vl_4b_tap24.safetensors` | the red ball on a wood table |
117
 
118
  `‖W_identity‖ = 50.6` against `‖W_learned‖ = 52.4` — near-identical energy, so the difference is structural, not a matter of scale.
119
 
h3_control_identity.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bd90824280210c53a5a2de4ddfb6c61a6f921ea3ab5c45fbb55e977629947b9
3
+ size 52511408
h3_control_zero.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73842bed5a64979e639cff06e0e9fab00e124f908b8a19b85b83130ffa54e3cb
3
+ size 52511400
h3_qwen3vl_4b_CONDPROJ_tap24.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3cf71cc9ebdcf2b092039eb7243a91b6739c782a497622c20869ed81217948d
3
+ size 52511512
h3_qwen3vl_8b_CONDPROJ_tap24.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:add65889e33549dd613b5e05eaa66ad73b10331861c6158cf3970f17fea6b273
3
+ size 83981080
obsolete/h3_qwen3vl_4b_int8convrot_tap24.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:21080e95bbe2bee7211fe853935646b0d18c69154fd6cf96263801866ba16b39
3
+ size 52511488
obsolete/h3_qwen3vl_4b_tap24.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0f7ecaf4f431f62055888cbd214c607bf4215feb002ef4aa43a3ba6205ac325
3
+ size 52511480
obsolete/h3_qwen3vl_4b_tap24_200prompts.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f73417573fc34474058d0f1f15e8903970898682d9308e93342aa93105b6966f
3
+ size 52511320
obsolete/h3_qwen3vl_8b_tap24.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c589f977406ec2587809e7846193db769224b8bed0a2df62334351c01322e03a
3
+ size 83981048