Sandpies Claude Opus 5 commited on
Commit
ddcc824
·
1 Parent(s): a9a905d

0.4.4: fix what two models proved the prompt never said

Browse files

An A/B of qwen and gemma on the same 3-hop brief, each given
prompt_pack/SYSTEM_PROMPT.md and nothing else, produced the same two
faults -- which makes them the prompt's, not the models'.

Both wrote @kitchen in a beat, exactly as rule 10 instructs, and then
invented "tag": "kitchen_plate" for the register, so the run stopped on
an unknown reference. The string _plate appears nowhere in the prompt;
two different model families reached the same convention independently.
The cause was in the file: rule 10's only concrete place tag sits in a
beat, the register example held one ref (a person, no place), and the
two were never shown together. The invariant was stated, in a field
bullet 150 lines later -- and models copy examples, not bullets.
PROMPTING.md, the human guide, has had a three-ref example including
kitchen all along. The machine prompt was trimmed and lost it.

qwen also wrote "name": "@cook_face" into subjects. That parses,
resolves, renders, and is wrong: name is what resolve_tags substitutes
for a subject's tag from hop 2 on, so the tag resolved to itself and a
literal at-sign reached the text encoder on two of three hops. Nothing
caught it -- check_prompts.py would have, but only for the two shipped
workflows.

- prompt: the register example now carries a place tag on both sides of
the round trip, and rule 10 says the two spellings are one string
- refs.py: reject an @tag in name, locked or context rather than
encoding it
- run panel: help text was read from widget.options.tooltip, which this
frontend does not populate, so every field the panel drew had no
tooltip while the native dials kept theirs. tooltipFor() tries the
widget and then the node definition.

Devlog section 22 has the A/B and the one difference no parser can see:
handed a hallway, one model moved rule 4's example refrigerator into it
and the other wrote a hallway light.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015MGjcAV8bDy93qJfmLi9kw

README.md CHANGED
@@ -37,6 +37,19 @@ of a second, separate generation. The pack exists so that you cannot tell which.
37
  > `avcodec_open2`. Both fixed. The Starter now also ships the seam report wired.
38
  > Measurements are in [`docs/DEVLOG.md`](docs/DEVLOG.md) section 21.
39
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  **Writing for it:** [PROMPTING.md](PROMPTING.md) is the authoring guide — the rules that come from what this model actually does, not from taste. [prompt_pack/](prompt_pack/) has a copy-paste prompt that gets a language model to write plans for you.
41
 
42
  Each hop is native **MiniMax H3 Reference-to-Video**. Hops after the first are guided by the **previous hop's sampler AV latent** via `ComfyUI-H3-Motion-Context` when that pack is installed (22 picture frames + 24-frame end-aligned audio). Stock `MiniMaxH3AddGuide` is the fallback when Motion-Context is missing or the previous hop was a pixel cache hit. Voice stays as a reference every hop. Identity stills ride hop 1; later hops use the pin for wardrobe and room unless a ref lists those hops in `shots`. A 5 s hop drops the airlock on a continuous join — validate seams at 8 s or 15 s.
 
37
  > `avcodec_open2`. Both fixed. The Starter now also ships the seam report wired.
38
  > Measurements are in [`docs/DEVLOG.md`](docs/DEVLOG.md) section 21.
39
 
40
+ > **0.4.4 — 2026-08-30.** Two local models were given `prompt_pack/SYSTEM_PROMPT.md`
41
+ > and the same scene, and both made the same two mistakes — so both were the
42
+ > prompt's fault, not theirs. The register example had no place tag in it, so
43
+ > each model invented its own convention (`kitchen_plate`) while leaving
44
+ > `@kitchen` in the beat, and the run stopped on an unknown reference. One model
45
+ > also wrote `"name": "@cook_face"` into `subjects`, which parsed, rendered, and
46
+ > quietly put a literal at-sign in front of the text encoder on every
47
+ > continuation hop. The prompt now shows a place tag on both sides of the round
48
+ > trip and says the two spellings are one string; `refs.py` now rejects an
49
+ > `@tag` in `name`, `locked` or `context` instead of encoding it. The RUN panel
50
+ > also has its tooltips back — it read them from `widget.options`, where this
51
+ > frontend does not keep them. Section 22 of the devlog has the A/B.
52
+
53
  **Writing for it:** [PROMPTING.md](PROMPTING.md) is the authoring guide — the rules that come from what this model actually does, not from taste. [prompt_pack/](prompt_pack/) has a copy-paste prompt that gets a language model to write plans for you.
54
 
55
  Each hop is native **MiniMax H3 Reference-to-Video**. Hops after the first are guided by the **previous hop's sampler AV latent** via `ComfyUI-H3-Motion-Context` when that pack is installed (22 picture frames + 24-frame end-aligned audio). Stock `MiniMaxH3AddGuide` is the fallback when Motion-Context is missing or the previous hop was a pixel cache hit. Voice stays as a reference every hop. Identity stills ride hop 1; later hops use the pin for wardrobe and room unless a ref lists those hops in `shots`. A 5 s hop drops the airlock on a continuous join — validate seams at 8 s or 15 s.
docs/DEVLOG.md CHANGED
@@ -921,3 +921,51 @@ fast-preview button it sounds like. `dry_run` is the fast-preview button.
921
  and started rendering at hop 4.
922
  - **Over-delivery lint**: exercised offline; never fired in ComfyUI because both
923
  shipped plans and the test plan stay clean. Unproven against a real positive.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
921
  and started rendering at hop 4.
922
  - **Over-delivery lint**: exercised offline; never fired in ComfyUI because both
923
  shipped plans and the test plan stay clean. Unproven against a real positive.
924
+
925
+ ## 22. Two models, one prompt, the same two mistakes (2026-08-30)
926
+
927
+ A 3-hop / 10 s brief was written to load six traps into thirty seconds, and
928
+ handed to two local models in LM Studio with `prompt_pack/SYSTEM_PROMPT.md` in
929
+ the system box, temperature 0.4, and nothing else. `EXAMPLE_6_HOP.md` was
930
+ deliberately withheld -- it argues for six hops and would have contaminated a
931
+ 3-hop test. Grading ran every reply through the real parsers rather than by eye.
932
+
933
+ | | qwen | gemma 26b-a4b |
934
+ |---|---|---|
935
+ | FAIL | 4 | 2 |
936
+ | after the shared prompt bug | 2 | 0 |
937
+
938
+ **Both models made the same tag mistake, which makes it the prompt's.** Each
939
+ wrote `@kitchen` in the beat -- correctly, that is rule 10's own example -- and
940
+ then invented `"tag": "kitchen_plate"` for the register. The string `_plate`
941
+ appears nowhere in the prompt; they arrived at the same convention
942
+ independently. The cause was in the file: rule 10's only concrete place tag
943
+ lives in a *beat*, the register example held one ref (`hero_face`, a person),
944
+ and the two were never shown together. The invariant *was* stated, in a field
945
+ bullet 150 lines later -- and models copy examples, not bullets. `PROMPTING.md`,
946
+ the human guide, has had a three-ref example including `kitchen` all along; the
947
+ machine prompt was trimmed and lost it.
948
+
949
+ **The silent one.** qwen wrote `"name": "@cook_face"` into `subjects`. That
950
+ parses, resolves, renders, and is wrong: `name` is what `resolve_tags`
951
+ substitutes for a subject's tag from hop 2 on, so the tag resolved to itself and
952
+ a literal at-sign reached the encoder on two of three hops. Nothing caught it.
953
+ `check_prompts.py` would have, but only for the two shipped workflows.
954
+
955
+ **Where the models actually differed** is invisible to any parser. Both lifted
956
+ rule 2's worked answer verbatim (*"water runs in slow threads down the window
957
+ glass"*). But hop 3 moves to a hallway, and qwen also copied rule 4's example
958
+ *object* -- putting "a single click from the refrigerator" in a corridor, the
959
+ kitchen appliance following her out of the room. gemma copied rule 4's *method*
960
+ and wrote "the low hum of a hallway light". Recitation versus transfer, and only
961
+ one of them survives a change of location.
962
+
963
+ **Both** also left `tail` off hops 1-2, describing the arrival at rest in prose
964
+ instead of directing it -- so the over-delivery lint, which only arms after a
965
+ `settle`, still has not fired on a real positive.
966
+
967
+ Fixed here: the register example carries a place tag on both sides of the round
968
+ trip and says the two spellings are one string; `refs.py` rejects an `@tag` in
969
+ `name`, `locked` or `context`. Not fixed, because it is a brief-writing lesson
970
+ rather than a bug: ending a chain in a location no plate describes earns the
971
+ place-handoff warning, and that was the brief's fault, not either model's.
js/editor/run_panel.js CHANGED
@@ -13,12 +13,14 @@
13
  * is byte-identical to one produced by the stock UI, and anything this panel
14
  * cannot express is still reachable by dropping it from the group list.
15
  *
16
- * Help text is read from `widget.options.tooltip`, i.e. the `tooltip` written in
17
- * INPUT_TYPES. Retyping those sentences here would be a second copy that goes
18
- * stale, for exactly the reason routes.py exists.
 
 
19
  */
20
 
21
- import { el, widgetByName, widgetType, widgetOptions } from "./widget_utils.js";
22
 
23
  const OPEN_PROP = "h3_run_open";
24
 
@@ -257,8 +259,16 @@ export function createRunPanel(node, { onChange, suppressed, hopCount } = {}) {
257
  l.appendChild(el("span", null, labelFor(w)));
258
  l.appendChild(f.input);
259
  // The tooltip is the widget's own, straight from INPUT_TYPES.
260
- const tip = widgetOptions(w).tooltip;
261
- if (tip) l.title = String(tip);
 
 
 
 
 
 
 
 
262
  grid.appendChild(l);
263
  owned.push(name);
264
  readers.push(f.read);
 
13
  * is byte-identical to one produced by the stock UI, and anything this panel
14
  * cannot express is still reachable by dropping it from the group list.
15
  *
16
+ * Help text comes from the `tooltip` written in INPUT_TYPES, via
17
+ * `tooltipFor()`. Retyping those sentences here would be a second copy that
18
+ * goes stale, for exactly the reason routes.py exists. Reading only
19
+ * `widget.options.tooltip` was not enough and left the whole panel with no
20
+ * tooltips at all while the native dials kept theirs -- see tooltipFor().
21
  */
22
 
23
+ import { el, widgetByName, widgetType, widgetOptions, tooltipFor } from "./widget_utils.js";
24
 
25
  const OPEN_PROP = "h3_run_open";
26
 
 
259
  l.appendChild(el("span", null, labelFor(w)));
260
  l.appendChild(f.input);
261
  // The tooltip is the widget's own, straight from INPUT_TYPES.
262
+ // Set on the label, which wraps both the caption and the
263
+ // control, so the hover target is the whole field rather than
264
+ // the six pixels of text. `title` on the input as well because
265
+ // a DOM <select>/<input> does not always inherit the ancestor's
266
+ // tooltip while it has focus.
267
+ const tip = tooltipFor(node, w);
268
+ if (tip) {
269
+ l.title = tip;
270
+ f.input.title = tip;
271
+ }
272
  grid.appendChild(l);
273
  owned.push(name);
274
  readers.push(f.read);
js/editor/widget_utils.js CHANGED
@@ -165,6 +165,30 @@ export function widgetByName(node, name) {
165
  return node.widgets?.find((w) => w.name === name) || null;
166
  }
167
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
168
  /** Hide every widget in `names`, show the rest of `names`' complement set. */
169
  export function setWidgetVisibility(node, hiddenNames) {
170
  const hide = new Set(hiddenNames);
 
165
  return node.widgets?.find((w) => w.name === name) || null;
166
  }
167
 
168
+ /**
169
+ * The `tooltip` a widget was given in INPUT_TYPES, wherever this frontend
170
+ * keeps it.
171
+ *
172
+ * `widgetOptions(w).tooltip` on its own is not enough, and returned nothing
173
+ * for the entire RUN panel: ComfyUI's own tooltip layer does not read the
174
+ * widget object at all, it reads the node DEFINITION. So every native dial
175
+ * showed its help text while every field this editor drew showed none, which
176
+ * reads as "the panel has no tooltips" rather than as a lookup miss.
177
+ *
178
+ * Which of the three places holds the string depends on the frontend version,
179
+ * so all three are tried instead of pinning one. Never retype the sentences
180
+ * here: INPUT_TYPES is the single source, for the same reason routes.py exists.
181
+ */
182
+ export function tooltipFor(node, w) {
183
+ if (!w) return "";
184
+ const direct = w.tooltip || widgetOptions(w).tooltip;
185
+ if (direct) return String(direct);
186
+ const spec = node?.constructor?.nodeData?.input;
187
+ const entry = spec?.required?.[w.name] || spec?.optional?.[w.name];
188
+ const tip = Array.isArray(entry) ? entry[1]?.tooltip : null;
189
+ return tip ? String(tip) : "";
190
+ }
191
+
192
  /** Hide every widget in `names`, show the rest of `names`' complement set. */
193
  export function setWidgetVisibility(node, hiddenNames) {
194
  const hide = new Set(hiddenNames);
prompt_pack/AUTHORING_PROMPT.md CHANGED
@@ -102,6 +102,11 @@ These are not style preferences. They are how this model fails.
102
  write reaches the encoder literally, so formatting becomes noise in the
103
  conditioning.
104
 
 
 
 
 
 
105
  11. **Every object that persists gets an adjective.** A bare noun is
106
  unanchored: each hop encodes "the bowl" from scratch and is free to make it
107
  steel in one hop and porcelain in the next. Name a colour or a material the
@@ -249,7 +254,10 @@ then pinned into the next hop's audio.
249
  "refs": [
250
  {"tag": "hero_face", "file": "face.jpg", "subject": 1,
251
  "retention": "fully_preserved", "shots": [1, 4],
252
- "desc": "head-and-shoulders, even light"}
 
 
 
253
  ],
254
  "subjects": {
255
  "1": {"name": "the cook",
@@ -259,6 +267,17 @@ then pinned into the next hop's audio.
259
  }
260
  ```
261
 
 
 
 
 
 
 
 
 
 
 
 
262
  Valid ref fields: `tag`, `file`, `subject`, `retention`, `desc`, `shots`. (There
263
  is also `slot`, which is derived from list position — never author it.)
264
 
 
102
  write reaches the encoder literally, so formatting becomes noise in the
103
  conditioning.
104
 
105
+ **The tag in the beat and the `tag` in the register are the same string,
106
+ character for character.** Writing `@kitchen` in a beat commits you to
107
+ `"tag": "kitchen"` in `ref_plan` -- not `kitchen_plate`, not `kitchen_ref`.
108
+ Decide the name once and use it in both documents.
109
+
110
  11. **Every object that persists gets an adjective.** A bare noun is
111
  unanchored: each hop encodes "the bowl" from scratch and is free to make it
112
  steel in one hop and porcelain in the next. Name a colour or a material the
 
254
  "refs": [
255
  {"tag": "hero_face", "file": "face.jpg", "subject": 1,
256
  "retention": "fully_preserved", "shots": [1, 4],
257
+ "desc": "head-and-shoulders, even light"},
258
+ {"tag": "kitchen", "file": "kitchen_wide.jpg",
259
+ "retention": "reference", "shots": [1, 2, 3],
260
+ "desc": "wide shot of the counter and window"}
261
  ],
262
  "subjects": {
263
  "1": {"name": "the cook",
 
267
  }
268
  ```
269
 
270
+ The place entry is `"tag": "kitchen"` because the beat says `@kitchen`. **The
271
+ two spellings are one string.** Naming the register entry `kitchen_plate`,
272
+ `kitchen_ref` or `kitchen_bg` while the beat still reads `@kitchen` stops the
273
+ run: the node resolves tags by exact name and has no way to guess that two
274
+ different words meant the same picture.
275
+
276
+ `subjects` holds **plain prose, never tags.** `name` is the phrase that replaces
277
+ a subject's `@tag` from hop 2 on, so `"name": "@hero_face"` would resolve to
278
+ itself and put a literal at-sign in front of the encoder. Write `"the cook"`.
279
+ The same goes for `locked` and `context`.
280
+
281
  Valid ref fields: `tag`, `file`, `subject`, `retention`, `desc`, `shots`. (There
282
  is also `slot`, which is derived from list position — never author it.)
283
 
prompt_pack/SYSTEM_PROMPT.md CHANGED
@@ -89,6 +89,11 @@ These are not style preferences. They are how this model fails.
89
  write reaches the encoder literally, so formatting becomes noise in the
90
  conditioning.
91
 
 
 
 
 
 
92
  11. **Every object that persists gets an adjective.** A bare noun is
93
  unanchored: each hop encodes "the bowl" from scratch and is free to make it
94
  steel in one hop and porcelain in the next. Name a colour or a material the
@@ -236,7 +241,10 @@ then pinned into the next hop's audio.
236
  "refs": [
237
  {"tag": "hero_face", "file": "face.jpg", "subject": 1,
238
  "retention": "fully_preserved", "shots": [1, 4],
239
- "desc": "head-and-shoulders, even light"}
 
 
 
240
  ],
241
  "subjects": {
242
  "1": {"name": "the cook",
@@ -246,6 +254,17 @@ then pinned into the next hop's audio.
246
  }
247
  ```
248
 
 
 
 
 
 
 
 
 
 
 
 
249
  Valid ref fields: `tag`, `file`, `subject`, `retention`, `desc`, `shots`. (There
250
  is also `slot`, which is derived from list position — never author it.)
251
 
 
89
  write reaches the encoder literally, so formatting becomes noise in the
90
  conditioning.
91
 
92
+ **The tag in the beat and the `tag` in the register are the same string,
93
+ character for character.** Writing `@kitchen` in a beat commits you to
94
+ `"tag": "kitchen"` in `ref_plan` -- not `kitchen_plate`, not `kitchen_ref`.
95
+ Decide the name once and use it in both documents.
96
+
97
  11. **Every object that persists gets an adjective.** A bare noun is
98
  unanchored: each hop encodes "the bowl" from scratch and is free to make it
99
  steel in one hop and porcelain in the next. Name a colour or a material the
 
241
  "refs": [
242
  {"tag": "hero_face", "file": "face.jpg", "subject": 1,
243
  "retention": "fully_preserved", "shots": [1, 4],
244
+ "desc": "head-and-shoulders, even light"},
245
+ {"tag": "kitchen", "file": "kitchen_wide.jpg",
246
+ "retention": "reference", "shots": [1, 2, 3],
247
+ "desc": "wide shot of the counter and window"}
248
  ],
249
  "subjects": {
250
  "1": {"name": "the cook",
 
254
  }
255
  ```
256
 
257
+ The place entry is `"tag": "kitchen"` because the beat says `@kitchen`. **The
258
+ two spellings are one string.** Naming the register entry `kitchen_plate`,
259
+ `kitchen_ref` or `kitchen_bg` while the beat still reads `@kitchen` stops the
260
+ run: the node resolves tags by exact name and has no way to guess that two
261
+ different words meant the same picture.
262
+
263
+ `subjects` holds **plain prose, never tags.** `name` is the phrase that replaces
264
+ a subject's `@tag` from hop 2 on, so `"name": "@hero_face"` would resolve to
265
+ itself and put a literal at-sign in front of the encoder. Write `"the cook"`.
266
+ The same goes for `locked` and `context`.
267
+
268
  Valid ref fields: `tag`, `file`, `subject`, `retention`, `desc`, `shots`. (There
269
  is also `slot`, which is derived from list position — never author it.)
270
 
pyproject.toml CHANGED
@@ -1,6 +1,6 @@
1
  [project]
2
  name = "comfyui-hand-tie-clips"
3
- version = "0.4.3"
4
  description = "Native MiniMax H3 Ref2VA chain: persistent references, previous-segment clip+audio pin, one queue."
5
  readme = "README.md"
6
  # SPDX string, not `{ file = "LICENSE" }`. The registry does not resolve the file
 
1
  [project]
2
  name = "comfyui-hand-tie-clips"
3
+ version = "0.4.4"
4
  description = "Native MiniMax H3 Ref2VA chain: persistent references, previous-segment clip+audio pin, one queue."
5
  readme = "README.md"
6
  # SPDX string, not `{ file = "LICENSE" }`. The registry does not resolve the file
refs.py CHANGED
@@ -75,11 +75,30 @@ def _norm_subject(key, raw):
75
  if unknown:
76
  _fail(f"subject {num}: unknown field(s) {unknown}. "
77
  f"Valid: {list(SUBJECT_FIELDS)}")
78
- return num, {
79
  "name": str(raw.get("name") or "").strip(),
80
  "locked": str(raw.get("locked") or "").strip(),
81
  "context": str(raw.get("context") or "").strip(),
82
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
 
84
 
85
  def _norm_ref(raw, i):
 
75
  if unknown:
76
  _fail(f"subject {num}: unknown field(s) {unknown}. "
77
  f"Valid: {list(SUBJECT_FIELDS)}")
78
+ out = {
79
  "name": str(raw.get("name") or "").strip(),
80
  "locked": str(raw.get("locked") or "").strip(),
81
  "context": str(raw.get("context") or "").strip(),
82
  }
83
+ # All three fields are PROSE that reaches the encoder verbatim -- `name`
84
+ # is what `resolve_tags` substitutes for a subject's tag on hop 2+, and
85
+ # `locked`/`context` are copied into the continuity line. A reference tag
86
+ # written here therefore resolves to itself: a model that wrote
87
+ # `"name": "@cook_face"` put a literal at-sign into the conditioning of
88
+ # every continuation hop, and nothing anywhere caught it -- the plan
89
+ # parsed, the tags resolved, the render completed, and two of three hops
90
+ # were conditioned on a token no one intended. Loud here, or silent
91
+ # forever.
92
+ import re # local, as everywhere else in this module
93
+ for field, text in out.items():
94
+ hit = re.search(r"@[A-Za-z0-9_]+", text)
95
+ if hit:
96
+ _fail(f"subject {num}: `{field}` contains the reference tag "
97
+ f"'{hit.group(0)}'. These fields are prose that reaches the "
98
+ f"text encoder literally -- @tags only resolve inside a "
99
+ f"shot's beat. Write the plain words instead, e.g. "
100
+ f"\"the cook\".")
101
+ return num, out
102
 
103
 
104
  def _norm_ref(raw, i):