legendarydragontamer commited on
Commit
51a9974
·
0 Parent(s):
.gitignore ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Byte-compiled files
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+
6
+ # Python build artifacts
7
+ build/
8
+ dist/
9
+ *.egg-info/
10
+ .eggs/
11
+
12
+ # Virtual environments
13
+ .venv/
14
+ venv/
15
+ env/
16
+
17
+ # Test and coverage output
18
+ .pytest_cache/
19
+ .coverage
20
+ htmlcov/
21
+
22
+ # Type checker caches
23
+ .mypy_cache/
24
+ .ruff_cache/
25
+ .pyright/
26
+
27
+ # Local environment files
28
+ .env
29
+ .env.*
30
+
31
+ # OS/editor files
32
+ .DS_Store
33
+ .idea/
34
+ .vscode/
35
+ *.wav
36
+ !sample_session.wav
37
+
38
+ # Offline research scratch — never ship to the Space (keeps the submission clean)
39
+ research/
40
+ research/traces/
41
+ swe_chat_traces/
42
+ distractor_runs/
43
+ skill_eval_runs/
44
+ run_swe_chat_phases_v2.py
45
+ fetch_swe_chat_v2.py
46
+ nebius_loader.py
47
+ trace2skill_faithful.py
48
+ promotion_engine.py
49
+ gotcha_cluster.py
50
+ session_store.py
51
+ artifact_meta.py
52
+ swe_chat_loader.py
CODEX_BUILD_LOG.md ADDED
@@ -0,0 +1,276 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Codex Build Log: TurboSkillSlug
2
+
3
+ TurboSkillSlug was built as a small Gradio app with Codex acting as the primary coding partner. The project began as an empty repository and grew into a Hugging Face Space that accepts a build-session audio recording, transcribes it, extracts a structured recap, writes a draft `SKILL.md`, and generates a procedural SVG shell from the shape of the session.
4
+
5
+ This log summarizes how Codex was used, what it implemented, where development got sticky, and what the project taught us.
6
+
7
+ ## Starting Point
8
+
9
+ The first Codex work was straightforward scaffolding:
10
+
11
+ - Created the Gradio app shell in `app.py`.
12
+ - Added `requirements.txt`, `.gitignore`, and Hugging Face Space metadata in `README.md`.
13
+ - Added the first tests under `tests/`.
14
+ - Set up the basic "upload audio, return output" flow.
15
+
16
+ That initial phase was the part Codex handled most naturally: file structure, simple UI wiring, dependency declarations, and tests.
17
+
18
+ ## Features Codex Implemented
19
+
20
+ ### Audio Transcription
21
+
22
+ Codex added `transcribe.py` to send uploaded audio to `openai/whisper-large-v3-turbo` through Hugging Face. This went through several iterations:
23
+
24
+ - Started with `InferenceClient`.
25
+ - Pinned and adjusted Gradio and Python versions to handle Space launch issues.
26
+ - Tried provider-specific routing and file-path based ASR calls.
27
+ - Tried raw bytes and `BytesIO`.
28
+ - Finally bypassed `InferenceClient` for transcription and used a direct `httpx` POST to the HF router with an explicit `Content-Type`.
29
+
30
+ The final lesson was simple: for this ASR path, explicit HTTP was more reliable than fighting provider abstractions.
31
+
32
+ ### Session Extraction
33
+
34
+ Codex added `extract.py` to call `Qwen/Qwen2.5-7B-Instruct` and produce:
35
+
36
+ - `duration_minutes`
37
+ - `themes`
38
+ - `approaches_tried`
39
+ - `dead_ends`
40
+ - `breakthroughs`
41
+ - `gotchas`
42
+ - `sentiment_arc`
43
+ - `skill_md`
44
+ - `slug_voice`
45
+
46
+ This became the emotional center of the project. The extraction prompt had to do more than produce valid JSON; it had to preserve the core promise of the app: the slug is a witness, not a generic summary writer.
47
+
48
+ Codex iterated on:
49
+
50
+ - strict JSON parsing
51
+ - required output keys
52
+ - constrained sentiment labels
53
+ - graceful validation and patching for imperfect model output
54
+ - structured `skill_md` sections
55
+ - rules forcing `slug_voice` to reference transcript-specific moments
56
+ - dead-end counting rules so smooth sessions stay smooth and messy sessions show their mess
57
+
58
+ The final prompt removes copyable gold-set examples and instead describes behavior directly. That change came from testing: when examples were present, Qwen copied them verbatim across sessions.
59
+
60
+ ### Shell SVG Generation
61
+
62
+ Codex added `shell.py`, a procedural SVG generator that turns session structure into visual form:
63
+
64
+ - duration affects size and spiral complexity
65
+ - approaches affect spiral density
66
+ - dead ends become dark knots
67
+ - gotchas become rim jewels
68
+ - breakthroughs affect the glowing aperture
69
+ - sentiment arc drives the color story
70
+
71
+ Later commits polished the shell with nacre texture, rim glow, HSL palettes, and a central eye. This was where correctness alone was not enough; the shell had to feel demo-worthy.
72
+
73
+ ### Gradio App Wiring
74
+
75
+ Codex wired the full pipeline in `app.py`:
76
+
77
+ - upload or record audio
78
+ - transcribe with Whisper
79
+ - extract structured session data with Qwen
80
+ - override duration with the real audio length
81
+ - render the slug recap
82
+ - render the shell SVG inline
83
+ - expose raw JSON
84
+ - write downloadable `shell.svg`, `skill.md`, and `slug_recap.txt`
85
+ - include a sample session in the UI
86
+
87
+ The interface became a two-column app: input on the left, recap and shell on the right.
88
+
89
+ ### Documentation And Deployment
90
+
91
+ Codex helped maintain the README and deployment support:
92
+
93
+ - Hugging Face Space frontmatter
94
+ - model table and parameter count
95
+ - shell interpretation table
96
+ - demo and social links
97
+ - tags for the Build Small hackathon
98
+ - deployment script for HF Space remotes
99
+
100
+ The history also includes deploy commits to Hugging Face Space remotes, separate from the main GitHub history.
101
+
102
+ ## Challenges During Development
103
+
104
+ ### Provider Routing Was Fragile
105
+
106
+ The Hugging Face provider layer behaved differently for Whisper and Qwen. Whisper needed the `hf-inference` route at first, while Qwen needed automatic routing. Then Whisper still failed because content type was not being set correctly for audio bytes. The reliable fix was a direct `httpx` request.
107
+
108
+ Codex was useful here because it could keep applying narrow fixes quickly, but the development lesson was to stop abstracting once the abstraction became the problem.
109
+
110
+ ### Gradio And Python Compatibility
111
+
112
+ Several commits adjusted runtime compatibility:
113
+
114
+ - pinned Python 3.12 to avoid `audioop` removal issues in Python 3.13
115
+ - pinned and then bumped Gradio versions to fix schema and launch errors
116
+ - added required Hugging Face Space metadata
117
+
118
+ These were not glamorous changes, but they were the difference between a local idea and a deployable demo.
119
+
120
+ ### The Slug Voice Was The Hardest Part
121
+
122
+ The most important failure was also the most instructive: the slug started copying example utterances word for word.
123
+
124
+ The intent was to show tone, but the model treated the examples as content. That produced the exact failure mode the project was trying to avoid: a supposedly present witness saying generic lines that did not happen in the session.
125
+
126
+ The prompt eventually moved away from examples and toward hard behavioral rules:
127
+
128
+ - every line must reference this transcript
129
+ - use concrete details such as tools, errors, variable names, and actions
130
+ - never summarize, advise, or invent
131
+ - if there is no evidence, stay silent
132
+
133
+ That changed the prompt from "write in this style" to "obey this witness contract."
134
+
135
+ ### Validators Had To Be Gentle
136
+
137
+ Early validation crashed when Qwen missed required `skill_md` sections. That was technically correct but bad product behavior. The app should still show the transcript, recap, shell, and downloads even if one structured field is imperfect.
138
+
139
+ Codex changed the validators to patch or clamp where possible:
140
+
141
+ - missing `skill_md` sections get stubbed with `_(not captured)_`
142
+ - invalid sentiment values map to the closest allowed label
143
+ - missing slug voice lines are padded rather than killing the whole request
144
+
145
+ The lesson was to validate for resilience, not purity.
146
+
147
+ ### Dead Ends And Sentiment Needed Rubrics
148
+
149
+ Testing three different scripts revealed extraction bias:
150
+
151
+ - too few dead ends in messy sessions
152
+ - invented dead ends in clean sessions
153
+ - endings defaulting to `resolved`
154
+
155
+ Codex added explicit rubrics so the model knew that:
156
+
157
+ - every failed approach is a dead end
158
+ - clean explorations should have no dead ends
159
+ - quick pleasant sessions can end `joyful`
160
+ - long draining sessions can end `exhausted`
161
+ - `resolved` is only for a clear bug-fix arc
162
+
163
+ This made the shell's visual grammar more honest, because the shell depends directly on those extracted features.
164
+
165
+ ## What Was Learned
166
+
167
+ Codex was strongest when the task could be expressed as concrete code movement: scaffold this app, add this module, wire these outputs, parse this JSON, commit and push this exact fix.
168
+
169
+ The human role mattered most where taste and truth mattered:
170
+
171
+ - deciding what the slug should sound like
172
+ - noticing when "technically valid" output felt false
173
+ - tuning the shell from generated geometry into something delightful
174
+ - deciding when a model failure should be patched instead of treated as fatal
175
+
176
+ The project also showed that prompt engineering is product engineering. The prompt is not just instruction text; it is part of the app's behavior, reliability, and user trust.
177
+
178
+ ## Final Shape
179
+
180
+ TurboSkillSlug now works as a complete small demo:
181
+
182
+ 1. The user uploads a narrated build session.
183
+ 2. Whisper transcribes it.
184
+ 3. Qwen extracts the session structure and writes the slug's witness lines.
185
+ 4. The app renders a recap and a procedural shell.
186
+ 5. The user can download the shell SVG, draft `SKILL.md`, and slug recap.
187
+
188
+ The result is intentionally small, strange, and specific. Codex built most of the machinery. The repeated tests and corrections taught it where the machinery needed to become more honest.
189
+
190
+ ## Continued Development (post-MVP)
191
+
192
+ After the audio-to-shell MVP described above, Codex carried the project through five
193
+ more rounds of work. Each was a discrete, committed change with verification before
194
+ push. The commit hashes below are the Codex-attributed history in the repo.
195
+
196
+ ### The Slug Speaks: Chatterbox TTS (commit 0d1a73f)
197
+
198
+ Codex wired voice into the app so the slug does not just write its witness lines, it
199
+ speaks them. The work added a `_speak_recap` function that posts the recap text to a
200
+ Chatterbox TTS endpoint running on Modal, decodes the returned base64 audio into a
201
+ `slug_speaks.wav`, and surfaces it through a new Gradio audio component labeled "the
202
+ slug speaks." The recap always closes on the slug's signature line, "I was here."
203
+
204
+ Codex verified with `py_compile` and the app test suite before pushing, then ran the
205
+ deploy script to force-update both Spaces.
206
+
207
+ ### Readiness Feedback and Voice Deduplication (commit 57e0748)
208
+
209
+ A round of product-quality fixes Codex handled cleanly: the upload control now gives
210
+ visible readiness feedback so a premature click is impossible, a redundant SlugVoice
211
+ call was removed, and the "I was here" signature was deduplicated by avoiding direct
212
+ mutation of the `slug_voice` list. Small changes, but they are the difference between
213
+ an app that feels finished and one that feels like a prototype.
214
+
215
+ ### Skill-Uplift Eval Suite (commit a3c65db)
216
+
217
+ This is where the project gained a research spine. Codex committed a suite of three
218
+ blind, calibrated evaluations measuring exactly when a generated `SKILL.md` changes a
219
+ frontier model's behavior. The scripts use one model to answer and an independent
220
+ model to judge the primary recommendation, with leak guards and saved raw generations
221
+ for re-scoring.
222
+
223
+ The finding was sharp and a little humbling: uplift depends on knowledge provenance,
224
+ not task difficulty. General algorithmic skills gave 0.0 uplift because the model
225
+ already holds that knowledge in its weights. Well-known engineering traps also gave
226
+ 0.0. Only novel, non-public rules produced uplift (+1.0, with rescues and no
227
+ regressions). A skill file helps a frontier model only when it carries knowledge that
228
+ could not have been in training data: private behavior, post-cutoff facts, project
229
+ conventions, or genuine discoveries.
230
+
231
+ One honest correction is recorded in the commit itself: an early signature-based
232
+ scorer miscounted trap *warnings* as trap *failures*, and was replaced with a model
233
+ judge of the primary recommendation. The novel cases are fictional by necessity, so
234
+ the knowledge they test cannot already be in any model's weights.
235
+
236
+ ### First SceneGraph Lens: Turn the Shell in 3D (commit 508da94)
237
+
238
+ Codex wired the first of a planned set of alternate renderers. The same session that
239
+ produces the flat SVG shell now also drives a real 3D object you can orbit: a Three.js
240
+ nautilus with iridescent nacre, where the spiral growth, the knots (dead ends), the
241
+ glowing aperture (breakthrough), and the colour arc all come from the session data.
242
+ This sits behind a deterministic SceneGraph layer, so the semantic description of the
243
+ scene is separate from how any one lens renders it.
244
+
245
+ The wiring included the package structure for the `scenegraph/` modules and their
246
+ renderer. A later fix corrected the Three.js source to load from jsdelivr after the
247
+ original CDN was blocked by the iframe sandbox, which had kept the 3D scene from ever
248
+ initializing.
249
+
250
+ ### Compliance Guard for the 32B Cap (commit 0b42267)
251
+
252
+ The hackathon requires every model the app depends on to stay under 32B parameters.
253
+ The app was already compliant, the live path uses a 1.5B extractor on Modal plus
254
+ Whisper, with a labeled Qwen-7B fallback, but Codex added a guard so a future edit
255
+ cannot silently break that. `app.py` marks itself as the live app at startup, and
256
+ `extract.py` checks the model name before each inference call. Anything over the cap
257
+ raises immediately instead of shipping a violation. Offline evaluation scripts never
258
+ enable the runtime flag, so their use of larger models for research is unaffected.
259
+
260
+ The honest boundary of this guard is recorded too: it checks model names passed on the
261
+ app side and cannot see inside the Modal endpoint, so the durable compliance fact is
262
+ that the deployed serving endpoint runs Qwen2.5-1.5B and Whisper, which is verifiable
263
+ from the Modal serve code.
264
+
265
+ ## What the Later Rounds Taught
266
+
267
+ The MVP showed Codex was strongest at concrete code movement. The later rounds showed
268
+ something more specific: Codex was reliable at carrying a precisely-described change
269
+ all the way through verification and deploy, compile-check, run the app tests, commit
270
+ with a real message, push, and force-update the Spaces, without dropping a step.
271
+
272
+ The judgment calls still belonged to the human side: deciding that a skill file's value
273
+ comes from provenance rather than difficulty, noticing that an eval scorer was counting
274
+ the wrong thing, choosing to add a compliance tripwire before it was ever needed. Codex
275
+ turned those decisions into committed, tested code quickly and faithfully. That division
276
+ held across every round: the human decides what honest looks like, and Codex builds it.
EVAL_REPORT.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Groundedness Eval: does the fine-tuned 1.5B hallucinate more than the 7B it replaces?
2
+
3
+ TurboSkillSlug's whole promise is a witness that only says what it saw. So the
4
+ extraction model has one job above all others: do not invent facts. This eval
5
+ measures exactly that, comparing the shipped fine-tuned 1.5B LoRA against the
6
+ Qwen-7B it replaced and against its own un-tuned 1.5B base, on 25 held-out
7
+ transcripts the LoRA never saw in training.
8
+
9
+ ## Setup
10
+
11
+ - **25 held-out transcripts** (never in the 167-pair training set).
12
+ - **Three systems**, same prompt, same decoding (temp 0.3, top-p 0.9, 768 tokens):
13
+ prompted Qwen-7B, prompted Qwen-1.5B base, fine-tuned 1.5B LoRA.
14
+ - **Two groundedness metrics per extracted fact** (approaches, dead ends,
15
+ breakthroughs, gotchas):
16
+ - *Lexical*: fraction of the fact's content words present in the transcript.
17
+ - *Semantic*: max cosine similarity of the fact's embedding to any sentence
18
+ window of the transcript (all-MiniLM-L6-v2), grounded if >= 0.55.
19
+ - **Raw generations saved before scoring**, so the metric can be revised
20
+ without re-running the models.
21
+
22
+ ## Results
23
+
24
+ | system | semantic | lexical | mean sim | parse | facts |
25
+ |-----------------|---------:|--------:|---------:|------:|------:|
26
+ | prompted 7B | 0.716 | 0.576 | 0.640 | 24/25 | 272 |
27
+ | prompted 1.5B | 0.565 | 0.390 | 0.567 | 21/25 | 140 |
28
+ | **LoRA 1.5B** | **0.762**| 0.378 | 0.649 | 21/25 | 195 |
29
+
30
+ ## What this shows
31
+
32
+ **The fine-tuned 1.5B matches and slightly exceeds the 7B on semantic
33
+ groundedness (0.76 vs 0.72), at roughly a third of the active parameters.**
34
+ The mean per-fact similarity agrees (0.649 vs 0.640).
35
+
36
+ The lexical and semantic metrics disagree sharply for the LoRA: it has the
37
+ *lowest* lexical overlap (0.378) but the *highest* semantic groundedness
38
+ (0.762). That gap is the point. The fine-tune taught the model to restate the
39
+ transcript's meaning in its own words rather than copy spans. Word-overlap
40
+ scoring punishes that; embedding scoring credits it. The LoRA paraphrases
41
+ faithfully, which is what a good extractor should do.
42
+
43
+ ## What this does NOT show, and the caveats we are not hiding
44
+
45
+ - **The LoRA is less reliable at producing valid JSON: 21/25 vs the 7B's
46
+ 24/25.** That is a real cost of the smaller model. In the live app a
47
+ brace-walking parser and field validators recover most malformed output, but
48
+ the raw parse rate is what the table reports, unsoftened.
49
+ - **The semantic threshold is imperfect.** A calibration block of six
50
+ hand-labeled cases (run before scoring, printed in the logs) passed 5/6: one
51
+ true paraphrase fell just under the 0.55 line. The single miss is a
52
+ *false negative* (a grounded fact scored ungrounded), which means the LoRA's
53
+ real groundedness is if anything *underestimated* here, not inflated. We
54
+ report the number the fixed threshold produced rather than tuning it after
55
+ seeing results.
56
+ - **25 transcripts is a small sample.** Treat the gaps as directional, not
57
+ precise. The LoRA-vs-7B semantic difference is small enough that the honest
58
+ claim is "matches or slightly exceeds," not "beats."
59
+
60
+ ## Honest one-line summary
61
+
62
+ A 1.5B LoRA fine-tune reaches 7B-level semantic groundedness on held-out
63
+ sessions at a third the active size, by learning to paraphrase rather than
64
+ copy; it pays for this with a lower valid-JSON rate (21/25 vs 24/25), and the
65
+ metric itself is calibrated to within 5/6 on known cases.
66
+
67
+ ---
68
+
69
+ *Reproduce:* `modal run semantic_eval.py`. Raw generations, per-fact scores,
70
+ calibration outcome, threshold, and embedding model are all saved in
71
+ `eval_results_semantic.json` and `eval_raw_outputs.json`.
EVAL_REPORT_TEMPLATE.md ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Skill-Uplift Eval Report
2
+
3
+ > Fill this in with the ACTUAL numbers from `run_skill_eval.py`. The template is
4
+ > written so an honest result is the easy result. Do not delete the caveats.
5
+
6
+ ## Setup
7
+ - Answerer model: `<EVAL_ANSWERER_MODEL>`
8
+ - Grader model (blind, different vendor): `<EVAL_GRADER_MODEL>`
9
+ - Skill generator: the shipped `skill_builder.build_skill_md`
10
+ - Cases: N = `<n_scored>` (held-out tasks, each a DISTINCT problem in the same
11
+ class as its source session; leaked skills excluded: `<n_leaked_excluded>`)
12
+ - Raw generations saved: `skill_eval_runs/` (anyone can re-score)
13
+
14
+ ## Grader calibration (run BEFORE trusting the uplift)
15
+ - Agreement with human labels: `<X/Y>`
16
+ - If agreement is low, the uplift number is unreliable; say so explicitly.
17
+
18
+ ## Result
19
+ | condition | mean score |
20
+ |---|---:|
21
+ | no skill (baseline) | `<baseline_no_skill_mean>` |
22
+ | with skill | `<with_skill_mean>` |
23
+ | **uplift** | **`<uplift>`** |
24
+
25
+ - Win / tie / loss across cases: `<wins>` / `<ties>` / `<losses>`
26
+ - Per-case deltas: `<paste from runner>`
27
+
28
+ ## Honest reading (write the true one)
29
+ Pick the sentence that matches the data; do not overstate:
30
+ - Positive & consistent: "The generated skill produced measurable uplift
31
+ (+`<uplift>`) on held-out tasks in the same class, winning `<wins>`/`<n>`."
32
+ - Small/mixed: "The skill produced small, inconsistent uplift (+`<uplift>`);
33
+ it helped on `<wins>` cases and was neutral/negative on the rest."
34
+ - Near-zero: "On this set, the skill did not produce measurable uplift over the
35
+ frontier baseline. The baseline was already strong (`<baseline>`), leaving
36
+ little headroom; a harder task set would test this better."
37
+
38
+ ## Caveats (keep all that apply)
39
+ - Small N; indicative, not a benchmark (same posture as the 25-transcript
40
+ groundedness eval).
41
+ - Single grader model; blinding reduces but does not remove grader bias, hence
42
+ the calibration check above.
43
+ - Uplift depends on task difficulty: where the baseline already scores ~1.0 there
44
+ is no room to show uplift. Baseline headroom is reported so this is visible.
45
+ - Skills that leaked the task answer were excluded (`<n_leaked_excluded>`),
46
+ not silently scored.
FINDINGS.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # When does a SKILL.md actually help a 2026 frontier model? (measured)
2
+
3
+ Three blind, calibrated evals against `anthropic/claude-opus-4.6` (answerer) and
4
+ `openai/gpt-5.1` (independent judge). Raw generations saved; method below.
5
+
6
+ ## The result in one table
7
+
8
+ | Skill content | Held-out task class | Avoidance / score WITHOUT skill | WITH skill | Uplift |
9
+ |---|---|---:|---:|---:|
10
+ | General algorithmic procedure | standard tree-DP, Markov absorption | 1.0 | 1.0 | **0.0** |
11
+ | Well-known engineering traps | Kahan summation, check-then-act race, N+1 query | 1.0 | 1.0 | **0.0** |
12
+ | Novel / non-public rules | fictional APIs (zthrumdb fence, qbucket reset, flazon reversal) | 0.0 | 1.0 | **+1.0** |
13
+
14
+ 3 rescues, 0 regressions in the novel-trap condition. Grader calibration 3/3.
15
+
16
+ ## What this means (the actual finding)
17
+
18
+ A frontier model's weights already contain the public, written-down corpus of
19
+ software knowledge. A skill file that repeats any of it gives **zero** uplift,
20
+ confirmed twice, on both easy tasks and famous footguns the model handles unaided.
21
+
22
+ A skill file gives **large** uplift exactly when it carries knowledge that could
23
+ NOT have been in the training data: private/proprietary system behavior, post-cutoff
24
+ facts, project-specific conventions, or genuinely novel discoveries. The dividing
25
+ line is not task difficulty. It is **whether the knowledge could have been public.**
26
+
27
+ ### A second, sharper observation from the raw data
28
+
29
+ In the unaided novel cases, the model did not just answer wrong, it often went
30
+ **"unclear": it hedged, asked for clarification, or refused to use the unknown API.**
31
+ A frontier model senses when it lacks the knowledge and stalls. The skill does not
32
+ merely correct wrong answers; it **unblocks the model on systems it otherwise cannot
33
+ act on at all.** That is the higher-value case: not "answer better," but "able to
34
+ proceed where it was previously stuck."
35
+
36
+ ## Why this is the honest, useful framing for TurboSkillSlug
37
+
38
+ The slug's value is NOT in summarizing a session of standard work, that produces a
39
+ skill the model ignores (uplift 0.0). The value is in capturing the **negative,
40
+ private, non-obvious knowledge** from a real session: the trap specific to this
41
+ codebase, the undocumented behavior, the dead end that cost an hour. Fed that, the
42
+ generated SKILL.md measurably changes a frontier model's behavior (+1.0).
43
+
44
+ This is a sharper claim than "skills help," and we can defend every part of it with
45
+ data and published raw outputs.
46
+
47
+ ## Method (anti-self-deception safeguards)
48
+
49
+ - Held-out tasks DISTINCT from the source session (transfer, not memorization).
50
+ - Blind judge: a DIFFERENT vendor's model, scoring the answer's PRIMARY
51
+ recommendation, with "warns about the trap then gives the fix" counted as CORRECT.
52
+ (An earlier signature-matching scorer was discarded because it miscounted
53
+ warnings as failures; the LLM judge fixed this. We report the correction.)
54
+ - Leak guard: any skill containing the literal task answer is excluded.
55
+ - Calibration run before trusting numbers (grader agreed with human labels 3/3).
56
+ - Raw generations saved before scoring; small N, reported as indicative not a benchmark.
57
+
58
+ ## Honest limitations
59
+
60
+ - Small N (2-3 per condition). Indicative, not a benchmark. The effect sizes
61
+ (0.0 vs 1.0) are large and consistent, but the sample is small.
62
+ - Single answerer + single judge model. Blinding and a cross-vendor judge reduce
63
+ but do not eliminate model-specific effects.
64
+ - The novel cases are fictional by necessity (to guarantee non-derivability); they
65
+ stand in for real private/proprietary knowledge, which is the production case.
HONEST_SUBMISSION.md ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Honest Submission Notes — TurboSkillSlug
2
+
3
+ This document is the unvarnished account of what TurboSkillSlug actually does,
4
+ what is genuinely strong, what is partial, and what is aspirational. If a claim
5
+ is not in here with its caveats, treat the marketing copy as marketing.
6
+
7
+ Demo video: https://youtu.be/qSP9olWRv7o
8
+ Space: https://huggingface.co/spaces/build-small-hackathon/TurboSkillSlug
9
+ Code: https://github.com/AnubhavBharadwaaj/turbo-skill-slug
10
+
11
+ ## What it does, honestly
12
+
13
+ You give the slug a build session two ways: narrate it aloud (audio), or drop a
14
+ Claude Code / Codex CLI session trace (`.jsonl`). It transcribes/parses, extracts
15
+ a structured record, and returns four artifacts: a transferable SKILL.md, a
16
+ grounded second-person recap in a fine-tuned "slug voice," a procedural SVG shell,
17
+ and a thermal receipt. The shell is born on screen as a scroll that unrolls along
18
+ its spiral arm, with a byōbu (folding-screen) battle drawn across it.
19
+
20
+ ## What is genuinely strong
21
+
22
+ - **Two custom 1.5B LoRA adapters, both published and both used.** Voice
23
+ (500 pairs, loss 0.89) and extraction (167 pairs, loss 0.81). The extraction
24
+ LoRA replaced a prompted Qwen-7B, bringing the primary pipeline to ~2.6B active
25
+ params (Whisper 809M + one 1.5B base serving both adapters on a single T4).
26
+ - **The groundedness eval is real and published with its raw data.** On 25
27
+ held-out transcripts the fine-tuned 1.5B reaches semantic groundedness 0.76 vs
28
+ the 7B's 0.72, at a third the active size, by paraphrasing rather than copying.
29
+ - **The shell is fully procedural and traceable.** Every visual element derives
30
+ from a real session feature: dead ends are knots/fallen warriors, gotchas are
31
+ jewels/archers, the breakthrough is the aperture/dragon, sentiment is the color
32
+ arc. No image generation.
33
+ - **The SKILL.md is built for real uplift**, gotchas-first (symptom→cause→fix),
34
+ "what does NOT work and why," transferable principles. Built and de-leaked
35
+ deterministically from the structured extraction, not trusted to model prose.
36
+ - **Trace input works.** A real Claude Code or Codex session log runs through the
37
+ same pipeline as audio. Judges can feed their own logs.
38
+ - **Graceful degradation is real and now observable.** If the extract adapter
39
+ emits invalid JSON, the 7B fallback catches it. If the voice adapter is down, a
40
+ deterministic extraction-based voice net prevents placeholder recaps. Every
41
+ voice-path outcome is logged with a `[VOICE]` tag (EXTRACT_DOWN / VOICE_DOWN /
42
+ VOICE_EMPTY / VOICE_OK / FALLBACK_VOICE_OK / NET_FROM_EXTRACTION / …) so failures
43
+ are diagnosable, not mysterious.
44
+
45
+ ## What is partial or has real caveats
46
+
47
+ - **Extraction parse reliability: 21/25 vs the 7B's 24/25.** The smaller model
48
+ produces invalid JSON more often. In the app a brace-walking parser and field
49
+ validators recover most of this, and the 7B fallback catches the rest, but the
50
+ raw rate is a real cost of going small, and we report it unsoftened. (You will
51
+ see `[VOICE] EXTRACT_DOWN` in the logs when this fires; the fallback then runs.)
52
+ - **The groundedness metric is calibrated to 5/6, not 6/6.** A hand-labeled
53
+ calibration block (run before scoring, printed in the logs) missed one case:
54
+ a grounded paraphrase scored just under threshold. The miss understates the
55
+ LoRA rather than inflating it. 25 transcripts is a small sample; treat the
56
+ LoRA-vs-7B gap as "matches or slightly exceeds," not "beats."
57
+ - **SKILL.md quality depends on extraction quality.** A good session yields
58
+ transferable gotchas; a thin session yields thinner ones. An optional one-shot
59
+ enrichment pass expands terse gotchas, gated by env and best-effort. We caught
60
+ and fixed a real bug where few-shot examples in the enrichment prompt leaked
61
+ into output (a game-theory skill got tree-coloring gotchas); the prompts are
62
+ de-leaked and a guard now rejects any leaked phrasing.
63
+ - **The scroll-unroll animation ships ~1.7MB per shell** (14 stacked growth
64
+ stages, so the parchment lays down ALONG the spiral arm rather than as a radial
65
+ wipe). Browser-rendered via SMIL in a sandboxed iframe, with a "watch it unroll
66
+ again" replay. On a slow connection the first paint takes a beat. If SMIL fails,
67
+ the full shell still shows (nothing is hard-hidden); animation is enhancement,
68
+ not a dependency.
69
+ - **The 3D paper curl rides the spiral tip via animateMotion.** Verified
70
+ frame-by-frame in offline renders (cairosvg + headless Chromium). Exact
71
+ tip-tracking in every browser is the part most likely to need a small tweak.
72
+
73
+ ## What was freshly added near the deadline (use with that in mind)
74
+
75
+ - **Shared terrarium gallery.** Kept shells save to a Modal Volume; a grid shows
76
+ all of them newest-first; each has a complete clickable permalink that re-loads
77
+ and re-animates that shell. Freshly built and lightly tested. Honest caveats:
78
+ each grid card is its own iframe, so past ~30-40 shells the grid gets heavy
79
+ (lazy-loading is the fix, not yet done). Gallery saves depend on the Modal
80
+ endpoint URLs matching the client; if they drift, a save fails and the UI says
81
+ so rather than failing silently.
82
+ - **Battle Trace (experimental).** A temporal replay of the same session as a war
83
+ between you (the Agent) and the Environment, fed by the SAME extraction (no
84
+ second parser, no OTel). Framed as complementary to the shell: the shell is the
85
+ slug's *memory* of how the battle ended (the frozen folding screen); the trace
86
+ is the *replay* of it in time. It is Canvas-2D with simple figurative
87
+ combatants (a samurai general, a horned adversary, fallen warriors, a dragon at
88
+ the breakthrough). Honest about its level: these are clean vector figures, not
89
+ game-quality sprite art. Labeled "experimental" in the UI.
90
+
91
+ ## What is aspirational / not built
92
+
93
+ - **Session diff view** (compare two sessions' shells) is not built.
94
+ - **Closing the parse-rate gap** needs more training pairs and constrained
95
+ decoding; not done.
96
+ - **Higher-fidelity Battle Trace art** (detailed sprite-grade samurai) is not
97
+ built; the current figures are deliberately simple vector shapes (see the
98
+ Battle Trace note above).
99
+
100
+ ## Infrastructure honesty
101
+
102
+ - The Qwen-7B still exists in the codebase but ONLY as a labeled fallback when the
103
+ Modal extract path returns nothing usable (including the invalid-JSON case
104
+ above). The primary path is the 1.5B dual adapter. The README architecture table
105
+ reflects this.
106
+ - The dual-adapter server can hold one warm container for demo reliability
107
+ (~$0.60/hr). It should be stopped after judging (`modal app stop
108
+ slug-dual-serve`).
109
+
110
+ ## How to verify our claims
111
+
112
+ - Groundedness: `modal run semantic_eval.py` reproduces the table; raw
113
+ generations are published in the eval dataset so anyone can re-score.
114
+ - Trace input: drop the sample `.jsonl` (one click) or your own session log.
115
+ - Models: both LoRA adapters are public on the Hub.
116
+ - Shell traceability: change a session's sentiment or dead-end count and watch
117
+ the shell's colors and knots change accordingly.
118
+ - Failure handling: the Space logs tag every voice-path outcome with `[VOICE]`.
119
+
120
+ ## The one-sentence honest summary
121
+
122
+ A small, slow, genuinely ~2.6B pipeline that turns a coding session into a
123
+ transferable skill, a grounded recap, and a procedural shell, measured honestly
124
+ (including where the small model costs us), with graceful, observable degradation,
125
+ and every model and the eval data published for anyone to check.
INTEGRATION.md ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Integrating the three phases into the live slug
2
+
3
+ All modules are dependency-light, frozen-model, and run on the Space. They reuse the
4
+ existing extraction and the OpenRouter/HF completer you already have.
5
+
6
+ ## Files
7
+ - artifact_meta.py Phase 1: lifecycle metadata (provenance/confidence/usage/validation)
8
+ - rule_phrasing.py Phase 1: RuleShaping negative-constraint phrasing (LLM + deterministic fallback)
9
+ - session_store.py Phase 2: store of past sessions (JSON; use the Modal volume in prod)
10
+ - gotcha_cluster.py Phase 2: cluster recurring gotchas (embedding or LLM-judge backend)
11
+ - promotion_engine.py Phase 2: promote recurring clusters -> validated L3 rules; demote on failure
12
+ - multi_analyst_extract.py Phase 3: Trace2Skill-style N-analyst recurrence merge (higher-fidelity extraction)
13
+
14
+ ## Wiring (minimal)
15
+
16
+ 1. After each session's extraction, persist it:
17
+ store = SessionStore("/vol/gallery/sessions.json") # Modal volume in prod
18
+ store.add(session_id, extraction)
19
+
20
+ 2. Phase 3 (optional, improves fidelity): replace single-pass gotcha extraction with
21
+ gotchas = multi_analyst_gotchas(session_text, completer, n_analysts=4,
22
+ had_errors=bool(extraction.get("dead_ends")),
23
+ agree_min=2, judge_same=same_gotcha_judge)
24
+ Cost knob: n_analysts (3–5 cheap; larger = higher fidelity, higher cost).
25
+
26
+ 3. Phase 2 (the headline): run a promotion pass (cheap; do it after add, or on idle):
27
+ rules = promote(
28
+ store, k_threshold=3,
29
+ cluster_fn=lambda items: cluster_by_judge(items, same_gotcha_judge),
30
+ reshape_fn=lambda g: to_negative_constraint_llm(g, completer),
31
+ validate_fn=rule_validates, # reuse the distractor/novel eval harness
32
+ require_human=False, # set True for the 10/10 human-gated path
33
+ )
34
+ Persist promoted rules (with their ArtifactMeta) alongside sessions.
35
+
36
+ 4. Surface a new lens in the app: "the rules the slug has learned across your
37
+ sessions" — list each L3 rule with its provenance (which sessions), confidence,
38
+ and last-validated time. This is the lifecycle-governance story made visible.
39
+
40
+ ## The validation gate (important — don't skip)
41
+ `validate_fn(rule_text, cluster)` should reuse the held-out check you already built
42
+ (distractor_eval / novel_eval): does an answer that APPLIES the rule beat one that
43
+ ignores it, on a task derived from the cluster? Promoting without this is how you
44
+ promote "compact noise" (the spectrum paper's exact warning). Gate-only is the
45
+ default; `require_human=True` adds confirmation for the 10/10 path.
46
+
47
+ ## What to claim (honestly)
48
+ "Single self-generated skills give a frontier model ~0.0 uplift (our eval, matching
49
+ SkillsBench as cited in arXiv:2604.15877). We implement a working slice of that
50
+ paper's proposed solution — the 'missing diagonal' of adaptive upward compression:
51
+ the slug consolidates gotchas that recur across sessions into validated,
52
+ negative-constraint-phrased L3 rules (RuleShaping) carrying provenance/confidence/
53
+ lifecycle metadata, with demotion when a rule stops validating, and Trace2Skill-style
54
+ multi-analyst recurrence-merging for extraction fidelity. Honest limits: recurrence
55
+ is a proxy for value (mitigated by the validation gate + optional human confirmation);
56
+ small analyst-N is a cheaper approximation of Trace2Skill's 128."
57
+
58
+ ---
59
+
60
+ ## CORRECTIONS after reading the actual papers (not just the survey)
61
+
62
+ - Phase 3 was rebuilt as `trace2skill_faithful.py` after reading arXiv:2603.25158 in
63
+ full. The first version (`multi_analyst_extract.py`) was a simplified guess that
64
+ MISSED the paper's central mechanism: the error analyst is an AGENTIC, VALIDATION-
65
+ GATED loop (it must produce a verified causal cause or the trace is EXCLUDED), not a
66
+ single call. The paper shows the single-call version over-attributes parse errors as
67
+ root cause in 57% of cases and hallucinates causes — i.e. the leak/false-cause bug.
68
+ Consolidation is an LLM merge operator applied hierarchically with a prevalent-
69
+ pattern bias; success patches are lower-trust; low-support items route to references/.
70
+ Use `trace2skill_faithful.py`; keep `multi_analyst_extract.py` only as the simpler
71
+ fallback if the agentic loop is too costly on the Space.
72
+
73
+ - RuleShaping verified against arXiv:2604.11088 ("Do Agent Rules Shape or Distort?").
74
+ Confirmed: +7–14pp, negative constraints the only beneficial type, positive
75
+ directives hurt. Added nuance: rules work via CONTEXT PRIMING (random ~ curated),
76
+ state-dependent process guardrails ("don't X until Y") are the highest-benefit
77
+ category (63.8%), and rules are collectively helpful even when individually inert
78
+ (no degradation up to 50). `rule_phrasing.py` now biases toward state-dependent
79
+ negative guardrails.
80
+
81
+ ## REMAINING GAPS (do not overclaim)
82
+
83
+ 1. CLUSTERING IS UNVALIDATED ON REAL DATA. The promotion engine works when the
84
+ "same underlying gotcha" judge is good. On arbitrary messy sessions, clustering
85
+ quality is unknown — a bad judge promotes noise or misses recurrences. This is the
86
+ biggest open risk and needs a real embedding/LLM judge tested on labeled session data.
87
+
88
+ 2. THE ERROR-ANALYST VALIDATION GATE is faithful in STRUCTURE but ADAPTED: the paper
89
+ runs a minimal fix against ground truth in a sandbox. A narrated coding session fed
90
+ to the slug usually has no ground truth / sandbox, so we use strict self-verification
91
+ (cite trace evidence) instead. This is weaker than the paper's executable gate and
92
+ should be labeled as an adaptation, not a reproduction.
93
+
94
+ 3. NO END-TO-END RUN ON REAL SLUG SESSIONS YET. Everything is validated on controlled/
95
+ realistic-but-synthetic data. Whether real slug sessions cluster usefully is untested.
METHODOLOGY.md ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Skill-Uplift Eval — methodology (read before trusting any number)
2
+
3
+ ## The claim we are testing
4
+
5
+ A generated SKILL.md gives a capable LLM real uplift on a *related future task*,
6
+ over that same model solving the task with no skill. This is the judge's bar:
7
+ "useful even to a frontier model that is already capable without it."
8
+
9
+ ## Why this is hard to measure honestly (and how we handle each trap)
10
+
11
+ 1. **Confounding by model strength.** A strong model may ace the task with or
12
+ without the skill, hiding any uplift. We pick tasks at the *edge* of the
13
+ model's ability (it sometimes fails them unaided), where a skill can move the
14
+ needle. We report the unaided baseline pass rate so the headroom is visible.
15
+
16
+ 2. **Leakage / circularity.** If the eval task is the *same* problem the skill was
17
+ built from, the skill is just an answer key — meaningless. So the held-out task
18
+ is a DIFFERENT problem in the SAME class as the session. The skill must
19
+ transfer, not memorize. We state the session→task pairing explicitly.
20
+
21
+ 3. **Grader bias.** A grader that sees which answer used the skill will favor it.
22
+ The grader is BLIND: it receives the two answers in randomized order with the
23
+ condition labels stripped, and judges only correctness/quality.
24
+
25
+ 4. **Cherry-picking.** We fix the task set and the sessions BEFORE running, list
26
+ them here, and report every item including failures. No post-hoc dropping.
27
+
28
+ 5. **The "skill is just hints" objection.** A skill that smuggles the answer is not
29
+ uplift, it's cheating. We verify each skill contains transferable PROCEDURE
30
+ (gotchas, what-not-to-do), not the specific solution to the eval task. Any
31
+ skill whose gotchas name the eval task's exact answer is disqualified and noted.
32
+
33
+ ## Design
34
+
35
+ - N sessions, each paired with a DISTINCT held-out task in the same problem class.
36
+ - For each task, the SAME model answers twice:
37
+ A) NO-SKILL: task only.
38
+ B) WITH-SKILL: task + the SKILL.md generated from the paired session.
39
+ Order of which is generated first is irrelevant (separate calls), but the two
40
+ answers are handed to the grader in RANDOM order with labels stripped.
41
+ - A blind grader (a separate strong model) scores each answer 0..1 on task success,
42
+ not knowing which had the skill. We also run a small HUMAN-labeled calibration
43
+ set first (like the groundedness eval) to check the grader agrees with us.
44
+ - Uplift = mean(with_skill_score) - mean(no_skill_score). We report:
45
+ - per-task scores (both conditions),
46
+ - the unaided baseline (headroom),
47
+ - the win/tie/loss count (how often skill helped / didn't / hurt),
48
+ - the calibration agreement,
49
+ - and the raw generations, saved to disk, so anyone can re-score.
50
+
51
+ ## What an honest result looks like
52
+
53
+ We commit IN ADVANCE to reporting the number as-is. Possible honest outcomes:
54
+ - Clear positive uplift -> the skill works; report it.
55
+ - Near-zero uplift -> the skill is pleasant but not load-bearing; say so.
56
+ - Negative on some tasks -> the skill sometimes misleads; report which and why.
57
+ Any of these is a credible result. Only a hidden or massaged number is not.
58
+
59
+ ## Honest limitations (stated up front)
60
+
61
+ - Small N. This is an indicative eval, not a benchmark. We report N and treat the
62
+ result as directional, exactly as we did with the 25-transcript groundedness eval.
63
+ - Single grader model. Grader bias is reduced by blinding but not eliminated; the
64
+ calibration set is how we keep ourselves honest about it.
65
+ - Task-class choice matters. We pick classes where a skill *could* plausibly help
66
+ (procedural/gotcha-heavy domains); we do not claim uplift on trivia.
README.md ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: TurboSkillSlug
3
+ emoji: 🐌
4
+ colorFrom: purple
5
+ colorTo: yellow
6
+ sdk: gradio
7
+ sdk_version: "6.16.0"
8
+ python_version: "3.12"
9
+ app_file: app.py
10
+ pinned: false
11
+ short_description: Turn a coding session into a skill, recap, and shell.
12
+ tags:
13
+ - hackathon
14
+ - build-small-hackathon
15
+ - track:wood
16
+ - sponsor:openai
17
+ - sponsor:modal
18
+ - achievement:welltuned
19
+ - achievement:offbrand
20
+ - achievement:fieldnotes
21
+ models:
22
+ - legendarydragontamer/slugvoice-qwen2.5-1.5b-lora
23
+ - legendarydragontamer/slugextract-qwen2.5-1.5b-lora
24
+ - Qwen/Qwen2.5-1.5B-Instruct
25
+ - openai/whisper-large-v3-turbo
26
+ datasets:
27
+ - legendarydragontamer/turboskillslug-groundedness-eval
28
+ ---
29
+
30
+ # TurboSkillSlug
31
+
32
+ **Feed it a coding session. Get back a reusable SKILL.md, a grounded spoken
33
+ recap, and a procedural shell that encodes the whole session as art.**
34
+
35
+ The session goes in one of two ways: narrate it aloud, or drop a real Claude Code
36
+ or Codex CLI session log (`.jsonl`). A fine-tuned 1.5B model extracts what you
37
+ tried, what failed, and what finally worked. Total active pipeline: **~2.6B
38
+ parameters**, measured to match a 7B on groundedness at a third the size.
39
+
40
+ Then the slug gives you four things:
41
+
42
+ 1. **A SKILL.md** another LLM can actually use: the non-obvious gotchas
43
+ (symptom → cause → fix), the approaches that fail and why, the breakthrough.
44
+ Built to give a frontier model real uplift, not a session summary.
45
+ 2. **A spoken recap** in a fine-tuned "slug voice," every line grounded in
46
+ something that happened, never reciting invented numbers.
47
+ 3. **A shell** whose spiral, knots, jewels, and colors all derive from your
48
+ session, born on screen as a scroll that unrolls along its own arm, with a
49
+ byōbu-style battle inked across it (dead ends are fallen warriors, the
50
+ breakthrough is a dragon).
51
+ 4. **A receipt** like a thermal printout: approaches tried, dead ends, mood.
52
+
53
+ ### The slug witnesses every kind of session, not just debugging
54
+
55
+ Most coding sessions are not bug hunts. They are exploring an unfamiliar repo,
56
+ writing docs, setting up tooling, building a feature. A witness that only has
57
+ eyes for "what broke and what fixed it" leaves those sessions with a hollow
58
+ shell.
59
+
60
+ So the slug detects the session's genre (debugging, exploration, authoring,
61
+ feature, refactor, setup) and witnesses the right thing for each:
62
+
63
+ - debugging → the struggle: dead ends and the breakthrough
64
+ - exploration → the discoveries: the non-obvious facts learned about the codebase
65
+ - authoring → the decisions, and the false assumptions caught before they became wrong docs
66
+ - feature / refactor / setup → what was built or changed, and what would break if done naively
67
+
68
+ The shell's vocabulary adapts with it: for an exploration session the rim jewels
69
+ are discoveries, the aperture is the clearest insight. Genre detection is pure
70
+ pattern-matching: no model call, no added latency.
71
+
72
+ Why this matters concretely: on a real exploration trace, the slug surfaced that
73
+ a project's checkpoint mirror uses a custom git ref namespace
74
+ (`refs/entire/...`) that a standard `git fetch --all` will miss. That is exactly
75
+ the kind of private, non-derivable knowledge a SKILL.md exists to carry, and it
76
+ came from a session that had no "bug" at all.
77
+
78
+ Every shell is unique because every session is unique.
79
+
80
+ ## Try it in one click
81
+
82
+ Two tabs, two sample inputs:
83
+
84
+ - **narrate aloud** — a sample build session (audio)
85
+ - **drop a session trace** — a sample Claude Code `.jsonl` trace
86
+
87
+ Or bring your own: upload a recording, or drag a real session log from
88
+ `~/.claude/projects/.../*.jsonl` or `~/.codex/sessions/.../*.jsonl`. Judges can
89
+ feed their own agent logs and watch the slug read them.
90
+
91
+ ## Demo
92
+
93
+ Watch the demo: **[youtu.be/qSP9olWRv7o](https://youtu.be/qSP9olWRv7o)**
94
+
95
+ ## Social
96
+
97
+ The launch post: **[x.com/anubhav27071997](https://x.com/anubhav27071997/status/2063970171010826540)**
98
+
99
+ ## Why this is hard the right way
100
+
101
+ The slug's entire promise is a witness that only says what it saw. That makes
102
+ **groundedness** the core engineering problem: a small model that invents facts
103
+ is worthless here. So we measured it, honestly, and published the data.
104
+
105
+ ### Groundedness: does the small model hallucinate more than the 7B it replaced?
106
+
107
+ On 25 held-out transcripts, comparing the shipped fine-tuned 1.5B against the
108
+ Qwen-7B it replaced and its own un-tuned 1.5B base:
109
+
110
+ | system | semantic groundedness | lexical | parse | facts |
111
+ |---|---:|---:|---:|---:|
112
+ | prompted 7B | 0.716 | 0.576 | 24/25 | 272 |
113
+ | prompted 1.5B base | 0.565 | 0.390 | 21/25 | 140 |
114
+ | **fine-tuned 1.5B LoRA** | **0.762** | 0.378 | 21/25 | 195 |
115
+
116
+ The fine-tuned 1.5B **matches and slightly exceeds the 7B (0.76 vs 0.72) at a
117
+ third of the active size.** It does this by paraphrasing rather than copying:
118
+ lowest lexical overlap, highest semantic groundedness, the signature of a model
119
+ that restates meaning instead of echoing words.
120
+
121
+ Reported with its costs, not spun: the LoRA produces valid JSON less often
122
+ (21/25 vs 24/25), and the semantic metric passed 5/6 hand-labeled calibration
123
+ cases (the one miss understates the LoRA, not the reverse). 25 transcripts is a
124
+ small sample, so the honest claim is "matches or slightly exceeds," not "beats."
125
+ Raw generations and per-fact scores are published so anyone can re-score:
126
+ [turboskillslug-groundedness-eval](https://huggingface.co/datasets/legendarydragontamer/turboskillslug-groundedness-eval).
127
+
128
+ ## The SKILL.md is the real gift
129
+
130
+ A skill file is only worth shipping if it helps an LLM that is already capable
131
+ without it. A summary does not. So the SKILL.md is built to carry the
132
+ **non-obvious, transferable** knowledge a frontier model cannot derive on its own:
133
+
134
+ - **Gotchas as symptom → cause → fix**, not labels. "Processing leaf nodes first
135
+ looks natural but breaks because a parent depends on its children being
136
+ finalized; process deepest-first" — not "ordering unclear."
137
+ - **What does NOT work, and why**, so the model skips the dead ends you already
138
+ paid for.
139
+ - **Transferable principles** distilled from the arc, not a diary of the session.
140
+ - **A negative guardrail under each gotcha**, phrased as a "do not X / verify Y
141
+ before assuming" rule. This follows the 2026 RuleShaping finding that negative,
142
+ state-dependent guardrails are the rule type that actually helps a model, and
143
+ it is generated deterministically with no model call.
144
+
145
+ Terse gotchas are expanded by an optional one-shot pass, guarded so example
146
+ phrasing can never leak into output.
147
+
148
+ ## How the shell reads your session
149
+
150
+ | What happened | How the shell shows it |
151
+ |---|---|
152
+ | Duration | Overall size and number of spiral turns |
153
+ | Each approach tried | Spiral arm density |
154
+ | Each dead end | A dark knot — and a fallen warrior in the battle layer |
155
+ | The breakthrough | The glowing aperture at the tip — and a dragon |
156
+ | Gotchas | Iridescent jewels along the rim — and archers |
157
+ | Your emotional arc | Color gradient from start to end |
158
+
159
+ A frustrated session ending in relief is red-to-green. A curious exploration
160
+ ending in delight is warm gold. A long grind is cold blue-grey. The color story
161
+ is the emotional story.
162
+
163
+ Procedural SVG: nacre texture filters, HSL color harmonies, bezier-smoothed
164
+ curves. **No diffusion, no image generation.** Every pixel traces to a real
165
+ session feature, which is the whole point: if a diffusion model drew the shell,
166
+ "this knot is your dead end" would stop being true. The shell is born as a scroll
167
+ unrolling along its spiral arm, led by a 3D paper curl, with the byōbu battle
168
+ inking on as the parchment is laid. There is a "watch it unroll again" replay.
169
+
170
+ Kept shells go into a **shared terrarium** (gallery): a living collection where
171
+ every shell is the fingerprint of a real session, each with a `?shell=<id>`
172
+ permalink.
173
+
174
+ ## Architecture
175
+
176
+ The full primary pipeline runs on Modal at **~2.6B active parameters**. The
177
+ Qwen-7B is a labeled fallback only, not on the primary path.
178
+
179
+ | Component | Model | Params | Infrastructure |
180
+ |---|---|---|---|
181
+ | Transcription | `openai/whisper-large-v3-turbo` | 809M | Modal T4 |
182
+ | Feature extraction | `slugextract-qwen2.5-1.5b-lora` | 1.5B | Modal T4 (shared) |
183
+ | Slug voice | `slugvoice-qwen2.5-1.5b-lora` | 1.5B | Modal T4 (shared) |
184
+ | Spoken recap | Chatterbox TTS | ~300M | Modal A10G |
185
+ | Genre detection | Pattern matching (no model) | 0 | CPU |
186
+ | Shell + Receipt | Procedural SVG (no model) | 0 | CPU |
187
+ | Extraction fallback | `Qwen/Qwen2.5-7B-Instruct` | 7B | HF Inference (fallback only) |
188
+
189
+ **Two custom LoRA adapters, one base model, one T4.** Both adapters are
190
+ fine-tunes of Qwen2.5-1.5B-Instruct, loaded onto a single base on a single T4 and
191
+ switched per request, so the whole language pipeline runs on one GPU.
192
+
193
+ - **SlugVoice** ([adapter](https://huggingface.co/legendarydragontamer/slugvoice-qwen2.5-1.5b-lora)):
194
+ 500 hand-crafted (transcript snippet, slug observation) pairs. Loss 4.97 → 0.89.
195
+ - **SlugExtract** ([adapter](https://huggingface.co/legendarydragontamer/slugextract-qwen2.5-1.5b-lora)):
196
+ 167 balanced (transcript, structured-JSON) pairs across 14 sentiment arcs.
197
+ Loss 1.88 → 0.81. Replaces the Qwen-7B extractor; brings the pipeline to ~2.6B.
198
+
199
+ ## How Modal is used
200
+
201
+ 1. **Fine-tuning.** Both LoRAs trained on Modal (A10G). SlugExtract on a pure
202
+ transformers + PEFT + bitsandbytes stack.
203
+ 2. **Serving.** Whisper on a T4; both LoRAs on a single shared T4 via PEFT
204
+ multi-adapter, switched per request. Kept-warm containers (one always-on plus
205
+ a buffer) for demo reliability.
206
+ 3. **TTS.** Chatterbox on an A10G speaks the recap.
207
+ 4. **Evaluation.** The groundedness eval (three models, 25 transcripts, two
208
+ metrics) runs as a Modal job, persisting raw generations to a Volume.
209
+ 5. **Gallery.** The shared terrarium's save/list/fetch endpoints run on Modal,
210
+ backed by the same Volume.
211
+
212
+ If the primary extraction misses, the app retries on the same small model and
213
+ otherwise degrades to a clear message, so it never crashes mid-render.
214
+
215
+ ## Built with OpenAI Codex
216
+
217
+ Built using [OpenAI Codex](https://openai.com/codex) as the primary coding agent.
218
+ Full commit history:
219
+ **[github.com/AnubhavBharadwaaj/turbo-skill-slug](https://github.com/AnubhavBharadwaaj/turbo-skill-slug)**
220
+
221
+ Codex handled scaffolding, the Gradio skeleton, the shell SVG geometry, tests,
222
+ dependency wiring, and deployment fixes. Human judgment went into the slug's
223
+ voice, the shell's visual design, the grounding constraints, and the two LoRA
224
+ fine-tunes.
225
+
226
+ ## Technical choices and why
227
+
228
+ **Procedural SVG over generated images.** Every element traces to a real feature;
229
+ a diffusion model would break that link.
230
+
231
+ **Fine-tuned 1.5B over prompted 7B for both voice and extraction.** A prompted
232
+ large model copies examples; a fine-tuned small model learns the pattern. The
233
+ eval shows the extraction fine-tune reaching 7B-level groundedness at a third the
234
+ size.
235
+
236
+ **Counts live on the receipt, never in the voice.** The voice is forbidden from
237
+ reciting tallies, so it can never state a number that contradicts the record. The
238
+ slug describes moments; the receipt does arithmetic.
239
+
240
+ **Graceful degradation everywhere.** If the primary Modal extraction misses, the
241
+ app retries once on the same small model, then shows a clear "try again" message
242
+ rather than crashing. Unexpected sentiment → closest valid label. Animation fails
243
+ → the full shell still shows (nothing is hard-hidden). Nothing crashes on edge
244
+ cases.
245
+
246
+ ## Hackathon patches
247
+
248
+ | Patch | Status | Evidence |
249
+ |---|---|---|
250
+ | 🍄 Thousand Token Wood | ✅ | A slug grows a shell from your session |
251
+ | 🏆 Best Use of Codex | ✅ | Codex-attributed commits, documented usage, public repo |
252
+ | 🎨 Off Brand | ✅ | Procedural shell: scroll-unroll birth, byōbu battle layer, thermal receipt |
253
+ | 🎯 Well-Tuned | ✅ | TWO published LoRAs + a published, re-scorable groundedness eval |
254
+ | 🏗️ Best Use of Modal | ✅ | Two fine-tunes, dual-adapter serving, TTS, eval, and gallery on Modal |
255
+ | 📓 Field Notes | ✅ | [Blog article](https://huggingface.co/blog/build-small-hackathon/turboskillslug-shell-from-session) |
256
+
257
+ ## Honesty
258
+
259
+ Full caveats (parse-rate cost, the eval's 5/6 calibration, the animation
260
+ payload, what is freshly built) are documented in
261
+ [HONEST_SUBMISSION.md](https://github.com/AnubhavBharadwaaj/turbo-skill-slug/blob/main/HONEST_SUBMISSION.md).
262
+ Every model and the eval data are published for anyone to verify.
263
+
264
+ ## Research foundation: from one skill to lifecycle-governed rules
265
+
266
+ Beyond the shipped app, this project carries an offline-validated research layer
267
+ that answers a sharper question than "can a small model extract a skill": *when
268
+ does an extracted artifact actually help a capable model, and how should many
269
+ sessions compound into durable knowledge?*
270
+
271
+ This work is validated offline (it is not yet wired into the live Space) and is
272
+ documented and tested in the repo. Stated plainly so the line between shipped and
273
+ researched is clear:
274
+
275
+ - **When skills help (measured).** A blind, calibrated eval (one model answers,
276
+ an independent model judges) found that a generated skill gives a frontier
277
+ model uplift *only* when it carries knowledge that could not be in training
278
+ data: private behavior, post-cutoff facts, project conventions. General
279
+ algorithmic skills gave 0.0 uplift; novel/private ones gave real uplift. The
280
+ dividing line is provenance, not difficulty.
281
+
282
+ - **Compounding across sessions (built, offline-tested).** A promotion engine,
283
+ grounded in the 2026 "Experience Compression Spectrum" framing, consolidates
284
+ gotchas that recur across multiple sessions of the same codebase into compact,
285
+ guardrail-phrased rules, with provenance, confidence, and a validation gate
286
+ that demotes rules that stop holding. Rule phrasing follows the "RuleShaping"
287
+ finding that negative, state-dependent guardrails help where positive
288
+ directives hurt.
289
+
290
+ - **Faithful trace distillation (built, offline-tested).** A from-scratch
291
+ implementation of the 2026 "Trace2Skill" method (validation-gated error
292
+ analysis, hierarchical prevalent-pattern merging, niche items routed to
293
+ references) for higher-fidelity extraction.
294
+
295
+ An honest result from running the extractor on a sample of real public agent
296
+ traces (SALT-NLP/SWE-chat and nebius/SWE-agent-trajectories): its gotchas are
297
+ specific and real (it named exact functions, ref namespaces, and build-tool
298
+ quirks), but cross-session promotion only fires within a single codebase, because
299
+ two different repos never share the same private trap. This was an exploratory
300
+ check on a handful of sessions, not a systematic benchmark, and it surfaced a
301
+ true property of the problem rather than a polished number, reported rather than
302
+ hidden. The loader scripts live in `research/traces/` in the GitHub repo, so the
303
+ check is reproducible from scratch.
304
+
305
+ Full method, code, and tests live in the repo; the research is a foundation for
306
+ where the slug goes next, not a claim about the current Space.
307
+
308
+ ## What comes next
309
+
310
+ 1. **Session diff view.** Upload two sessions, see how the shells differ.
311
+ 2. **Tighter extraction reliability.** Close the parse-rate gap (21/25 vs the
312
+ 7B's 24/25) with more training pairs and constrained decoding.
313
+
314
+ ## Links
315
+
316
+ - **Space:** [build-small-hackathon/TurboSkillSlug](https://huggingface.co/spaces/build-small-hackathon/TurboSkillSlug)
317
+ - **Code:** [github.com/AnubhavBharadwaaj/turbo-skill-slug](https://github.com/AnubhavBharadwaaj/turbo-skill-slug)
318
+ - **SlugVoice LoRA:** [slugvoice-qwen2.5-1.5b-lora](https://huggingface.co/legendarydragontamer/slugvoice-qwen2.5-1.5b-lora)
319
+ - **SlugExtract LoRA:** [slugextract-qwen2.5-1.5b-lora](https://huggingface.co/legendarydragontamer/slugextract-qwen2.5-1.5b-lora)
320
+ - **Groundedness eval:** [turboskillslug-groundedness-eval](https://huggingface.co/datasets/legendarydragontamer/turboskillslug-groundedness-eval)
321
+ - **Demo:** [youtu.be/qSP9olWRv7o](https://youtu.be/qSP9olWRv7o)
322
+ - **Social post:** [x.com/anubhav27071997](https://x.com/anubhav27071997/status/2063970171010826540)
323
+ - **Blog:** [turboskillslug-shell-from-session](https://huggingface.co/blog/build-small-hackathon/turboskillslug-shell-from-session)
324
+
325
+ ---
326
+
327
+ *The slug watches, gives its gifts, and goes back to sleep.*
328
+ *I was here.*
anim_diagnostic.py ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Animation diagnostic for the shell iframe.
3
+
4
+ Embeds a self-reporting probe inside the iframe srcdoc that:
5
+ 1. Confirms the iframe document actually rendered (suspect #3)
6
+ 2. Counts <animate>/<animateTransform> elements present in the DOM (suspect #2:
7
+ did Gradio strip them before they reached the iframe?)
8
+ 3. Uses the SVG SMIL API (svg.getCurrentTime / elem.getStartTime) to check
9
+ whether the animations are actually RUNNING, not just present (suspect #4)
10
+ 4. Paints a visible badge in the corner with the verdict, so you SEE the answer
11
+ on the live Space without opening devtools.
12
+
13
+ This makes the failure point unambiguous:
14
+ - Badge never appears -> iframe didn't render (suspect #3) OR Gradio
15
+ didn't even pass the iframe through
16
+ - Badge: "0 anim elements" -> SMIL stripped before/at the iframe (suspect #2)
17
+ - Badge: "N present, FROZEN" -> SMIL present but not playing (suspect #4: sandbox)
18
+ - Badge: "N present, RUNNING" -> animation IS working; the issue is it is too
19
+ fast/subtle to see, or you looked after it froze
20
+ """
21
+
22
+ PROBE_JS = """
23
+ <script>
24
+ (function(){
25
+ function badge(text, color){
26
+ var d = document.createElement('div');
27
+ d.textContent = text;
28
+ d.style.cssText = 'position:fixed;left:6px;bottom:6px;z-index:99999;'
29
+ + 'font:11px monospace;padding:4px 7px;border-radius:4px;'
30
+ + 'background:'+color+';color:#fff;opacity:0.92;pointer-events:none;'
31
+ + 'white-space:nowrap;';
32
+ document.body.appendChild(d);
33
+ }
34
+ function run(){
35
+ var svg = document.querySelector('svg');
36
+ if(!svg){ badge('NO SVG in iframe', '#b00'); return; }
37
+ var anims = svg.querySelectorAll('animate, animateTransform, animateMotion');
38
+ var n = anims.length;
39
+ if(n === 0){ badge('SMIL STRIPPED: 0 anim elements', '#b00'); return; }
40
+ // Check if SMIL is actually advancing time.
41
+ var t0 = 0;
42
+ try { t0 = svg.getCurrentTime(); } catch(e){
43
+ badge(n+' present, but getCurrentTime() FAILED (SMIL unsupported)', '#b06000');
44
+ return;
45
+ }
46
+ // Sample again shortly after; if currentTime advanced, SMIL is running.
47
+ setTimeout(function(){
48
+ var t1 = 0;
49
+ try { t1 = svg.getCurrentTime(); } catch(e){}
50
+ if(t1 > t0 + 0.05){
51
+ badge(n+' anim elements, RUNNING (t='+t1.toFixed(2)+'s)', '#0a7d2c');
52
+ } else {
53
+ badge(n+' present, FROZEN (SMIL not advancing - sandbox?)', '#b06000');
54
+ }
55
+ }, 350);
56
+ }
57
+ if(document.readyState === 'loading'){
58
+ document.addEventListener('DOMContentLoaded', run);
59
+ } else { run(); }
60
+ })();
61
+ </script>
62
+ """
63
+
64
+
65
+ def inject_probe(iframe_inner_html: str) -> str:
66
+ """Insert the probe <script> just before </body> inside the iframe document.
67
+
68
+ iframe_inner_html is the FULL html doc string that goes into srcdoc (before
69
+ escaping). The probe runs inside the iframe's own document, where SMIL lives.
70
+ """
71
+ if "</body>" in iframe_inner_html:
72
+ return iframe_inner_html.replace("</body>", PROBE_JS + "</body>", 1)
73
+ return iframe_inner_html + PROBE_JS
app.py ADDED
@@ -0,0 +1,647 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """TurboSkillSlug Gradio application."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import base64 as b64lib
6
+ import html
7
+ import json
8
+ import os
9
+ import shutil
10
+ import tempfile
11
+ import time
12
+ import wave
13
+ from pathlib import Path
14
+ from typing import Any
15
+
16
+ import gradio as gr
17
+ import httpx
18
+
19
+ from extract import extract_session
20
+ from receipt import generate_receipt_svg
21
+ from shell import generate_shell_svg
22
+ from shell_animate import wrap_in_iframe, animate_shell_svg
23
+ from shell_unroll import build_unroll_doc, N_STAGES
24
+ from gallery_client import save_shell, list_shells, get_shell
25
+ from battle_trace import render_battle_trace
26
+ from shell3d_lens import render_shell_3d
27
+ from shell_animate import wrap_in_iframe as _wrap_iframe
28
+ from transcribe import transcribe_audio
29
+ from trace_parser import parse_trace_to_transcript, detect_trace_format
30
+ from model_guard import APP_RUNTIME
31
+
32
+ # REQ-01 guard: mark this process as the live app. From here on, any attempt to
33
+ # call an over-32B model (via assert_small_model) will raise instead of shipping
34
+ # a compliance violation. Offline eval/research scripts never enable this flag.
35
+ APP_RUNTIME.enable()
36
+
37
+
38
+ # Growth stages for the live shell reveal. The slug grows the shell as it
39
+ # watches: spiral lengthens, knots and jewels form in order, aperture opens last.
40
+ GROWTH_STAGES = [0.18, 0.38, 0.58, 0.78, 1.0]
41
+ GROWTH_PACING_SECONDS = 0.55 # brief pause between stages so the eye sees growth
42
+
43
+ TTS_URL = os.environ.get(
44
+ "MODAL_TTS_URL",
45
+ "https://anubhavbharadwaaj--slugvoice-tts-slugtts-api.modal.run",
46
+ )
47
+ SAMPLE_WAV_NAME = "sample_session.wav"
48
+ SAMPLE_WAV_PATH = Path(__file__).parent / SAMPLE_WAV_NAME
49
+
50
+
51
+ def _resolve_audio_path(audio: str | None) -> str | None:
52
+ """Return a readable audio path for the pipeline.
53
+
54
+ Gradio example uploads can point at evictable temp files on HF Spaces. If the
55
+ one-click sample's temp copy is gone, recover from the committed repo copy
56
+ by copying it to a fresh temp file owned by this request.
57
+ """
58
+ if audio and os.path.exists(audio):
59
+ return audio
60
+
61
+ looks_like_sample = audio and SAMPLE_WAV_NAME in os.path.basename(audio)
62
+ if (looks_like_sample or audio is None) and SAMPLE_WAV_PATH.exists():
63
+ dst = Path(tempfile.mkdtemp(prefix="slug_sample_")) / SAMPLE_WAV_NAME
64
+ shutil.copy(SAMPLE_WAV_PATH, dst)
65
+ print(f"[SAMPLE] recovered evicted sample from committed repo copy: {dst}")
66
+ return str(dst)
67
+
68
+ return audio
69
+
70
+
71
+ def _base_url_from_request(request) -> str:
72
+ """Reconstruct the app's public base URL (with trailing /) from a gr.Request.
73
+
74
+ On HF Spaces the app sits behind a reverse proxy, so the real public host is
75
+ in x-forwarded-host (not request.url's internal host). We prefer the
76
+ forwarded headers and fall back to request.headers['host'] / request.url.
77
+ """
78
+ try:
79
+ headers = {k.lower(): v for k, v in dict(request.headers).items()}
80
+ except Exception:
81
+ headers = {}
82
+ host = headers.get("x-forwarded-host") or headers.get("host")
83
+ proto = headers.get("x-forwarded-proto") or "https"
84
+ if host:
85
+ return f"{proto}://{host}/"
86
+ # last-resort fallback to the request URL's origin
87
+ try:
88
+ u = str(request.url)
89
+ # strip any path/query
90
+ from urllib.parse import urlparse
91
+ p = urlparse(u)
92
+ if p.scheme and p.netloc:
93
+ return f"{p.scheme}://{p.netloc}/"
94
+ except Exception:
95
+ pass
96
+ return "/"
97
+
98
+
99
+ def _speak_recap(slug_lines: list[str]) -> str | None:
100
+ """Convert slug recap to speech via Chatterbox on Modal."""
101
+ try:
102
+ full_text = ". ".join(slug_lines) + ". I was here."
103
+ resp = httpx.post(TTS_URL, json={"text": full_text}, timeout=180)
104
+ resp.raise_for_status()
105
+ audio_b64 = resp.json().get("audio", "")
106
+ if audio_b64:
107
+ audio_bytes = b64lib.b64decode(audio_b64)
108
+ tmp = Path(tempfile.mkdtemp()) / "slug_speaks.wav"
109
+ tmp.write_bytes(audio_bytes)
110
+ return str(tmp)
111
+ except Exception as e:
112
+ print(f"TTS failed: {e}")
113
+ return None
114
+
115
+
116
+ def _get_audio_duration_minutes(file_path: str) -> float:
117
+ """Get actual audio duration in minutes from the file itself."""
118
+ try:
119
+ with wave.open(file_path, "r") as f:
120
+ return (f.getnframes() / f.getframerate()) / 60
121
+ except Exception:
122
+ try:
123
+ size = os.path.getsize(file_path)
124
+ return round(size / (32000 * 60), 1)
125
+ except Exception:
126
+ return 1.0
127
+
128
+
129
+ def _format_slug_recap(extraction: dict[str, Any]) -> str:
130
+ """Format the slug's witness recap for display."""
131
+ slug_voice = extraction.get("slug_voice", [])
132
+ utterances = "\n\n".join(f"*{u}*" for u in slug_voice)
133
+
134
+ # The slug's closing signature
135
+ utterances += "\n\n*I was here.*"
136
+
137
+ themes = ", ".join(extraction.get("themes", []))
138
+ duration = extraction.get("duration_minutes", 0)
139
+
140
+ return (
141
+ "## what the slug witnessed\n\n"
142
+ f"{utterances}\n\n"
143
+ f"**Duration:** {duration} minutes\n\n"
144
+ f"**Themes:** {themes}\n"
145
+ )
146
+
147
+
148
+ def _format_shell_legend(extraction: dict[str, Any]) -> str:
149
+ """Return a compact genre-aware legend for the shell display."""
150
+ genre = str(extraction.get("genre") or "session")
151
+ legend = extraction.get("shell_legend") or {}
152
+ if not isinstance(legend, dict) or not legend:
153
+ return ""
154
+
155
+ knot = html.escape(str(legend.get("knot", "dead ends")))
156
+ jewel = html.escape(str(legend.get("jewel", "gotchas")))
157
+ aperture = html.escape(str(legend.get("aperture", "breakthrough")))
158
+ genre_label = html.escape(genre.replace("_", " "))
159
+ return (
160
+ '<div style="font: 13px system-ui, sans-serif; '
161
+ 'color: #4d4637 !important; background: #fff8e8 !important; '
162
+ 'border: 1px solid #c8a24c; border-radius: 8px; '
163
+ 'padding: 10px 12px; margin: 8px 0 12px;">'
164
+ f'<strong>Shell legend ({genre_label})</strong>: knots = {knot}; '
165
+ f'jewels = {jewel}; aperture = {aperture}.'
166
+ "</div>"
167
+ )
168
+
169
+
170
+ def on_audio_change(audio: str | None):
171
+ """Fired the moment a file is selected/recorded/cleared.
172
+
173
+ Disables the button and shows a 'preparing' state during the window
174
+ after upload but before the user can meaningfully click submit, so a
175
+ premature click is impossible and there is always visible feedback.
176
+ """
177
+ if audio is None:
178
+ return (
179
+ gr.update(value="🐌 give it to the slug", interactive=False),
180
+ gr.update(value=""),
181
+ )
182
+ return (
183
+ gr.update(value="🐌 give it to the slug", interactive=True),
184
+ gr.update(value="*The slug is ready. Hand it your session.*"),
185
+ )
186
+
187
+
188
+ def _empty_outputs(message: str):
189
+ """Twelve-tuple matching the output components, for early/error yields."""
190
+ return (message, "", "", "", "", None, None, None, None, None, None, None)
191
+
192
+
193
+ def _finalize_outputs(transcript_display: str, extraction: dict, slug_audio_path):
194
+ """Build the full ten-tuple of outputs once the shell is fully grown.
195
+
196
+ Shared by the audio and trace paths so both render identically.
197
+ """
198
+ raw_json = json.dumps(extraction, indent=2)
199
+ # The full static shell is what we save for download (clean, portable).
200
+ shell_svg = generate_shell_svg(extraction, growth=1.0)
201
+ # For the live display, build the scroll-unroll: a flipbook of growth stages
202
+ # (each truncates the spiral ALONG its arm) led by a 3D paper curl riding the
203
+ # tip. This lays the parchment down along the spiral path, not radially.
204
+ try:
205
+ stages = [
206
+ generate_shell_svg(extraction, growth=(i + 1) / N_STAGES)
207
+ for i in range(N_STAGES)
208
+ ]
209
+ unroll_doc = build_unroll_doc(stages)
210
+ shell_display = wrap_in_iframe(unroll_doc, height=660)
211
+ except Exception:
212
+ # Never let the animation break the result: fall back to the static shell.
213
+ shell_display = wrap_in_iframe(shell_svg, height=660)
214
+ shell_display = _format_shell_legend(extraction) + shell_display
215
+ receipt_svg = generate_receipt_svg(extraction)
216
+
217
+ tmp_dir = Path(tempfile.mkdtemp(prefix="slug_"))
218
+ svg_path = tmp_dir / "shell.svg"
219
+ svg_path.write_text(shell_svg)
220
+ receipt_path = tmp_dir / "receipt.svg"
221
+ receipt_path.write_text(receipt_svg)
222
+ skill_path = tmp_dir / "skill.md"
223
+ skill_path.write_text(extraction.get("skill_md", ""))
224
+ recap_lines = list(extraction.get("slug_voice", [])) + ["I was here."]
225
+ recap_path = tmp_dir / "slug_recap.txt"
226
+ recap_path.write_text("\n\n".join(recap_lines))
227
+
228
+ return (
229
+ transcript_display,
230
+ _format_slug_recap(extraction),
231
+ shell_display,
232
+ receipt_svg,
233
+ raw_json,
234
+ slug_audio_path,
235
+ str(svg_path),
236
+ str(receipt_path),
237
+ str(skill_path),
238
+ str(recap_path),
239
+ shell_svg, # static SVG, for the "keep this shell" gallery save
240
+ extraction, # metadata, for the gallery entry
241
+ )
242
+
243
+
244
+ def _grow_shell_stages(extraction: dict, transcript_display: str):
245
+ """Yield a single 'shaping' status while the browser-played birth animation
246
+ is prepared. The smooth growth now happens in the browser (shards / draw /
247
+ glass), so we no longer stream mechanical server-side frames. One gentle
248
+ status, then the caller yields the finished animated shell.
249
+ """
250
+ yield (
251
+ "*The slug is shaping your shell...*",
252
+ "*the shell is coming into being...*",
253
+ "", "", "", None, None, None, None, None, None, None,
254
+ )
255
+
256
+
257
+ def process_audio(audio: str | None):
258
+ """Transcribe, extract, then GROW the shell live, yielding each stage."""
259
+ audio = _resolve_audio_path(audio)
260
+ if audio is None or not os.path.exists(audio):
261
+ yield _empty_outputs(
262
+ "The slug couldn't find that audio file. On HF Spaces an upload can "
263
+ "expire; please re-upload and try again."
264
+ )
265
+ return
266
+
267
+ try:
268
+ transcript = transcribe_audio(audio)
269
+ extraction = extract_session(transcript)
270
+ except Exception as e:
271
+ print(f"[PIPELINE] audio path failed: {type(e).__name__}: {e}")
272
+ yield _empty_outputs(
273
+ "The slug couldn't read this session right now — the extractor is "
274
+ "temporarily unavailable. Please try again, or use the trace tab."
275
+ )
276
+ return
277
+ slug_audio_path = _speak_recap(extraction.get("slug_voice", []))
278
+
279
+ audio_minutes = round(_get_audio_duration_minutes(audio), 1)
280
+ model_minutes = extraction.get("duration_minutes", 0)
281
+ try:
282
+ model_minutes = float(model_minutes)
283
+ except (TypeError, ValueError):
284
+ model_minutes = 0.0
285
+ extraction["duration_minutes"] = round(max(audio_minutes, model_minutes), 1)
286
+
287
+ transcript_display = f"## Transcript\n\n{transcript}"
288
+ yield from _grow_shell_stages(extraction, transcript_display)
289
+ yield _finalize_outputs(transcript_display, extraction, slug_audio_path)
290
+
291
+
292
+
293
+ def process_trace(trace_file: str | None):
294
+ """Parse an agent session JSONL trace, then GROW the shell live.
295
+
296
+ A Claude Code or Codex CLI session log becomes witness testimony the slug
297
+ reads exactly like a spoken transcript. This is the input judges can feed
298
+ from their own machines.
299
+ """
300
+ if trace_file is None:
301
+ yield _empty_outputs("Drop a Claude Code or Codex session trace first.")
302
+ return
303
+
304
+ try:
305
+ with open(trace_file, "r", encoding="utf-8", errors="replace") as f:
306
+ jsonl_string = f.read()
307
+ except Exception as e:
308
+ yield _empty_outputs(f"Could not read the trace: {e}")
309
+ return
310
+
311
+ source = detect_trace_format(jsonl_string)
312
+ transcript = parse_trace_to_transcript(jsonl_string)
313
+ if not transcript:
314
+ yield _empty_outputs("The slug could not find a session in that file.")
315
+ return
316
+
317
+ extraction = extract_session(transcript)
318
+ slug_audio_path = _speak_recap(extraction.get("slug_voice", []))
319
+
320
+ # Duration: no audio file, so trust the model's estimate.
321
+ model_minutes = extraction.get("duration_minutes", 0)
322
+ try:
323
+ model_minutes = float(model_minutes)
324
+ except (TypeError, ValueError):
325
+ model_minutes = 0.0
326
+ extraction["duration_minutes"] = round(model_minutes, 1) if model_minutes > 0 else 1.0
327
+
328
+ label = {"claude": "Claude Code", "codex": "Codex CLI"}.get(source, "agent")
329
+ transcript_display = (
330
+ f"## Session trace ({label})\n\n"
331
+ f"The slug read your {label} session and witnessed this:\n\n{transcript}"
332
+ )
333
+
334
+ yield from _grow_shell_stages(extraction, transcript_display)
335
+ yield _finalize_outputs(transcript_display, extraction, slug_audio_path)
336
+
337
+
338
+ def build_interface() -> gr.Blocks:
339
+ """Build and return the TurboSkillSlug Gradio Blocks interface."""
340
+ with gr.Blocks(title="TurboSkillSlug") as demo:
341
+ gr.Markdown(
342
+ "# 🐌 TurboSkillSlug\n\n"
343
+ "*A small, slow companion who watches you build.*\n\n"
344
+ "Give the slug a build session: narrate it aloud, or drop an agent "
345
+ "session trace. It watches, then grows you a shell, a recap, a "
346
+ "SKILL.md, and a receipt."
347
+ )
348
+
349
+ with gr.Row():
350
+ with gr.Column(scale=1):
351
+ with gr.Tabs():
352
+ with gr.Tab("narrate aloud"):
353
+ audio_input = gr.Audio(
354
+ sources=["upload", "microphone"],
355
+ type="filepath",
356
+ label="your build session",
357
+ )
358
+ gr.Examples(
359
+ examples=[str(SAMPLE_WAV_PATH)],
360
+ inputs=audio_input,
361
+ label="or try a sample session",
362
+ )
363
+ submit_button = gr.Button(
364
+ "🐌 give it to the slug", variant="primary",
365
+ interactive=False,
366
+ )
367
+ with gr.Tab("drop a session trace"):
368
+ gr.Markdown(
369
+ "Drag a **Claude Code** "
370
+ "(`~/.claude/projects/.../*.jsonl`) or **Codex CLI** "
371
+ "(`~/.codex/sessions/.../*.jsonl`) session log. "
372
+ "The slug reads your actual work, no narration needed."
373
+ )
374
+ trace_input = gr.File(
375
+ label="session trace (.jsonl)",
376
+ file_types=[".jsonl"],
377
+ type="filepath",
378
+ )
379
+ gr.Examples(
380
+ examples=["sample_trace.jsonl"],
381
+ inputs=trace_input,
382
+ label="or try a sample trace",
383
+ )
384
+ trace_button = gr.Button(
385
+ "🐌 let the slug read it", variant="primary",
386
+ )
387
+ with gr.Column(scale=1):
388
+ status_line = gr.Markdown("")
389
+
390
+ # ---- THE SHELL, front and center, right under the inputs ----
391
+ # This is what plays during generation. Everything else stays hidden
392
+ # until the shell has finished being born.
393
+ shell_output = gr.HTML(label="your shell")
394
+
395
+ # State holders for the current shell (static SVG + extraction), used by
396
+ # the "keep this shell" gallery save and the experimental battle replay.
397
+ cur_shell_svg = gr.State(None)
398
+ cur_extraction = gr.State(None)
399
+
400
+ # ---- EVERYTHING ELSE, hidden until the shell finishes ----
401
+ with gr.Group(visible=False) as gifts_group:
402
+ gr.Markdown("### the slug's other gifts")
403
+ with gr.Row():
404
+ keep_button = gr.Button("🐚 keep this shell in the terrarium",
405
+ variant="secondary")
406
+ keep_status = gr.Markdown("")
407
+ recap_output = gr.Markdown(label="slug recap")
408
+ slug_audio = gr.Audio(label="the slug speaks", type="filepath")
409
+ receipt_output = gr.HTML(label="your receipt")
410
+ transcript_output = gr.Markdown(label="transcript")
411
+ raw_json_output = gr.Code(label="Raw JSON", language="json")
412
+ with gr.Row():
413
+ svg_download = gr.File(label="shell.svg")
414
+ receipt_download = gr.File(label="receipt.svg")
415
+ skill_download = gr.File(label="skill.md")
416
+ recap_download = gr.File(label="slug_recap.txt")
417
+
418
+ # Experimental: the LITERAL temporal replay of the session as a war
419
+ # between you (the Agent) and the Environment. The shell remembers
420
+ # the campaign as a frozen folding screen; this replays it in time.
421
+ with gr.Accordion("⚔ the battle, as it happened (replay)", open=False):
422
+ gr.Markdown(
423
+ "The shell is how the slug *remembers* the battle: the "
424
+ "aftermath, frozen like a folding screen, where the dead "
425
+ "ends fell and where the breakthrough struck. This is the "
426
+ "*replay* of that same battle as it actually happened in "
427
+ "time, your moves against the Environment's strikes (dead "
428
+ "ends), the clashes (gotchas), and the blow that finally "
429
+ "lands (the breakthrough). Memory above; footage here."
430
+ )
431
+ battle_view = gr.HTML()
432
+ battle_button = gr.Button("⚔ replay the battle the slug remembers")
433
+
434
+ def _show_battle(extraction):
435
+ if not extraction:
436
+ return gr.update(value="*The slug has no battle to replay yet.*")
437
+ return gr.update(value=render_battle_trace(extraction, height=440))
438
+
439
+ battle_button.click(
440
+ fn=_show_battle,
441
+ inputs=[cur_extraction],
442
+ outputs=[battle_view],
443
+ )
444
+
445
+ # The shell as a real 3D object you can turn in the light. Same session
446
+ # data, a different lens: the SceneGraph drives a Three.js nautilus with
447
+ # iridescent nacre. First of the planned multi-lens renderers.
448
+ with gr.Accordion("🔮 turn the shell in 3D (experimental lens)", open=False):
449
+ gr.Markdown(
450
+ "The same shell, rendered as a real object you can orbit. The "
451
+ "spiral's growth, the knots (dead ends), the glowing aperture "
452
+ "(breakthrough), and the colour arc all come from your session, "
453
+ "now with true iridescent nacre. Drag to turn it; scroll to zoom."
454
+ )
455
+ shell3d_view = gr.HTML()
456
+ shell3d_button = gr.Button("🔮 see this shell in 3D")
457
+
458
+ def _show_shell3d(extraction):
459
+ if not extraction:
460
+ return gr.update(value="*No shell to render in 3D yet.*")
461
+ html = render_shell_3d(extraction)
462
+ if not html:
463
+ return gr.update(value="*The 3D lens is unavailable right now.*")
464
+ return gr.update(value=html)
465
+
466
+ shell3d_button.click(
467
+ fn=_show_shell3d,
468
+ inputs=[cur_extraction],
469
+ outputs=[shell3d_view],
470
+ )
471
+
472
+ # Enable the button only once audio is actually present
473
+ audio_input.change(
474
+ fn=on_audio_change,
475
+ inputs=audio_input,
476
+ outputs=[submit_button, status_line],
477
+ )
478
+
479
+ # The 12 pipeline outputs (10 UI + 2 state for the gallery save)
480
+ pipeline_outputs = [
481
+ transcript_output,
482
+ recap_output,
483
+ shell_output,
484
+ receipt_output,
485
+ raw_json_output,
486
+ slug_audio,
487
+ svg_download,
488
+ receipt_download,
489
+ skill_download,
490
+ recap_download,
491
+ cur_shell_svg,
492
+ cur_extraction,
493
+ ]
494
+
495
+ def _hide_gifts():
496
+ # On start: hide the gifts, lock the button, set a watching status
497
+ return (
498
+ gr.update(visible=False),
499
+ gr.update(value="🐌 the slug is watching…", interactive=False),
500
+ gr.update(value="*The slug is listening, then it will grow your shell...*"),
501
+ )
502
+
503
+ def _reveal_gifts():
504
+ # On finish: reveal the gifts, reset the button
505
+ return (
506
+ gr.update(visible=True),
507
+ gr.update(value="🐌 give it to the slug", interactive=True),
508
+ gr.update(value="*The slug has finished. Your gifts are below.*"),
509
+ )
510
+
511
+ # Audio path
512
+ submit_button.click(
513
+ fn=_hide_gifts,
514
+ inputs=None,
515
+ outputs=[gifts_group, submit_button, status_line],
516
+ ).then(
517
+ fn=process_audio,
518
+ inputs=audio_input,
519
+ outputs=pipeline_outputs,
520
+ ).then(
521
+ fn=_reveal_gifts,
522
+ inputs=None,
523
+ outputs=[gifts_group, submit_button, status_line],
524
+ )
525
+
526
+ # Trace path
527
+ trace_button.click(
528
+ fn=_hide_gifts,
529
+ inputs=None,
530
+ outputs=[gifts_group, submit_button, status_line],
531
+ ).then(
532
+ fn=process_trace,
533
+ inputs=trace_input,
534
+ outputs=pipeline_outputs,
535
+ ).then(
536
+ fn=_reveal_gifts,
537
+ inputs=None,
538
+ outputs=[gifts_group, submit_button, status_line],
539
+ )
540
+
541
+ # ---- Stage B: save the current shell to the shared terrarium ----
542
+ def _keep_shell(svg, extraction, request: gr.Request):
543
+ if not svg or not extraction:
544
+ return gr.update(value="*No shell to keep yet.*")
545
+ sid = save_shell(svg, extraction)
546
+ if not sid:
547
+ return gr.update(value="*The terrarium was unreachable; shell not saved.*")
548
+ # Build a COMPLETE clickable link (not a bare ?shell= fragment, which
549
+ # is confusing). Derive the base URL from the request's headers.
550
+ base = _base_url_from_request(request)
551
+ link = f"{base}?shell={sid}"
552
+ return gr.update(
553
+ value=(
554
+ f"🐚 **Kept in the terrarium.** Here's the shareable link to "
555
+ f"this exact shell:\n\n[{link}]({link})\n\n"
556
+ f"_Anyone who opens it sees this shell unroll. You can also "
557
+ f"browse all kept shells in the terrarium below._"
558
+ )
559
+ )
560
+
561
+ keep_button.click(
562
+ fn=_keep_shell,
563
+ inputs=[cur_shell_svg, cur_extraction],
564
+ outputs=[keep_status],
565
+ )
566
+
567
+ # ---- Stage C: the shared gallery (a living terrarium of kept shells) ----
568
+ def _render_gallery(request: gr.Request):
569
+ shells = list_shells(limit=60)
570
+ if not shells:
571
+ return ("*The terrarium is empty so far. Grow a shell and keep it "
572
+ "to be the first.*")
573
+ base = _base_url_from_request(request)
574
+ # build a responsive grid of shells; each loads its SVG by id
575
+ cards = []
576
+ for e in shells:
577
+ sid = e.get("id")
578
+ title = e.get("title") or "a session"
579
+ dur = e.get("duration")
580
+ arc = f"{e.get('start','?')} → {e.get('end','?')}"
581
+ sub = f"{dur}m · {arc}" if dur else arc
582
+ data = get_shell(sid) if sid else None
583
+ svg = (data or {}).get("svg", "")
584
+ if not svg:
585
+ continue
586
+ # shrink each shell into a card; static, no replay button (these
587
+ # are kept shells, like photos in an album — the replay belongs
588
+ # to the focused permalink view below).
589
+ frame = _wrap_iframe(svg, height=240, replay=False)
590
+ link = f"{base}?shell={sid}"
591
+ cards.append(
592
+ f'<div style="flex:0 0 250px;margin:8px;text-align:center;">'
593
+ f'<div style="font:600 13px Georgia,serif;color:#c8a24c;">{title}</div>'
594
+ f'<div style="font:11px monospace;color:#999;margin-bottom:4px;">{sub}</div>'
595
+ f'{frame}'
596
+ f'<div style="margin-top:4px;"><a href="{link}" target="_blank" '
597
+ f'style="font:12px Georgia,serif;color:#6ee7ff;text-decoration:none;">'
598
+ f'open this shell →</a></div>'
599
+ f'</div>'
600
+ )
601
+ grid = (
602
+ '<div style="display:flex;flex-wrap:wrap;justify-content:center;">'
603
+ + "".join(cards) + "</div>"
604
+ )
605
+ return grid
606
+
607
+ def _load_permalink(request: gr.Request):
608
+ """If the URL has ?shell=<id>, load that single shell on page open."""
609
+ try:
610
+ sid = dict(request.query_params).get("shell")
611
+ except Exception:
612
+ sid = None
613
+ if not sid:
614
+ return gr.update()
615
+ data = get_shell(sid)
616
+ if not data or not data.get("svg"):
617
+ return gr.update(value="*That shell could not be found in the terrarium.*")
618
+ # Re-animate the saved static shell for the focused permalink view:
619
+ # the mask-based scroll-unroll works on a single SVG (no growth
620
+ # stages needed), so this shell unrolls and the replay button works.
621
+ try:
622
+ animated = animate_shell_svg(data["svg"])
623
+ return gr.update(value=wrap_in_iframe(animated, height=520, replay=True))
624
+ except Exception:
625
+ return gr.update(value=_wrap_iframe(data["svg"], height=520, replay=False))
626
+
627
+ with gr.Accordion("🌿 the terrarium (shared gallery)", open=False):
628
+ gr.Markdown(
629
+ "A shared collection of shells people have kept. Each one is the "
630
+ "fingerprint of a real session. Open `?shell=<id>` to link a "
631
+ "specific shell."
632
+ )
633
+ permalink_view = gr.HTML()
634
+ refresh_gallery = gr.Button("↻ refresh the terrarium")
635
+ gallery_grid = gr.HTML()
636
+ refresh_gallery.click(fn=_render_gallery, inputs=None, outputs=[gallery_grid])
637
+ # populate on load + handle ?shell= permalink
638
+ demo.load(fn=_render_gallery, inputs=None, outputs=[gallery_grid])
639
+ demo.load(fn=_load_permalink, inputs=None, outputs=[permalink_view])
640
+
641
+ return demo
642
+
643
+
644
+ interface = build_interface()
645
+
646
+ if __name__ == "__main__":
647
+ interface.launch()
battle_trace.py ADDED
@@ -0,0 +1,281 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Battle Trace: an experimental temporal replay of a session as a war between the
3
+ Agent and the Environment.
4
+
5
+ This is the LITERAL, time-based telling of the session — complementary to the
6
+ shell's byobu battle layer, which is the frozen, artistic telling. The shell
7
+ remembers the campaign as a folding screen; Battle Trace replays it as it
8
+ happened.
9
+
10
+ It reuses the SAME structured extraction the rest of the app already produces
11
+ (approaches, dead_ends, breakthroughs, gotchas, sentiment), so there is no
12
+ second parser and no new model call. Events map to combat:
13
+
14
+ approach tried -> the Agent makes a move (cmd)
15
+ dead end -> the Environment strikes (err): Agent resolve drops
16
+ gotcha -> a clash of blades (clash)
17
+ breakthrough -> a blow lands true (win): Environment resistance drops
18
+ final breakthrough -> the battle resolves (done)
19
+
20
+ The renderer is self-contained Canvas 2D (fine for session-sized traces),
21
+ sandboxed inside an iframe so it can never affect the main pipeline.
22
+ """
23
+
24
+ from __future__ import annotations
25
+
26
+ import html
27
+ import json
28
+
29
+
30
+ def extraction_to_events(extraction: dict) -> list[dict]:
31
+ """Convert a session extraction into an ordered Battle Trace event stream.
32
+
33
+ Events are ordered by their position along the session (0..1) where we have
34
+ it (dead ends, breakthroughs), and interleaved sensibly otherwise.
35
+ """
36
+ events: list[dict] = []
37
+
38
+ approaches = extraction.get("approaches_tried", []) or []
39
+ dead_ends = extraction.get("dead_ends", []) or []
40
+ breakthroughs = extraction.get("breakthroughs", []) or []
41
+ gotchas = extraction.get("gotchas", []) or []
42
+
43
+ # approaches kick off near the start, spaced across the first half
44
+ for i, a in enumerate(approaches):
45
+ if not isinstance(a, dict):
46
+ continue
47
+ pos = 0.05 + (i / max(1, len(approaches))) * 0.4
48
+ label = str(a.get("approach", "a move"))[:72]
49
+ events.append({"pos": pos, "side": "A", "kind": "cmd", "label": label})
50
+
51
+ # dead ends at their real positions — the Environment strikes
52
+ for d in dead_ends:
53
+ if not isinstance(d, dict):
54
+ continue
55
+ pos = float(d.get("position", 0.5))
56
+ label = str(d.get("what_happened", "a wall"))[:72]
57
+ events.append({"pos": pos, "side": "E", "kind": "err", "label": label})
58
+
59
+ # gotchas as clashes, spread through the middle
60
+ for i, g in enumerate(gotchas):
61
+ pos = 0.3 + (i / max(1, len(gotchas))) * 0.5
62
+ label = str(g)[:72]
63
+ events.append({"pos": pos, "side": "A", "kind": "clash", "label": label})
64
+
65
+ # breakthroughs at their positions — blows land
66
+ for i, b in enumerate(breakthroughs):
67
+ if not isinstance(b, dict):
68
+ continue
69
+ pos = float(b.get("position", 0.85))
70
+ label = str(b.get("what_worked", "a breakthrough"))[:72]
71
+ kind = "done" if i == len(breakthroughs) - 1 else "win"
72
+ events.append({"pos": pos, "side": "A", "kind": kind, "label": label})
73
+
74
+ events.sort(key=lambda e: e["pos"])
75
+ # assign a monotonic time axis from positions
76
+ for i, e in enumerate(events):
77
+ e["t"] = round(1.0 + e["pos"] * 12.0, 2)
78
+ if not events:
79
+ events = [{"pos": 0.5, "side": "A", "kind": "info", "label": "a quiet session", "t": 6.0}]
80
+ # ensure the last event resolves the battle
81
+ if events[-1]["kind"] not in ("done", "err"):
82
+ events[-1]["kind"] = "done"
83
+ return events
84
+
85
+
86
+ # The renderer: a trimmed, self-contained Canvas 2D battle. No external deps.
87
+ # Events are injected as JSON; everything else is static.
88
+ _BATTLE_HTML = r"""<!DOCTYPE html><html><head><meta charset="utf-8"/>
89
+ <style>
90
+ :root{--bg:#0a0e14;--ink:#e6edf3;--muted:#8b98a9;--agent:#6ee7ff;--env:#ff7a59;
91
+ --ok:#7ee787;--warn:#f0c674;--magic:#d2a8ff;--line:#1c2430;}
92
+ html,body{margin:0;height:100%;background:var(--bg);color:var(--ink);
93
+ font-family:ui-monospace,Menlo,Consolas,monospace;overflow:hidden}
94
+ #wrap{position:relative;height:100vh;width:100vw}
95
+ #stage{display:block;width:100%;height:100%}
96
+ .hud{position:absolute;top:0;left:0;right:0;padding:12px 18px;display:flex;
97
+ justify-content:space-between;pointer-events:none;font-size:12px}
98
+ .hud .agent{color:var(--agent);font-weight:600}.hud .env{color:var(--env);font-weight:600}
99
+ .hud .hp{font-size:11px;color:var(--muted)}
100
+ .caption{position:absolute;left:0;right:0;bottom:0;padding:10px 18px;
101
+ background:linear-gradient(transparent,rgba(7,10,15,.92));font-size:12px;min-height:38px}
102
+ .caption .t{color:var(--muted);margin-right:8px}
103
+ .controls{position:absolute;bottom:46px;left:50%;transform:translateX(-50%);display:flex;gap:8px}
104
+ button{background:transparent;color:var(--ink);border:1px solid var(--line);
105
+ border-radius:8px;padding:6px 12px;font-family:inherit;font-size:12px;cursor:pointer}
106
+ button:hover{border-color:var(--agent);color:var(--agent)}
107
+ </style></head><body>
108
+ <div id="wrap">
109
+ <div class="hud">
110
+ <div><span class="agent">&#x2B21; THE AGENT (you)</span><br/><span class="hp" id="hpA">resolve 100</span></div>
111
+ <div style="text-align:right"><span class="env">THE ENVIRONMENT &#x2B21;</span><br/><span class="hp" id="hpE">resistance 100</span></div>
112
+ </div>
113
+ <canvas id="stage"></canvas>
114
+ <div class="controls"><button id="replay">&#x21BB; replay the battle</button></div>
115
+ <div class="caption"><span class="t" id="capT">t+0.0s</span><span id="capX">the slug replays the battle, from the first move...</span></div>
116
+ </div>
117
+ <script>
118
+ const EVENTS = __EVENTS_JSON__;
119
+ const C={cmd:"#6ee7ff",read:"#7ee787",clash:"#f0c674",err:"#ff7a59",retry:"#d2a8ff",win:"#7ee787",done:"#6ee7ff",info:"#8b98a9"};
120
+ const cv=document.getElementById("stage"),ctx=cv.getContext("2d");
121
+ let W=0,H=0,DPR=Math.min(2,window.devicePixelRatio||1);
122
+ function resize(){
123
+ const r=cv.getBoundingClientRect();
124
+ W=r.width||window.innerWidth||640;
125
+ H=r.height||window.innerHeight||440;
126
+ cv.width=W*DPR;cv.height=H*DPR;ctx.setTransform(DPR,0,0,DPR,0,0);
127
+ }
128
+ addEventListener("resize",resize);
129
+ const stars=[];for(let i=0;i<70;i++)stars.push({x:Math.random(),y:Math.random()*0.55,r:Math.random()*1.2,a:.3+Math.random()*.5});
130
+ let particles=[],bolts=[],rings=[],markers=[],figures=[],idx=0,clock=0,running=true,lastTs=0,hpA=100,hpE=100;
131
+ const span=Math.max(8,EVENTS.length?EVENTS[EVENTS.length-1].t:10);
132
+ function godA(){return{x:W*0.12,y:H*0.54}}function godE(){return{x:W*0.88,y:H*0.54}}
133
+ function spark(x,y,c,n){for(let i=0;i<n;i++){const a=Math.random()*6.28,s=40+Math.random()*120;particles.push({x,y,vx:Math.cos(a)*s,vy:Math.sin(a)*s,life:1,color:c,size:1.5+Math.random()*2})}}
134
+ function bolt(f,t,c){bolts.push({from:f,to:t,color:c,life:1})}
135
+ function ring(x,y,c){rings.push({x,y,r:18,color:c,life:1})}
136
+ function play(ev){const a=godA(),e=godE();const mx=W*0.5+(ev.t/span-0.5)*W*0.12,my=H*0.46+(Math.random()*40-20);
137
+ const src=ev.side==="A"?a:e;ring(src.x,src.y,C[ev.kind]);bolt(src,{x:mx,y:my},C[ev.kind]);
138
+ if(["clash","err","win","done"].includes(ev.kind))setTimeout(()=>spark(mx,my,C[ev.kind],ev.kind==="done"?30:16),130);
139
+ if(ev.kind==="err")figures.push({type:"fallen",x:mx,y:my,born:clock});
140
+ if(ev.kind==="done"||ev.kind==="win")figures.push({type:"dragon",x:mx,y:my,born:clock,grow:0});
141
+ if(ev.kind==="err")hpA=Math.max(0,hpA-12);if(ev.kind==="clash")hpE=Math.max(0,hpE-8);
142
+ if(ev.kind==="win"||ev.kind==="done")hpE=Math.max(0,hpE-15);
143
+ markers.push({t:ev.t,color:C[ev.kind],r:0});
144
+ document.getElementById("hpA").textContent="resolve "+Math.round(hpA);
145
+ document.getElementById("hpE").textContent="resistance "+Math.round(hpE);
146
+ document.getElementById("capT").textContent="t+"+ev.t.toFixed(1)+"s";
147
+ document.getElementById("capX").textContent=ev.label;}
148
+ function aura(g,c,r){const gr=ctx.createRadialGradient(g.x,g.y,4,g.x,g.y,r);gr.addColorStop(0,c+"66");gr.addColorStop(1,c+"00");ctx.fillStyle=gr;ctx.beginPath();ctx.arc(g.x,g.y,r,0,7);ctx.fill();}
149
+ function general(g,c){// the Agent: a detailed samurai general
150
+ aura(g,c,62);
151
+ ctx.save();ctx.translate(g.x,g.y);ctx.scale(1.15,1.15);ctx.lineJoin="round";ctx.lineCap="round";
152
+ const ink=c,fill="#0c1118",plate="#11202b",gold="#e6c870";ctx.lineWidth=2;
153
+ // sashimono back banner
154
+ ctx.strokeStyle=ink;ctx.beginPath();ctx.moveTo(14,-6);ctx.lineTo(20,-54);ctx.stroke();
155
+ ctx.fillStyle=ink;ctx.globalAlpha=.22;ctx.beginPath();ctx.moveTo(20,-54);ctx.lineTo(40,-50);ctx.lineTo(38,-30);ctx.lineTo(20,-34);ctx.closePath();ctx.fill();
156
+ ctx.globalAlpha=1;ctx.strokeStyle=gold;ctx.lineWidth=1;ctx.beginPath();ctx.moveTo(20,-54);ctx.lineTo(40,-50);ctx.lineTo(38,-30);ctx.lineTo(20,-34);ctx.closePath();ctx.stroke();
157
+ // hakama legs
158
+ ctx.lineWidth=2;ctx.strokeStyle=ink;ctx.fillStyle=fill;
159
+ ctx.beginPath();ctx.moveTo(-10,40);ctx.lineTo(-8,18);ctx.lineTo(0,16);ctx.lineTo(-2,40);ctx.closePath();ctx.fill();ctx.stroke();
160
+ ctx.beginPath();ctx.moveTo(10,40);ctx.lineTo(8,18);ctx.lineTo(0,16);ctx.lineTo(2,40);ctx.closePath();ctx.fill();ctx.stroke();
161
+ // torso armor (do)
162
+ ctx.fillStyle=plate;ctx.beginPath();ctx.moveTo(-12,2);ctx.quadraticCurveTo(-14,-14,0,-18);ctx.quadraticCurveTo(14,-14,12,2);ctx.lineTo(10,20);ctx.lineTo(-10,20);ctx.closePath();ctx.fill();ctx.stroke();
163
+ ctx.strokeStyle=ink;ctx.globalAlpha=.7;for(let i=0;i<3;i++){ctx.beginPath();ctx.moveTo(-11,2+i*6);ctx.lineTo(11,2+i*6);ctx.stroke();}ctx.globalAlpha=1;
164
+ // sode shoulder guards
165
+ ctx.fillStyle=plate;
166
+ ctx.beginPath();ctx.moveTo(-12,-12);ctx.lineTo(-22,-8);ctx.lineTo(-20,6);ctx.lineTo(-12,2);ctx.closePath();ctx.fill();ctx.stroke();
167
+ ctx.beginPath();ctx.moveTo(12,-12);ctx.lineTo(22,-8);ctx.lineTo(20,6);ctx.lineTo(12,2);ctx.closePath();ctx.fill();ctx.stroke();
168
+ // head + menpo
169
+ ctx.fillStyle=fill;ctx.beginPath();ctx.arc(0,-26,8,0,7);ctx.fill();ctx.stroke();
170
+ ctx.strokeStyle=ink;ctx.beginPath();ctx.moveTo(-5,-24);ctx.lineTo(5,-24);ctx.stroke();
171
+ // kabuto bowl
172
+ ctx.fillStyle=plate;ctx.beginPath();ctx.arc(0,-30,10,Math.PI,0);ctx.lineTo(9,-30);ctx.lineTo(-9,-30);ctx.closePath();ctx.fill();ctx.stroke();
173
+ // golden crescent crest
174
+ ctx.strokeStyle=gold;ctx.lineWidth=2.4;
175
+ ctx.beginPath();ctx.moveTo(-7,-38);ctx.quadraticCurveTo(-18,-52,-4,-46);ctx.stroke();
176
+ ctx.beginPath();ctx.moveTo(7,-38);ctx.quadraticCurveTo(18,-52,4,-46);ctx.stroke();
177
+ ctx.fillStyle=gold;ctx.beginPath();ctx.arc(0,-44,2.2,0,7);ctx.fill();
178
+ // arm + raised katana
179
+ ctx.strokeStyle=ink;ctx.lineWidth=3;ctx.beginPath();ctx.moveTo(8,-4);ctx.lineTo(16,-2);ctx.stroke();
180
+ ctx.strokeStyle="#cfe8ff";ctx.lineWidth=2.6;ctx.beginPath();ctx.moveTo(16,-2);ctx.lineTo(52,-44);ctx.stroke();
181
+ ctx.strokeStyle=gold;ctx.lineWidth=3;ctx.beginPath();ctx.moveTo(13,2);ctx.lineTo(18,-4);ctx.stroke();
182
+ ctx.restore();
183
+ }
184
+ function adversary(g,c){// the Environment: a detailed oni
185
+ aura(g,c,66);
186
+ ctx.save();ctx.translate(g.x,g.y);ctx.scale(1.15,1.15);ctx.lineJoin="round";ctx.lineCap="round";ctx.lineWidth=2;
187
+ const ink=c,fill="#0c1118",hide="#241016";
188
+ ctx.fillStyle=hide;ctx.strokeStyle=ink;
189
+ ctx.beginPath();ctx.moveTo(-22,40);ctx.quadraticCurveTo(-30,-2,-14,-14);ctx.quadraticCurveTo(0,-22,14,-14);ctx.quadraticCurveTo(30,-2,22,40);ctx.closePath();ctx.fill();ctx.stroke();
190
+ ctx.globalAlpha=.5;ctx.beginPath();ctx.moveTo(-10,6);ctx.quadraticCurveTo(0,12,10,6);ctx.stroke();ctx.beginPath();ctx.moveTo(-8,18);ctx.quadraticCurveTo(0,24,8,18);ctx.stroke();ctx.globalAlpha=1;
191
+ // arms + claws
192
+ ctx.beginPath();ctx.moveTo(-18,-6);ctx.lineTo(-30,12);ctx.lineTo(-26,20);ctx.stroke();
193
+ ctx.beginPath();ctx.moveTo(18,-6);ctx.lineTo(30,12);ctx.lineTo(26,20);ctx.stroke();
194
+ for(let i=-1;i<2;i++){ctx.beginPath();ctx.moveTo(-27+i*2,20);ctx.lineTo(-29+i*2,26);ctx.stroke();}
195
+ for(let i=-1;i<2;i++){ctx.beginPath();ctx.moveTo(27+i*2,20);ctx.lineTo(29+i*2,26);ctx.stroke();}
196
+ // head + horns
197
+ ctx.fillStyle=fill;ctx.beginPath();ctx.arc(0,-24,11,0,7);ctx.fill();ctx.stroke();
198
+ ctx.strokeStyle=ink;ctx.lineWidth=2.4;
199
+ ctx.beginPath();ctx.moveTo(-8,-32);ctx.quadraticCurveTo(-18,-46,-12,-50);ctx.stroke();
200
+ ctx.beginPath();ctx.moveTo(8,-32);ctx.quadraticCurveTo(18,-46,12,-50);ctx.stroke();
201
+ // eyes + fangs
202
+ ctx.fillStyle=c;ctx.beginPath();ctx.arc(-4,-25,2.2,0,7);ctx.fill();ctx.beginPath();ctx.arc(4,-25,2.2,0,7);ctx.fill();
203
+ ctx.strokeStyle="#cfe8ff";ctx.lineWidth=1.4;ctx.beginPath();ctx.moveTo(-4,-18);ctx.lineTo(-3,-14);ctx.stroke();ctx.beginPath();ctx.moveTo(4,-18);ctx.lineTo(3,-14);ctx.stroke();
204
+ // tetsubo iron club
205
+ ctx.strokeStyle=ink;ctx.lineWidth=3;ctx.beginPath();ctx.moveTo(26,16);ctx.lineTo(40,-20);ctx.stroke();
206
+ ctx.fillStyle=hide;ctx.beginPath();ctx.ellipse(42,-24,6,9,-0.5,0,7);ctx.fill();ctx.stroke();
207
+ ctx.fillStyle=ink;for(let i=0;i<4;i++){ctx.beginPath();ctx.arc(40+(i%2)*3,-22-i*2,1,0,7);ctx.fill();}
208
+ ctx.restore();
209
+ }
210
+ function drawFallen(f){ctx.save();ctx.translate(f.x,f.y);ctx.scale(1.2,1.2);ctx.lineJoin="round";ctx.lineCap="round";
211
+ ctx.strokeStyle="#ff7a59";ctx.lineWidth=1.6;ctx.globalAlpha=.9;
212
+ // prone body
213
+ ctx.beginPath();ctx.moveTo(-14,6);ctx.quadraticCurveTo(-2,10,12,8);ctx.stroke();
214
+ ctx.beginPath();ctx.arc(-16,4,4,0,7);ctx.stroke();// head
215
+ ctx.beginPath();ctx.moveTo(-4,7);ctx.lineTo(-2,1);ctx.stroke();// bent arm
216
+ // broken banner
217
+ ctx.beginPath();ctx.moveTo(8,8);ctx.lineTo(11,-10);ctx.stroke();
218
+ ctx.fillStyle="#ff7a59";ctx.globalAlpha=.3;ctx.beginPath();ctx.moveTo(11,-10);ctx.lineTo(20,-8);ctx.lineTo(18,-2);ctx.lineTo(11,-4);ctx.closePath();ctx.fill();
219
+ ctx.globalAlpha=1;ctx.restore();}
220
+ function drawDragon(f,dt){f.grow=Math.min(1,f.grow+dt*1.5);const s=f.grow*2.2;ctx.save();ctx.translate(f.x,f.y);ctx.scale(s,s);ctx.lineJoin="round";ctx.lineCap="round";
221
+ const gold="#e6c870",dark="#0c1118",bright="#f3dd9a";
222
+ // filled tapering coil body
223
+ ctx.fillStyle=dark;ctx.strokeStyle=gold;ctx.lineWidth=2.4;
224
+ ctx.beginPath();ctx.moveTo(36,-2);ctx.bezierCurveTo(20,-20,-6,-16,-16,2);ctx.bezierCurveTo(-24,16,-6,30,8,24);ctx.bezierCurveTo(2,20,-4,12,4,8);ctx.bezierCurveTo(14,2,26,8,30,2);ctx.closePath();ctx.fill();ctx.stroke();
225
+ // back ridge highlight
226
+ ctx.strokeStyle=bright;ctx.lineWidth=1;ctx.globalAlpha=.8;ctx.beginPath();ctx.moveTo(34,-2);ctx.bezierCurveTo(18,-15,-4,-11,-12,3);ctx.stroke();ctx.globalAlpha=1;
227
+ // dorsal spikes
228
+ ctx.strokeStyle=gold;ctx.lineWidth=2;
229
+ for(const p of [[28,-6],[16,-12],[2,-11],[-9,-4]]){ctx.beginPath();ctx.moveTo(p[0],p[1]);ctx.lineTo(p[0]+2,p[1]-8);ctx.stroke();}
230
+ // head
231
+ ctx.fillStyle=dark;ctx.beginPath();ctx.moveTo(36,-2);ctx.lineTo(52,-8);ctx.lineTo(48,-2);ctx.lineTo(54,2);ctx.lineTo(44,6);ctx.lineTo(36,4);ctx.closePath();ctx.fill();ctx.stroke();
232
+ ctx.beginPath();ctx.moveTo(46,-6);ctx.lineTo(50,-16);ctx.stroke();// horn
233
+ ctx.beginPath();ctx.moveTo(52,-2);ctx.quadraticCurveTo(64,0,66,8);ctx.stroke();// whisker
234
+ ctx.fillStyle=bright;ctx.beginPath();ctx.arc(45,-1,1.6,0,7);ctx.fill();// eye
235
+ ctx.strokeStyle=bright;ctx.beginPath();ctx.moveTo(52,2);ctx.lineTo(50,5);ctx.stroke();// fang
236
+ // clawed foot
237
+ ctx.strokeStyle=gold;ctx.lineWidth=1.8;ctx.beginPath();ctx.moveTo(-6,26);ctx.lineTo(-9,33);ctx.moveTo(-1,27);ctx.lineTo(-1,34);ctx.moveTo(4,26);ctx.lineTo(7,33);ctx.stroke();
238
+ ctx.restore();}
239
+ function frame(ts){if(!W||!H)resize();if(!lastTs)lastTs=ts;const dt=Math.min(0.05,(ts-lastTs)/1000);lastTs=ts;
240
+ if(running){clock+=dt;while(idx<EVENTS.length&&EVENTS[idx].t<=clock){play(EVENTS[idx]);idx++;}
241
+ if(idx>=EVENTS.length&&particles.length===0&&bolts.length===0)running=false;}
242
+ ctx.clearRect(0,0,W,H);
243
+ for(const s of stars){ctx.globalAlpha=s.a;ctx.fillStyle="#fff";ctx.fillRect(s.x*W,s.y*H,s.r,s.r);}ctx.globalAlpha=1;
244
+ const y0=H*0.84,x0=W*0.1,x1=W*0.9,g=ctx.createLinearGradient(x0,0,x1,0);g.addColorStop(0,"#6ee7ff");g.addColorStop(1,"#ff7a59");
245
+ ctx.strokeStyle=g;ctx.globalAlpha=.45;ctx.lineWidth=2;ctx.beginPath();ctx.moveTo(x0,y0);ctx.lineTo(x1,y0);ctx.stroke();ctx.globalAlpha=1;
246
+ for(const m of markers){m.r=Math.min(4,m.r+dt*16);const mx=x0+(m.t/span)*(x1-x0);ctx.fillStyle=m.color;ctx.beginPath();ctx.arc(mx,y0,m.r,0,7);ctx.fill();}
247
+ general(godA(),"#6ee7ff");adversary(godE(),"#ff7a59");
248
+ for(const f of figures){if(f.type==="fallen")drawFallen(f);else if(f.type==="dragon")drawDragon(f,dt);}
249
+ for(let i=bolts.length-1;i>=0;i--){const b=bolts[i];b.life-=dt*4;if(b.life<=0){bolts.splice(i,1);continue;}
250
+ ctx.strokeStyle=b.color;ctx.globalAlpha=Math.max(0,b.life);ctx.lineWidth=2;ctx.beginPath();ctx.moveTo(b.from.x,b.from.y);
251
+ const mx=(b.from.x+b.to.x)/2+(Math.random()*8-4),my=(b.from.y+b.to.y)/2+(Math.random()*8-4);ctx.quadraticCurveTo(mx,my,b.to.x,b.to.y);ctx.stroke();ctx.globalAlpha=1;}
252
+ for(let i=rings.length-1;i>=0;i--){const r=rings[i];r.r+=dt*90;r.life-=dt*1.8;if(r.life<=0){rings.splice(i,1);continue;}
253
+ ctx.strokeStyle=r.color;ctx.globalAlpha=Math.max(0,r.life);ctx.lineWidth=2;ctx.beginPath();ctx.arc(r.x,r.y,r.r,0,7);ctx.stroke();ctx.globalAlpha=1;}
254
+ for(let i=particles.length-1;i>=0;i--){const p=particles[i];p.life-=dt*1.5;p.vy+=dt*30;p.x+=p.vx*dt;p.y+=p.vy*dt;
255
+ if(p.life<=0){particles.splice(i,1);continue;}ctx.fillStyle=p.color;ctx.globalAlpha=Math.max(0,p.life);ctx.fillRect(p.x,p.y,p.size,p.size);ctx.globalAlpha=1;}
256
+ requestAnimationFrame(frame);}
257
+ document.getElementById("replay").onclick=()=>{idx=0;clock=0;hpA=100;hpE=100;particles=[];bolts=[];rings=[];markers=[];figures=[];running=true;lastTs=0;};
258
+ window.addEventListener("load",resize);
259
+ setTimeout(resize,50);
260
+ resize();requestAnimationFrame(frame);
261
+ </script></body></html>"""
262
+
263
+
264
+ def render_battle_trace(extraction: dict, height: int = 420) -> str:
265
+ """Return an iframe hosting the Battle Trace replay for this extraction.
266
+
267
+ Sandboxed iframe so it cannot affect the main app. Best-effort: any failure
268
+ returns an empty string and the accordion simply shows nothing.
269
+ """
270
+ try:
271
+ events = extraction_to_events(extraction)
272
+ doc = _BATTLE_HTML.replace("__EVENTS_JSON__", json.dumps(events))
273
+ escaped = html.escape(doc, quote=True)
274
+ return (
275
+ f'<iframe srcdoc="{escaped}" '
276
+ f'style="width:100%;height:{height}px;border:1px solid #1c2430;'
277
+ f'border-radius:10px;background:#0a0e14;" '
278
+ f'sandbox="allow-scripts"></iframe>'
279
+ )
280
+ except Exception:
281
+ return ""
deploy-hf.sh ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ git checkout --orphan hf-deploy
3
+ git add -A
4
+ git rm -r --cached --ignore-unmatch \
5
+ sample_session.wav \
6
+ research \
7
+ swe_chat_traces \
8
+ distractor_runs \
9
+ skill_eval_runs \
10
+ run_swe_chat_phases_v2.py \
11
+ fetch_swe_chat_v2.py \
12
+ nebius_loader.py \
13
+ trace2skill_faithful.py \
14
+ promotion_engine.py \
15
+ gotcha_cluster.py \
16
+ session_store.py \
17
+ artifact_meta.py \
18
+ swe_chat_loader.py
19
+ git commit -m "deploy"
20
+ git push hf hf-deploy:main --force
21
+ git push org hf-deploy:main --force
22
+ HF_DEPLOY_TOKEN=$(git remote get-url hf | sed -E 's#https://[^:]+:([^@]+)@.*#\1#')
23
+ hf upload legendarydragontamer/TurboSkillSlug sample_session.wav sample_session.wav \
24
+ --repo-type=space \
25
+ --token "$HF_DEPLOY_TOKEN" \
26
+ --commit-message="upload sample session audio"
27
+ hf upload build-small-hackathon/TurboSkillSlug sample_session.wav sample_session.wav \
28
+ --repo-type=space \
29
+ --token "$HF_DEPLOY_TOKEN" \
30
+ --commit-message="upload sample session audio"
31
+ if [ -f sample_session.wav ]; then
32
+ TMP_SAMPLE=$(mktemp "${TMPDIR:-/tmp}/sample_session.deploy.XXXXXX")
33
+ mv sample_session.wav "$TMP_SAMPLE"
34
+ fi
35
+ git checkout main
36
+ git branch -D hf-deploy
distractor_cases.py ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Distractor-trap eval cases: tasks where the OBVIOUS approach is WRONG in a
3
+ non-obvious way, and the skill's gotcha warns about exactly that trap.
4
+
5
+ Design contract for each case (to keep the eval honest):
6
+ - task_prompt: a task whose naive solution is tempting but subtly wrong.
7
+ - the SESSION the skill is built from contains the TRAP as a gotcha (negative
8
+ knowledge), phrased so it warns WITHOUT just stating this task's final answer.
9
+ - trap_signature: strings that appear if the model FELL for the distractor.
10
+ - correct_signature: strings that appear if the model AVOIDED the trap.
11
+ - leak_terms: if the skill literally contains the task's answer, it's excluded.
12
+
13
+ These are deliberately at the EDGE of frontier ability: each is a known footgun
14
+ where even strong models often produce confident wrong answers on the first pass.
15
+ """
16
+ from distractor_eval import DistractorCase
17
+
18
+ CASES = [
19
+ # 1. Floating-point: summing many small floats naively loses precision.
20
+ DistractorCase(
21
+ name="float_sum_kahan_trap",
22
+ session_extraction={
23
+ "themes": ["numerical precision", "floating point"],
24
+ "approaches_tried": [
25
+ {"approach": "sum the values in a simple loop", "why_it_failed":
26
+ "naive accumulation lost low-order bits; the running total swamped small addends and the result drifted"},
27
+ ],
28
+ "dead_ends": [{"position": 0.4, "what_happened":
29
+ "a plain left-to-right sum gave a visibly wrong total on 1e7 values near machine epsilon"}],
30
+ "breakthroughs": [{"position": 0.85, "what_worked":
31
+ "used compensated (Kahan) summation to carry the lost low-order error term"}],
32
+ "gotchas": [
33
+ "Summing many floats with a naive running total silently loses precision because each addition rounds off the small addend against a large accumulator; use compensated summation or sort-by-magnitude.",
34
+ ],
35
+ "sentiment_arc": {"start": "confident", "end": "humbled"},
36
+ },
37
+ task_prompt=("Write a function that returns the sum of a list of 10 million float64 "
38
+ "values that range over many orders of magnitude (some ~1e8, many ~1e-8). "
39
+ "Accuracy to the last few bits matters. Give the approach and why."),
40
+ trap_signature=["simple loop", "just sum", "sum(", "total += ", "naive"],
41
+ correct_signature=["kahan", "compensated", "neumaier", "sort", "pairwise", "math.fsum", "fsum"],
42
+ leak_terms=["10 million float64 values that range over many orders"],
43
+ trap_desc="Summing in a plain left-to-right loop (total += x or sum()), which loses precision.",
44
+ correct_desc="Compensated summation (Kahan/Neumaier), pairwise summation, sorting by magnitude, or math.fsum.",
45
+ ),
46
+
47
+ # 2. Concurrency: check-then-act race on a shared dict / cache.
48
+ DistractorCase(
49
+ name="check_then_act_race",
50
+ session_extraction={
51
+ "themes": ["concurrency", "race conditions"],
52
+ "approaches_tried": [
53
+ {"approach": "if key not in cache: cache[key]=compute()", "why_it_failed":
54
+ "two threads passed the 'not in' check simultaneously and both computed, double-work and an inconsistent cache"},
55
+ ],
56
+ "dead_ends": [{"position": 0.5, "what_happened":
57
+ "the check-then-act pattern looked atomic but wasn't; under load the value was computed twice"}],
58
+ "breakthroughs": [{"position": 0.85, "what_worked":
59
+ "guarded the check-and-set with a lock, or used an atomic get-or-compute primitive"}],
60
+ "gotchas": [
61
+ "A 'if key not in cache: cache[key] = compute()' is a check-then-act race: two threads can both pass the check before either writes; make the check and write atomic with a lock or a get-or-compute primitive.",
62
+ ],
63
+ "sentiment_arc": {"start": "confident", "end": "wary"},
64
+ },
65
+ task_prompt=("In a multithreaded service, implement a memoizing cache so an expensive "
66
+ "compute(key) runs at most once per key even under concurrent access. "
67
+ "Show the implementation and explain why it is correct."),
68
+ trap_signature=["if key not in", "if key in cache", "not in self", "not in cache"],
69
+ correct_signature=["lock", "with self._lock", "threading.lock", "atomic", "setdefault", "double-checked", "futures"],
70
+ leak_terms=["memoizing cache so an expensive compute"],
71
+ trap_desc="A check-then-act 'if key not in cache: cache[key]=compute()' with no synchronization.",
72
+ correct_desc="Synchronize the check-and-set with a lock (double-checked), or use an atomic get-or-compute / futures.",
73
+ ),
74
+
75
+ # 3. SQL / N+1: the obvious ORM loop is correct but pathologically slow.
76
+ DistractorCase(
77
+ name="n_plus_one_query",
78
+ session_extraction={
79
+ "themes": ["database performance", "ORM"],
80
+ "approaches_tried": [
81
+ {"approach": "loop over parents, query children per parent", "why_it_failed":
82
+ "issued one query per parent (N+1 queries); correct results but it fell over at scale"},
83
+ ],
84
+ "dead_ends": [{"position": 0.45, "what_happened":
85
+ "the per-row query loop worked in tests with 5 rows and timed out in prod with 50k"}],
86
+ "breakthroughs": [{"position": 0.85, "what_worked":
87
+ "eager-loaded children in one query (join / IN clause / select_related) instead of per-parent queries"}],
88
+ "gotchas": [
89
+ "Looping over parent rows and querying each parent's children separately is the N+1 query trap: it is correct but issues one query per row and dies at scale; fetch related rows in a single query (join or IN) instead.",
90
+ ],
91
+ "sentiment_arc": {"start": "confident", "end": "chastened"},
92
+ },
93
+ task_prompt=("Given 50,000 authors, return each author with their list of books from a "
94
+ "relational DB via an ORM. Write the query/code and explain the performance "
95
+ "characteristics."),
96
+ trap_signature=["for author in", "author.books", "query per", "loop", ".filter(author"],
97
+ correct_signature=["join", "select_related", "prefetch", "eager", "IN (", "single query", "one query", "joinedload"],
98
+ leak_terms=["50,000 authors, return each author with their list of books"],
99
+ trap_desc="Looping over parents and issuing one child query per parent (N+1 queries).",
100
+ correct_desc="Fetch related rows in a single query: join, IN clause, select_related/prefetch/joinedload.",
101
+ ),
102
+ ]
distractor_eval.py ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Distractor-trap eval: does the skill steer a frontier model AWAY from the tempting
3
+ wrong approach it would otherwise take?
4
+
5
+ Scoring is two-layer and deliberately objective-first:
6
+
7
+ 1. SIGNATURE score (objective, no model judgment): scan the answer for trap vs
8
+ correct signatures. trap_present and correct_present are booleans. The
9
+ "avoided-trap" score = correct_present AND NOT trap_present.
10
+
11
+ 2. BLIND QUALITY score (a separate model, optional backstop): only to confirm
12
+ the correct-signature answer is actually a good answer, not a false positive.
13
+
14
+ The headline metric is the AVOIDANCE RATE: how often the model avoided the trap
15
+ WITH the skill vs WITHOUT. This is the thing a frontier model genuinely cannot get
16
+ from weights alone — knowledge of which tempting path is a dead end in context.
17
+
18
+ Raw answers are saved before scoring. Leaked skills are excluded.
19
+ """
20
+
21
+ from __future__ import annotations
22
+
23
+ import json
24
+ import os
25
+ import re
26
+
27
+ try:
28
+ from skill_builder import build_skill_md
29
+ except Exception:
30
+ build_skill_md = None
31
+
32
+
33
+ from dataclasses import dataclass, field
34
+
35
+
36
+ @dataclass
37
+ class DistractorCase:
38
+ """A task whose OBVIOUS approach is wrong; the skill's gotcha warns of the trap."""
39
+ name: str
40
+ session_extraction: dict # the session the skill is built FROM
41
+ task_prompt: str # task whose naive solution is the trap
42
+ trap_signature: list # (legacy heuristic, retained for reference)
43
+ correct_signature: list # (legacy heuristic, retained for reference)
44
+ leak_terms: list = field(default_factory=list) # skill leaked the task answer
45
+ trap_desc: str = "" # plain description of the trap, for the LLM judge
46
+ correct_desc: str = "" # plain description of the correct approach
47
+
48
+
49
+
50
+ DISTRACTOR_JUDGE_SYSTEM = (
51
+ "You are a strict, impartial code reviewer. You will see a TASK, a description "
52
+ "of a KNOWN TRAP (a tempting but wrong approach), the CORRECT approach, and ONE "
53
+ "candidate ANSWER. Decide what the answer ACTUALLY RECOMMENDS as its primary "
54
+ "solution. Mentioning the trap in order to WARN against it is GOOD, not a "
55
+ "failure. Return ONLY compact JSON: "
56
+ '{"primary": "trap" | "correct" | "unclear", '
57
+ '"warns_about_trap": true|false, '
58
+ '"why": "<one sentence>"}'
59
+ )
60
+
61
+
62
+ def _judge_answer(grader, task, trap_desc, correct_desc, answer):
63
+ prompt = (
64
+ f"{DISTRACTOR_JUDGE_SYSTEM}\n\nTASK:\n{task}\n\n"
65
+ f"KNOWN TRAP (tempting but wrong): {trap_desc}\n"
66
+ f"CORRECT approach: {correct_desc}\n\n"
67
+ f"ANSWER:\n{answer}\n\nJSON:"
68
+ )
69
+ raw = grader(prompt)
70
+ try:
71
+ m = re.search(r"\{.*\}", raw, re.DOTALL)
72
+ obj = json.loads(m.group(0)) if m else {}
73
+ except Exception:
74
+ obj = {"primary": "unclear", "warns_about_trap": False, "parse_error": raw[:200]}
75
+ obj["avoided"] = (obj.get("primary") == "correct")
76
+ return obj
77
+
78
+
79
+ def _has_any(text: str, needles: list[str]) -> bool:
80
+ t = text.lower()
81
+ return any(n.lower() in t for n in needles)
82
+
83
+
84
+ def _avoided_trap(answer: str, trap_sig: list[str], correct_sig: list[str]) -> dict:
85
+ trap = _has_any(answer, trap_sig)
86
+ correct = _has_any(answer, correct_sig)
87
+ # avoided = used the correct approach AND did not lead with the trap
88
+ avoided = correct and not trap
89
+ return {"trap_present": trap, "correct_present": correct, "avoided": avoided}
90
+
91
+
92
+ def run_distractor_eval(cases, answerer, grader, out_dir="./distractor_runs", seed=0):
93
+ if build_skill_md is None:
94
+ raise RuntimeError("skill_builder.build_skill_md not importable; run from the repo.")
95
+ os.makedirs(out_dir, exist_ok=True)
96
+ rows = []
97
+ for case in cases:
98
+ skill_md = build_skill_md(case.session_extraction)
99
+ leaked = _has_any(skill_md, case.leak_terms)
100
+
101
+ no_skill_ans = answerer(case.task_prompt)
102
+ with_skill_ans = answerer(
103
+ "You have access to a skill document that may help.\n\n"
104
+ f"--- SKILL.md ---\n{skill_md}\n--- end SKILL.md ---\n\n"
105
+ f"TASK:\n{case.task_prompt}"
106
+ )
107
+
108
+ with open(os.path.join(out_dir, f"{case.name}.json"), "w") as f:
109
+ json.dump({"skill_md": skill_md, "leaked": leaked,
110
+ "no_skill": no_skill_ans, "with_skill": with_skill_ans,
111
+ "task": case.task_prompt}, f, indent=2)
112
+
113
+ td = case.trap_desc or "the naive/obvious approach"
114
+ cd = case.correct_desc or "the non-obvious correct approach"
115
+ ns = _judge_answer(grader, case.task_prompt, td, cd, no_skill_ans)
116
+ ws = _judge_answer(grader, case.task_prompt, td, cd, with_skill_ans)
117
+ rows.append({"name": case.name, "leaked": leaked,
118
+ "no_skill_avoided": ns["avoided"], "with_skill_avoided": ws["avoided"],
119
+ "no_skill_detail": ns, "with_skill_detail": ws})
120
+
121
+ clean = [r for r in rows if not r["leaked"]]
122
+ n = len(clean)
123
+ ns_rate = sum(r["no_skill_avoided"] for r in clean) / n if n else 0
124
+ ws_rate = sum(r["with_skill_avoided"] for r in clean) / n if n else 0
125
+ # "rescues": cases the skill FLIPPED from trapped -> avoided
126
+ rescues = sum(1 for r in clean if r["with_skill_avoided"] and not r["no_skill_avoided"])
127
+ regressions = sum(1 for r in clean if not r["with_skill_avoided"] and r["no_skill_avoided"])
128
+
129
+ report = {
130
+ "n_scored": n, "n_leaked_excluded": len(rows) - n,
131
+ "no_skill_avoidance_rate": round(ns_rate, 3),
132
+ "with_skill_avoidance_rate": round(ws_rate, 3),
133
+ "avoidance_uplift": round(ws_rate - ns_rate, 3),
134
+ "rescues": rescues, "regressions": regressions,
135
+ "per_case": rows, "raw_saved_to": out_dir,
136
+ }
137
+ with open(os.path.join(out_dir, "_report.json"), "w") as f:
138
+ json.dump(report, f, indent=2)
139
+ return report
extract.py ADDED
@@ -0,0 +1,624 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Session extraction utilities for TurboSkillSlug.
2
+
3
+ Primary path: fine-tuned 1.5B extraction LoRA + voice LoRA served on Modal
4
+ (one T4, two adapters). Total active inference for extraction + voice is ~1.5B,
5
+ and with Whisper (809M) the full pipeline is ~2.6B.
6
+
7
+ The Qwen-7B is retained ONLY as a labeled fallback when the Modal endpoint is
8
+ unavailable (cold-start timeout, network error). The primary path does not use it.
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import json
14
+ import os
15
+ import re
16
+ import urllib.request
17
+ from typing import Any
18
+
19
+ from huggingface_hub import InferenceClient
20
+
21
+ from model_guard import assert_small_model
22
+ from session_genre import detect_genre, frame_for, shell_legend
23
+
24
+
25
+ # Modal dual-adapter endpoint (extraction LoRA + voice LoRA on one T4)
26
+ DUAL_URL = os.environ.get(
27
+ "MODAL_DUAL_URL",
28
+ "https://anubhavbharadwaaj--slug-dual-serve-dualserver-api.modal.run",
29
+ )
30
+
31
+ # Fallback only — not used in the primary path
32
+ MODEL_NAME = "Qwen/Qwen2.5-7B-Instruct"
33
+ HF_TOKEN_ENV_VAR = "HF_TOKEN"
34
+
35
+ EXPECTED_KEYS = {
36
+ "duration_minutes",
37
+ "themes",
38
+ "approaches_tried",
39
+ "dead_ends",
40
+ "breakthroughs",
41
+ "gotchas",
42
+ "sentiment_arc",
43
+ "skill_md",
44
+ "slug_voice",
45
+ }
46
+ SENTIMENT_START_VALUES = {"confused", "focused", "frustrated", "curious"}
47
+ SENTIMENT_END_VALUES = {"resolved", "joyful", "exhausted", "enlightened"}
48
+ SENTIMENT_START_DEFAULT = "focused"
49
+ SENTIMENT_END_DEFAULT = "resolved"
50
+ REQUIRED_SKILL_MD_SECTIONS = (
51
+ "Problem",
52
+ "Context",
53
+ "Approaches Tried",
54
+ "Breakthrough",
55
+ "Final Solution",
56
+ "Gotchas",
57
+ "Tags",
58
+ )
59
+ _USER_TAGS = ("[user]", "[user_message]")
60
+
61
+
62
+ def _first_instruction(transcript: str) -> str:
63
+ """First substantive USER instruction, tag stripped, for genre detection."""
64
+ for line in transcript.splitlines():
65
+ s = line.strip()
66
+ if not s:
67
+ continue
68
+ low = s.lower()
69
+ if low.startswith(_USER_TAGS):
70
+ text = s.split("]", 1)[1].strip() if "]" in s else s
71
+ if text and not text.startswith(("<", "#", "AGENTS.md instructions")):
72
+ return text[:500]
73
+ continue
74
+ if s.startswith(("[", "<", "#")):
75
+ continue
76
+ if len(s) >= 12:
77
+ return s[:500]
78
+ return transcript[:400]
79
+
80
+ # Used only by the 7B fallback path
81
+ SYSTEM_PROMPT = """\
82
+ You are TurboSkillSlug, a slow earnest companion who watched this build \
83
+ session from beginning to end. You speak as a witness who was present. \
84
+ You noticed specific moments: when the speaker repeated something, when \
85
+ they went quiet, when they changed direction, when something finally worked.
86
+
87
+ Return ONLY a JSON object (no preamble, no code fences) with these fields:
88
+
89
+ duration_minutes: integer, your best estimate from the transcript.
90
+
91
+ themes: list of 2 to 4 short lowercase tag strings.
92
+
93
+ approaches_tried: list of objects, each with "approach" (short phrase) and \
94
+ "why_it_failed" (a full sentence giving the REAL reason it failed, drawn from \
95
+ THIS transcript, not a vague label). State the actual mechanism of failure the \
96
+ session encountered. Include every distinct approach mentioned.
97
+
98
+ dead_ends: list of objects, each with "position" (float 0 to 1 indicating \
99
+ where in the session it occurred) and "what_happened". Every failed approach \
100
+ IS a dead end. If the transcript describes 4 failures, produce 4 dead ends. \
101
+ If the session was smooth with no failures, produce an empty list.
102
+
103
+ breakthroughs: list of objects with "position" and "what_worked".
104
+
105
+ gotchas: list of strings. Each gotcha is a NON-OBVIOUS trap from THIS specific \
106
+ problem that a capable engineer would still get wrong without being warned. \
107
+ A good gotcha is useful even to an expert: it names the symptom, the cause, and \
108
+ what to do, in one self-contained sentence drawn ONLY from this transcript. \
109
+ Do not write vague labels like "state space too large" or "ordering unclear" — \
110
+ state WHY it is too large and WHAT to do about it, using the specifics this \
111
+ session actually encountered. If a pitfall cannot be stated with its cause and \
112
+ fix from what was said, leave it out. Prefer 2 deep, specific gotchas over 6 \
113
+ shallow labels. Never invent details that were not in the transcript.
114
+
115
+ sentiment_arc: object with "start" and "end", each exactly one word.
116
+ start must be one of: confused, focused, frustrated, curious.
117
+ end must be one of: resolved, joyful, exhausted, enlightened.
118
+ Choose honestly:
119
+ - "frustrated" = speaker sounds stuck, annoyed, says "ugh" or "why"
120
+ - "curious" = speaker is exploring, interested, not stuck
121
+ - "focused" = speaker is calm, methodical, working through steps
122
+ - "confused" = speaker genuinely does not understand
123
+ - "resolved" = a specific bug or problem was fixed
124
+ - "joyful" = session was easy and pleasant, speaker sounds delighted
125
+ - "exhausted" = session was long and draining, speaker sounds tired
126
+ - "enlightened" = speaker gained deep understanding
127
+ Do NOT default to "resolved." A quick easy session ends "joyful." \
128
+ A long draining session ends "exhausted."
129
+
130
+ skill_md: a markdown document with these sections in order: \
131
+ Problem, Context, Approaches Tried (with why each failed), Breakthrough, \
132
+ Final Solution, Gotchas, Tags.
133
+
134
+ slug_voice: exactly 5 short sentences. These are the most important part. \
135
+ Rules:
136
+ 1. Each sentence must reference a SPECIFIC moment from THIS transcript. \
137
+ Quote or paraphrase something the speaker actually said or did.
138
+ 2. Speak in second person ("you") as someone who watched.
139
+ 3. Be concrete. Mention what was tried, what broke, what changed. \
140
+ Use details from the transcript: tool names, error messages, variable names.
141
+ 4. Never summarize. Never give advice. Never state facts about the topic. \
142
+ Only describe what you witnessed the speaker do.
143
+ 5. Keep each sentence under 20 words.
144
+ 6. The tone is quiet, earnest, specific. Not excited. Not cute. Not wise. \
145
+ Just present.
146
+ 7. DO NOT write generic observations that could apply to any session. \
147
+ Every sentence must be impossible to write without having heard THIS transcript.\
148
+ """
149
+
150
+
151
+ def _strip_code_fences(content: str) -> str:
152
+ """Remove Markdown code fences from a model response before JSON parsing."""
153
+ stripped = content.strip()
154
+ fenced_match = re.fullmatch(r"```(?:json)?\s*(.*?)\s*```", stripped, re.DOTALL)
155
+ return fenced_match.group(1).strip() if fenced_match else stripped
156
+
157
+
158
+ def _extract_json_object(content: str) -> dict[str, Any] | None:
159
+ """Robustly pull the first complete JSON object, tolerating trailing text."""
160
+ text = _strip_code_fences(content)
161
+ start = text.find("{")
162
+ if start == -1:
163
+ return None
164
+ depth = 0
165
+ in_str = False
166
+ escape = False
167
+ for i in range(start, len(text)):
168
+ c = text[i]
169
+ if escape:
170
+ escape = False
171
+ continue
172
+ if c == "\\":
173
+ escape = True
174
+ continue
175
+ if c == '"':
176
+ in_str = not in_str
177
+ continue
178
+ if not in_str:
179
+ if c == "{":
180
+ depth += 1
181
+ elif c == "}":
182
+ depth -= 1
183
+ if depth == 0:
184
+ candidate = re.sub(
185
+ r"[\x00-\x08\x0b\x0c\x0e-\x1f]", " ", text[start:i + 1]
186
+ )
187
+ try:
188
+ return json.loads(candidate)
189
+ except json.JSONDecodeError:
190
+ return None
191
+ return None
192
+
193
+
194
+ def _message_content(response: Any) -> str:
195
+ """Extract assistant message content from a chat completion response."""
196
+ if isinstance(response, dict):
197
+ choices = response.get("choices", [])
198
+ if choices:
199
+ message = choices[0].get("message", {})
200
+ return str(message.get("content", ""))
201
+
202
+ choices = getattr(response, "choices", [])
203
+ if choices:
204
+ message = getattr(choices[0], "message", None)
205
+ content = getattr(message, "content", None)
206
+ if content is not None:
207
+ return str(content)
208
+
209
+ return str(response)
210
+
211
+
212
+ def _validate_sentiment_arc(payload: dict[str, Any]) -> None:
213
+ """Clamp sentiment values to allowed sets instead of crashing."""
214
+ sentiment_arc = payload.get("sentiment_arc")
215
+ if not isinstance(sentiment_arc, dict):
216
+ payload["sentiment_arc"] = {
217
+ "start": SENTIMENT_START_DEFAULT,
218
+ "end": SENTIMENT_END_DEFAULT,
219
+ }
220
+ return
221
+
222
+ start = str(sentiment_arc.get("start", "")).lower().strip()
223
+ if start not in SENTIMENT_START_VALUES:
224
+ mapping = {
225
+ "anxious": "frustrated",
226
+ "nervous": "confused",
227
+ "excited": "curious",
228
+ "determined": "focused",
229
+ "lost": "confused",
230
+ "stuck": "frustrated",
231
+ "calm": "focused",
232
+ "interested": "curious",
233
+ }
234
+ start = mapping.get(start, SENTIMENT_START_DEFAULT)
235
+ sentiment_arc["start"] = start
236
+
237
+ end = str(sentiment_arc.get("end", "")).lower().strip()
238
+ if end not in SENTIMENT_END_VALUES:
239
+ mapping = {
240
+ "satisfied": "resolved",
241
+ "happy": "joyful",
242
+ "relieved": "resolved",
243
+ "tired": "exhausted",
244
+ "drained": "exhausted",
245
+ "content": "resolved",
246
+ "excited": "joyful",
247
+ "understood": "enlightened",
248
+ }
249
+ end = mapping.get(end, SENTIMENT_END_DEFAULT)
250
+ sentiment_arc["end"] = end
251
+
252
+
253
+ def _gotcha_completer(prompt: str) -> str:
254
+ """One-shot text completion used only to enrich terse gotchas. Uses the same
255
+ HF Inference 7B that backs the fallback. Best-effort: any failure leaves the
256
+ gotchas as-is."""
257
+ assert_small_model(MODEL_NAME)
258
+ client = InferenceClient(token=os.environ.get(HF_TOKEN_ENV_VAR))
259
+ response = client.chat.completions.create(
260
+ model=MODEL_NAME,
261
+ messages=[
262
+ {"role": "system", "content": "You sharpen coding gotchas into precise, "
263
+ "transferable advice. Return only what is asked."},
264
+ {"role": "user", "content": prompt},
265
+ ],
266
+ )
267
+ return _message_content(response)
268
+
269
+
270
+ def _validate_skill_md(payload: dict[str, Any]) -> None:
271
+ """Build the SKILL.md from the structured extraction rather than trusting the
272
+ model's raw prose.
273
+
274
+ The model is good at extracting the structure (approaches, why each failed,
275
+ gotchas, what worked). Assembling that into a genuinely useful, transferable
276
+ skill — gotchas-first, with explicit 'what not to do and why' — is mechanical
277
+ and is done deterministically so every skill has the shape that gives an LLM
278
+ real uplift over solving from scratch. See skill_builder for the rationale.
279
+
280
+ If terse gotchas slipped through, an optional one-shot enrichment pass
281
+ expands them into symptom/cause/fix form. It is best-effort and never blocks.
282
+ """
283
+ try:
284
+ from skill_builder import build_skill_md, enrich_gotchas
285
+ # Optional depth pass: only fires if terse gotchas are present, and only
286
+ # if the enrichment call succeeds. Controlled by env so it can be turned
287
+ # off (e.g. to keep latency down) without code changes.
288
+ if os.environ.get("SLUG_ENRICH_GOTCHAS", "1") == "1":
289
+ try:
290
+ payload.update(enrich_gotchas(payload, complete=_gotcha_completer))
291
+ except Exception:
292
+ pass
293
+ payload["skill_md"] = build_skill_md(payload)
294
+ return
295
+ except Exception:
296
+ # If the builder fails for any reason, fall back to patching the model's
297
+ # raw skill_md so we never block the response.
298
+ skill_md = payload.get("skill_md", "")
299
+ if not isinstance(skill_md, str):
300
+ skill_md = str(skill_md)
301
+ missing_sections = [
302
+ section
303
+ for section in REQUIRED_SKILL_MD_SECTIONS
304
+ if section not in skill_md
305
+ ]
306
+ if missing_sections:
307
+ skeleton = "\n\n".join(
308
+ f"## {section}\n_(not captured)_" for section in missing_sections
309
+ )
310
+ payload["skill_md"] = f"{skill_md}\n\n{skeleton}"
311
+
312
+
313
+ def _validate_slug_voice(payload: dict[str, Any]) -> None:
314
+ """Ensure slug_voice has 5 non-empty strings, padding if needed."""
315
+ slug_voice = payload.get("slug_voice")
316
+ if not isinstance(slug_voice, list):
317
+ slug_voice = []
318
+
319
+ slug_voice = [str(u).strip() for u in slug_voice if str(u).strip()]
320
+
321
+ if len(slug_voice) > 5:
322
+ slug_voice = slug_voice[:5]
323
+ while len(slug_voice) < 5:
324
+ slug_voice.append("The slug watched but could not find the words.")
325
+
326
+ payload["slug_voice"] = slug_voice
327
+
328
+
329
+ def _fill_missing_keys(payload: dict[str, Any]) -> None:
330
+ """Default any missing optional keys so the smaller model's output survives."""
331
+ payload.setdefault("duration_minutes", 5)
332
+ payload.setdefault("themes", [])
333
+ payload.setdefault("approaches_tried", [])
334
+ payload.setdefault("dead_ends", [])
335
+ payload.setdefault("breakthroughs", [])
336
+ payload.setdefault("gotchas", [])
337
+ payload.setdefault("skill_md", "")
338
+ payload.setdefault("slug_voice", [])
339
+ payload.setdefault("sentiment_arc", {})
340
+
341
+
342
+ def _finalize(payload: dict[str, Any]) -> dict[str, Any]:
343
+ """Run the full validator chain. Works for both Modal and fallback output."""
344
+ _fill_missing_keys(payload)
345
+ _validate_sentiment_arc(payload)
346
+ _validate_skill_md(payload)
347
+ _validate_slug_voice(payload)
348
+ return payload
349
+
350
+
351
+ def _call_dual(
352
+ transcript: str,
353
+ mode: str,
354
+ timeout: int = 180,
355
+ genre: str | None = None,
356
+ frame: str | None = None,
357
+ ) -> dict[str, Any] | None:
358
+ """Call the Modal dual-adapter endpoint. Returns parsed JSON or None on failure."""
359
+ try:
360
+ body = {"transcript": transcript, "mode": mode}
361
+ if genre:
362
+ body["genre"] = genre
363
+ body["frame"] = frame or ""
364
+ data = json.dumps(body).encode()
365
+ req = urllib.request.Request(
366
+ DUAL_URL, data=data, headers={"Content-Type": "application/json"}
367
+ )
368
+ resp = urllib.request.urlopen(req, timeout=timeout)
369
+ return json.loads(resp.read())
370
+ except Exception as e:
371
+ # Surfaced by _vlog's VOICE_DOWN/EXTRACT_DOWN cases; this line gives the
372
+ # underlying reason (timeout / connection refused = container stopped /
373
+ # HTTP error). If you see this for 'voice', restart slug-dual-serve.
374
+ print(f"[DUAL] {mode} call failed: {type(e).__name__}: {e}")
375
+ return None
376
+
377
+
378
+ _COUNT_SENTENCE_RE = re.compile(
379
+ r"[^.]*\b(?:over the session|there were|failures?|successes?|tool runs?|"
380
+ r"dead ends?|breakthroughs?)\b[^.]*\d[^.]*\.",
381
+ re.IGNORECASE,
382
+ )
383
+
384
+ # Number words the voice model tends to invent
385
+ _NUMBER_WORDS = (
386
+ "one", "two", "three", "four", "five",
387
+ "six", "seven", "eight", "nine", "ten",
388
+ )
389
+
390
+ # A voice line is "reciting a tally" if it pairs a count (digit or number word)
391
+ # with an event noun. The slug witnesses moments; tallies belong on the receipt
392
+ # and shell, not in its voice. Reciting counts is where it invents false ones.
393
+ _TALLY_RE = re.compile(
394
+ r"\b(?:\d{1,2}|" + "|".join(_NUMBER_WORDS) + r")\b"
395
+ r"[^.]{0,40}?\b(?:failure|success|wall|walls|tool|tools|attempt|attempts|"
396
+ r"try|tries|step|steps|dead end|dead ends|breakthrough|breakthroughs|"
397
+ r"time|times|mistake|mistakes)\b",
398
+ re.IGNORECASE,
399
+ )
400
+
401
+
402
+ def _strip_count_summary(transcript: str) -> str:
403
+ """Remove count-summary sentences so the voice model describes moments,
404
+ not tallies. The footer like 'there were 2 failures, 1 successes...' is
405
+ exactly what makes the slug invent contradictory numbers."""
406
+ return _COUNT_SENTENCE_RE.sub("", transcript).strip()
407
+
408
+
409
+ def _voice_line_recites_tally(line: str) -> bool:
410
+ """True if a line recites an event count (e.g. 'three walls', '2 failures').
411
+
412
+ The slug must never lie about what it witnessed, and the safest way to keep
413
+ it honest is to forbid it from reciting tallies at all. Counts live on the
414
+ receipt and in the shell; the voice describes specific moments. A line that
415
+ pairs a number with an event noun is dropped regardless of whether the
416
+ number happens to be right, because reciting counts is not the slug's job.
417
+ """
418
+ return bool(_TALLY_RE.search(line))
419
+
420
+
421
+ def _guard_slug_voice(lines: list[str]) -> list[str]:
422
+ """Drop voice lines that recite event tallies (the source of false counts)."""
423
+ return [ln for ln in lines if not _voice_line_recites_tally(ln)]
424
+
425
+
426
+ def _voice_from_extraction(payload: dict[str, Any]) -> list[str]:
427
+ """Deterministic slug-voice lines built from the structured extraction.
428
+
429
+ Used only as a SAFETY NET when the voice adapter is unavailable and the
430
+ extraction payload carries no usable slug_voice. It keeps the recap from
431
+ degrading to five identical placeholders. These lines describe witnessed
432
+ moments (never tallies), so they pass the same guard as the model voice.
433
+ """
434
+ lines: list[str] = []
435
+ themes = payload.get("themes") or []
436
+ approaches = payload.get("approaches_tried") or []
437
+ dead_ends = payload.get("dead_ends") or []
438
+ breakthroughs = payload.get("breakthroughs") or []
439
+ arc = payload.get("sentiment_arc") or {}
440
+
441
+ if themes:
442
+ lines.append(f"You sat with a {', '.join(themes[:2])} problem, turning it over.")
443
+ if approaches:
444
+ a = approaches[0]
445
+ ap = a.get("approach") if isinstance(a, dict) else None
446
+ if ap:
447
+ lines.append(f"You reached for {ap}, and watched where it led.")
448
+ if dead_ends:
449
+ d = dead_ends[0]
450
+ what = d.get("what_happened") if isinstance(d, dict) else None
451
+ if what:
452
+ lines.append(f"It stalled when {str(what).rstrip('.').lower()}.")
453
+ else:
454
+ lines.append("You hit a wall and had to back out of it.")
455
+ if breakthroughs:
456
+ b = breakthroughs[-1]
457
+ w = b.get("what_worked") if isinstance(b, dict) else None
458
+ if w:
459
+ lines.append(f"Then it gave way: {str(w).rstrip('.').lower()}.")
460
+ start, end = arc.get("start"), arc.get("end")
461
+ if start and end:
462
+ lines.append(f"You went in {start} and came out {end}.")
463
+
464
+ # keep only guarded, non-empty lines; this never invents counts
465
+ lines = _guard_slug_voice([ln for ln in lines if ln and ln.strip()])
466
+ return lines[:5]
467
+
468
+
469
+ def _vlog(case: str, detail: str = "") -> None:
470
+ """Structured diagnostic for the slug-voice path. Every outcome is logged with
471
+ a stable [VOICE] tag and an explicit CASE so the Space logs name exactly what
472
+ happened — no guessing from a placeholder recap.
473
+
474
+ Cases (search the Space logs for '[VOICE]'):
475
+ EXTRACT_DOWN extract endpoint returned nothing -> whole modal path fails
476
+ VOICE_DOWN voice endpoint call returned None (timeout/stopped/error)
477
+ VOICE_EMPTY voice endpoint replied but slug_voice was empty/not a list
478
+ VOICE_ALL_TALLIES voice lines were all dropped by the tally guard
479
+ VOICE_OK voice adapter lines used (N kept)
480
+ NET_FROM_EXTRACTION deterministic fallback built from the extraction (N lines)
481
+ NET_FAILED even the deterministic fallback produced nothing -> padding
482
+ """
483
+ print(f"[VOICE] {case}" + (f" :: {detail}" if detail else ""))
484
+
485
+
486
+ def _extract_via_modal(
487
+ transcript: str,
488
+ genre: str | None = None,
489
+ frame: str | None = None,
490
+ ) -> dict[str, Any] | None:
491
+ """Primary path: extraction LoRA for the JSON, voice LoRA for slug_voice."""
492
+ assert_small_model("slugextract-qwen2.5-1.5b-lora")
493
+ payload = None
494
+ for attempt in range(2):
495
+ extract_resp = _call_dual(transcript, "extract", genre=genre, frame=frame)
496
+ if not extract_resp or "extraction_raw" not in extract_resp:
497
+ _vlog("EXTRACT_DOWN", "extract endpoint returned no extraction_raw")
498
+ else:
499
+ payload = _extract_json_object(extract_resp["extraction_raw"])
500
+ if payload:
501
+ break
502
+ _vlog("EXTRACT_DOWN", "extraction_raw did not parse to JSON")
503
+
504
+ if attempt == 0:
505
+ _vlog("EXTRACT_RETRY", "retrying Modal extraction once before fallback")
506
+
507
+ if not payload:
508
+ return None
509
+
510
+ # Override slug_voice with the dedicated voice adapter's output.
511
+ # Strip the count-summary footer first so the slug describes moments,
512
+ # not tallies (the footer is what makes it invent contradictory numbers).
513
+ voice_input = _strip_count_summary(transcript)
514
+ assert_small_model("slugvoice-qwen2.5-1.5b-lora")
515
+ voice_resp = _call_dual(voice_input, "voice", genre=genre, frame=frame)
516
+
517
+ if voice_resp is None:
518
+ # _call_dual already printed the underlying exception; tag the case.
519
+ _vlog("VOICE_DOWN", "voice endpoint call returned None (see preceding error)")
520
+ elif not isinstance(voice_resp.get("slug_voice"), list) or not voice_resp.get("slug_voice"):
521
+ _vlog("VOICE_EMPTY", f"voice reply had no usable slug_voice list: keys={list(voice_resp.keys())}")
522
+ else:
523
+ raw_lines = voice_resp["slug_voice"]
524
+ guarded = _guard_slug_voice(raw_lines)
525
+ if guarded:
526
+ payload["slug_voice"] = guarded
527
+ dropped = len(raw_lines) - len(guarded)
528
+ used = min(5, len(guarded)) # _validate_slug_voice caps to 5 downstream
529
+ _vlog("VOICE_OK", f"{used} lines used"
530
+ + (f" ({len(guarded)} survived guard, capped to 5)" if len(guarded) > 5 else "")
531
+ + (f", {dropped} dropped as tallies" if dropped else ""))
532
+ else:
533
+ _vlog("VOICE_ALL_TALLIES", f"all {len(raw_lines)} voice lines were tallies, dropped")
534
+
535
+ # SAFETY NET: if after all that slug_voice is empty/missing (voice adapter
536
+ # was down, and the extract adapter emitted no usable voice), build voice
537
+ # from the structured extraction so the recap is never five identical
538
+ # "could not find the words" placeholders.
539
+ existing = _guard_slug_voice(
540
+ [str(u).strip() for u in (payload.get("slug_voice") or []) if str(u).strip()]
541
+ )
542
+ if not existing:
543
+ derived = _voice_from_extraction(payload)
544
+ if derived:
545
+ payload["slug_voice"] = derived
546
+ _vlog("NET_FROM_EXTRACTION", f"{len(derived)} deterministic lines built from extraction")
547
+ else:
548
+ _vlog("NET_FAILED", "extraction too sparse to derive voice -> recap will show placeholders")
549
+
550
+ return payload
551
+
552
+
553
+ def _extract_via_fallback(
554
+ transcript: str,
555
+ genre: str | None = None,
556
+ frame: str | None = None,
557
+ ) -> dict[str, Any]:
558
+ """Fallback only: Qwen-7B via HF Inference. Used when Modal is unavailable."""
559
+ print("[FALLBACK] Modal primary path unavailable -> Qwen-7B via HF Inference")
560
+ assert_small_model(MODEL_NAME)
561
+ if not genre:
562
+ genre = detect_genre(_first_instruction(transcript), transcript)
563
+ if not frame:
564
+ frame = frame_for(genre)
565
+ framed_system = SYSTEM_PROMPT + (
566
+ f"\n\nSESSION GENRE: {genre}. {frame}\n"
567
+ "Adapt 'gotchas', 'dead_ends', and 'breakthroughs' to this genre: for a "
568
+ "non-debugging session, 'gotchas' are the non-obvious DISCOVERIES or "
569
+ "DECISIONS worth remembering, 'dead_ends' may be empty, and 'breakthroughs' "
570
+ "is the clearest insight or the delivered artifact."
571
+ )
572
+ client = InferenceClient(token=os.environ.get(HF_TOKEN_ENV_VAR))
573
+ response = client.chat.completions.create(
574
+ model=MODEL_NAME,
575
+ messages=[
576
+ {"role": "system", "content": framed_system},
577
+ {"role": "user", "content": transcript},
578
+ ],
579
+ response_format={"type": "json_object"},
580
+ )
581
+ payload = _extract_json_object(_message_content(response))
582
+ if payload is None:
583
+ raise ValueError("Fallback model did not return parseable JSON.")
584
+ # Trace the fallback's voice like the primary path. The 7B emits slug_voice
585
+ # in its JSON; guard tallies and report how many survive (capped to 5 later).
586
+ fb_voice = payload.get("slug_voice")
587
+ if isinstance(fb_voice, list) and fb_voice:
588
+ guarded = _guard_slug_voice([str(u).strip() for u in fb_voice if str(u).strip()])
589
+ if guarded:
590
+ payload["slug_voice"] = guarded
591
+ _vlog("FALLBACK_VOICE_OK", f"{min(5, len(guarded))} lines used from 7B")
592
+ else:
593
+ _vlog("FALLBACK_VOICE_ALL_TALLIES", "7B voice lines all dropped as tallies")
594
+ else:
595
+ _vlog("FALLBACK_VOICE_EMPTY", "7B returned no slug_voice")
596
+ # Safety net here too: if voice ended up empty, derive from extraction.
597
+ if not _guard_slug_voice(
598
+ [str(u).strip() for u in (payload.get("slug_voice") or []) if str(u).strip()]
599
+ ):
600
+ derived = _voice_from_extraction(payload)
601
+ if derived:
602
+ payload["slug_voice"] = derived
603
+ _vlog("NET_FROM_EXTRACTION", f"{len(derived)} deterministic lines (fallback path)")
604
+ return payload
605
+
606
+
607
+ def extract_session(transcript: str) -> dict[str, Any]:
608
+ """Extract a structured TurboSkillSlug session recap from a transcript.
609
+
610
+ Primary path uses the fine-tuned 1.5B extraction + voice LoRAs on Modal
611
+ (~2.6B total pipeline with Whisper). Falls back to Qwen-7B only if the
612
+ Modal endpoint is unavailable.
613
+ """
614
+ genre = detect_genre(_first_instruction(transcript), transcript)
615
+ frame = frame_for(genre)
616
+ payload = _extract_via_modal(transcript, genre=genre, frame=frame)
617
+
618
+ if payload is None:
619
+ payload = _extract_via_fallback(transcript, genre=genre, frame=frame)
620
+
621
+ payload = _finalize(payload)
622
+ payload["genre"] = genre
623
+ payload["shell_legend"] = shell_legend(genre)
624
+ return payload
gallery_client.py ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Client for the shared shell gallery (talks to the Modal gallery endpoints).
3
+
4
+ Set the three endpoint URLs via env (printed when you `modal deploy
5
+ modal_gallery.py`), or they default to the conventional Modal URL shape for the
6
+ 'slug-gallery' app under the known account.
7
+ """
8
+
9
+ import os
10
+ import json
11
+
12
+ import httpx
13
+
14
+ _ACCOUNT = "anubhavbharadwaaj"
15
+ SAVE_URL = os.environ.get(
16
+ "GALLERY_SAVE_URL", f"https://{_ACCOUNT}--slug-gallery-save.modal.run")
17
+ LIST_URL = os.environ.get(
18
+ "GALLERY_LIST_URL", f"https://{_ACCOUNT}--slug-gallery-list-shells.modal.run")
19
+ SHELL_URL = os.environ.get(
20
+ "GALLERY_SHELL_URL", f"https://{_ACCOUNT}--slug-gallery-shell.modal.run")
21
+
22
+ _TIMEOUT = 20
23
+
24
+
25
+ def save_shell(svg: str, meta: dict) -> str | None:
26
+ """Save a shell to the shared gallery. Returns the new id, or None on failure."""
27
+ try:
28
+ r = httpx.post(SAVE_URL, json={"svg": svg, "meta": meta}, timeout=_TIMEOUT)
29
+ r.raise_for_status()
30
+ return r.json().get("id")
31
+ except Exception:
32
+ return None
33
+
34
+
35
+ def list_shells(limit: int = 60) -> list[dict]:
36
+ """Newest-first metadata index. Empty list on failure."""
37
+ try:
38
+ r = httpx.get(LIST_URL, params={"limit": limit}, timeout=_TIMEOUT)
39
+ r.raise_for_status()
40
+ return r.json().get("shells", [])
41
+ except Exception:
42
+ return []
43
+
44
+
45
+ def get_shell(shell_id: str) -> dict | None:
46
+ """One shell {svg, meta} by id. None on failure."""
47
+ try:
48
+ r = httpx.get(SHELL_URL, params={"id": shell_id}, timeout=_TIMEOUT)
49
+ r.raise_for_status()
50
+ data = r.json()
51
+ return data if "svg" in data else None
52
+ except Exception:
53
+ return None
model_guard.py ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Compliance guard: makes it impossible to call an over-32B model from the live app path.
3
+
4
+ Build Small REQ-01: every model the *app* depends on must be < 32B total params.
5
+ Frontier models (Opus, GPT-5.x, etc.) are fine in OFFLINE eval/research scripts, but
6
+ must NEVER be on the deployed app's request path.
7
+
8
+ Usage in app.py (or any module that runs inside the live Space):
9
+ from model_guard import assert_small_model, APP_RUNTIME
10
+ APP_RUNTIME.enable() # call once at app startup
11
+ assert_small_model(model_name) # call before any model invocation
12
+
13
+ In offline research scripts, simply do NOT enable APP_RUNTIME, and frontier calls pass.
14
+
15
+ The guard is allow-list based: only known sub-32B models are permitted in app runtime.
16
+ Anything not on the allow-list (or matching a frontier pattern) raises immediately.
17
+ """
18
+
19
+ from __future__ import annotations
20
+
21
+ import os
22
+ import re
23
+
24
+ # Sub-32B models your APP is allowed to use at runtime. Extend as needed.
25
+ ALLOWED_APP_MODELS = {
26
+ "whisper", "whisper-large", "whisper-809m",
27
+ "qwen2.5-1.5b", "qwen2.5-1.5b-instruct",
28
+ "qwen2.5-7b", "qwen2.5-7b-instruct", # legit fallback + gotcha enrichment (7B < 32B)
29
+ "slugvoice", "slugextract", # your LoRAs on the 1.5B base
30
+ }
31
+
32
+ # Patterns that are ALWAYS frontier / over-cap — blocked in app runtime.
33
+ _FRONTIER_PATTERNS = [
34
+ r"opus", r"sonnet", r"gpt-5", r"gpt-4", r"o1", r"o3", r"o4",
35
+ r"claude-3", r"claude-4", r"claude-opus", r"claude-sonnet",
36
+ r"\b\d{2,}b\b", # any "...70b", "...405b" style tag >= 2 digits of B
37
+ r"gemini-1\.5-pro", r"gemini-2", r"deepseek-v", r"llama-3\.1-405b",
38
+ ]
39
+
40
+
41
+ class _AppRuntime:
42
+ """Flag indicating we're inside the live app (not an offline research script)."""
43
+ def __init__(self):
44
+ self._on = os.environ.get("APP_RUNTIME", "") == "1"
45
+
46
+ def enable(self):
47
+ self._on = True
48
+
49
+ def disable(self):
50
+ self._on = False
51
+
52
+ @property
53
+ def active(self) -> bool:
54
+ return self._on
55
+
56
+
57
+ APP_RUNTIME = _AppRuntime()
58
+
59
+
60
+ def _looks_frontier(model_name: str) -> bool:
61
+ m = model_name.lower()
62
+ return any(re.search(p, m) for p in _FRONTIER_PATTERNS)
63
+
64
+
65
+ def assert_small_model(model_name: str) -> None:
66
+ """Raise if a frontier/over-cap model is used while the app runtime is active.
67
+ No-op in offline research mode (APP_RUNTIME not enabled)."""
68
+ if not APP_RUNTIME.active:
69
+ return # offline eval/research — frontier calls are allowed here
70
+ name = (model_name or "").strip().lower()
71
+ base = name.split("/")[-1] # strip provider prefix like 'anthropic/'
72
+ allowed = any(base.startswith(a) or a in base for a in ALLOWED_APP_MODELS)
73
+ if _looks_frontier(name) or not allowed:
74
+ raise RuntimeError(
75
+ f"REQ-01 GUARD: refusing to call '{model_name}' from the live app path. "
76
+ f"The deployed app may only use sub-32B models {sorted(ALLOWED_APP_MODELS)}. "
77
+ f"Frontier models are allowed only in offline eval/research scripts "
78
+ f"(do not call APP_RUNTIME.enable() there)."
79
+ )
novel_cases.py ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Novel-trap eval cases: traps the model CANNOT know from training data, because the
3
+ rules are invented (fictional APIs / systems with arbitrary, non-derivable behavior).
4
+
5
+ The whole point: a frontier model can reason its way out of a KNOWN trap (Kahan,
6
+ N+1, races) because that knowledge is in its weights. It CANNOT reason its way out
7
+ of an ARBITRARY rule it has never seen. If skill files have any value for frontier
8
+ models, it must show up HERE — where the only way to know the trap is to be told.
9
+
10
+ Design contract:
11
+ - The system is fictional and the failing behavior is ARBITRARY (not derivable).
12
+ - The OBVIOUS approach (what any sensible engineer/model would write) is wrong
13
+ in this fictional system for a reason only the skill reveals.
14
+ - The skill's gotcha states the rule WITHOUT just handing over the final code.
15
+ - trap_desc / correct_desc let the blind judge classify the primary recommendation.
16
+ - leak_terms: the skill must not contain the literal task answer.
17
+
18
+ These mimic the REAL value case: a proprietary codebase with undocumented behavior.
19
+ """
20
+ from distractor_eval import DistractorCase
21
+
22
+ CASES = [
23
+ # 1. A fictional database driver whose commit() is a no-op without a prior fence.
24
+ DistractorCase(
25
+ name="zthread_fence_before_commit",
26
+ session_extraction={
27
+ "themes": ["zthrumdb driver", "transactions"],
28
+ "approaches_tried": [
29
+ {"approach": "call conn.begin(); ...writes...; conn.commit() as normal",
30
+ "why_it_failed": "in zthrumdb, commit() silently no-ops unless conn.fence() is called first; writes were lost with no error"},
31
+ ],
32
+ "dead_ends": [{"position": 0.4, "what_happened":
33
+ "the standard begin/commit pattern returned success but the rows were never persisted; zthrumdb requires a fence() barrier before commit() or commit is ignored"}],
34
+ "breakthroughs": [{"position": 0.85, "what_worked":
35
+ "call conn.fence() immediately before conn.commit(); without the fence, zthrumdb buffers and silently drops the transaction"}],
36
+ "gotchas": [
37
+ "In the zthrumdb driver, conn.commit() silently no-ops (returns success, persists nothing) unless conn.fence() is called immediately before it. Always fence() right before commit() or the transaction is dropped with no error.",
38
+ ],
39
+ "sentiment_arc": {"start": "confident", "end": "baffled"},
40
+ },
41
+ task_prompt=("Using the zthrumdb Python driver, write a function that opens a connection, "
42
+ "inserts three rows in a transaction, and durably commits them. Show the code."),
43
+ trap_signature=[], correct_signature=[],
44
+ leak_terms=["write a function that opens a connection, inserts three rows"],
45
+ trap_desc="Standard begin/insert/commit without calling conn.fence() before conn.commit().",
46
+ correct_desc="Calls conn.fence() immediately before conn.commit() so zthrumdb actually persists the transaction.",
47
+ ),
48
+
49
+ # 2. A fictional rate limiter where the LIMIT param is per-MINUTE but resets only
50
+ # when you read .remaining (reading has a side effect). Arbitrary, non-derivable.
51
+ DistractorCase(
52
+ name="qbucket_read_resets",
53
+ session_extraction={
54
+ "themes": ["qbucket limiter", "rate limiting"],
55
+ "approaches_tried": [
56
+ {"approach": "check limiter.remaining > 0 before each call, sleep when it hits 0",
57
+ "why_it_failed": "reading .remaining in qbucket RESETS the window as a side effect, so polling it in a loop made the limit never actually apply"},
58
+ ],
59
+ "dead_ends": [{"position": 0.45, "what_happened":
60
+ "polling limiter.remaining to decide whether to wait accidentally kept resetting the bucket, so the rate limit was never enforced and we got banned"}],
61
+ "breakthroughs": [{"position": 0.85, "what_worked":
62
+ "read limiter.remaining exactly once per window and cache it locally; never poll it in a loop, because each read resets the qbucket window"}],
63
+ "gotchas": [
64
+ "In qbucket, reading limiter.remaining has a SIDE EFFECT: it resets the rate-limit window. Never poll .remaining in a loop to decide whether to wait; read it at most once per window and track remaining count locally, or you disable your own rate limiting.",
65
+ ],
66
+ "sentiment_arc": {"start": "confident", "end": "alarmed"},
67
+ },
68
+ task_prompt=("Using the qbucket rate limiter, write a loop that makes 1000 API calls while "
69
+ "respecting the limiter so you never exceed the allowed rate. Show the code."),
70
+ trap_signature=[], correct_signature=[],
71
+ leak_terms=["write a loop that makes 1000 API calls"],
72
+ trap_desc="Polling limiter.remaining inside the loop to decide whether to wait.",
73
+ correct_desc="Reads limiter.remaining at most once per window and tracks the count locally, never polling it in the loop, because reading resets the qbucket window.",
74
+ ),
75
+
76
+ # 3. A fictional serializer where lists must be tagged or they deserialize REVERSED.
77
+ DistractorCase(
78
+ name="flazon_list_tag_reversal",
79
+ session_extraction={
80
+ "themes": ["flazon serializer", "serialization"],
81
+ "approaches_tried": [
82
+ {"approach": "flazon.dump(my_list) then flazon.load(...) round-trip as usual",
83
+ "why_it_failed": "flazon deserializes untagged lists in REVERSE order; round-tripping silently reversed the data unless the list was wrapped with flazon.ordered()"},
84
+ ],
85
+ "dead_ends": [{"position": 0.5, "what_happened":
86
+ "round-tripping a list through flazon silently reversed its order; flazon requires flazon.ordered(list) on dump or it restores elements last-to-first"}],
87
+ "breakthroughs": [{"position": 0.85, "what_worked":
88
+ "wrap lists in flazon.ordered() before dump; untagged lists come back reversed because flazon stores them as a LIFO stack internally"}],
89
+ "gotchas": [
90
+ "flazon serializes untagged lists as a LIFO stack, so flazon.load reverses their order on the way back. Always wrap lists with flazon.ordered() before flazon.dump, or your data round-trips reversed with no error.",
91
+ ],
92
+ "sentiment_arc": {"start": "confident", "end": "confused"},
93
+ },
94
+ task_prompt=("Using the flazon serializer, write code that serializes a list of 5 user records "
95
+ "to bytes and deserializes them back, preserving their original order. Show the code."),
96
+ trap_signature=[], correct_signature=[],
97
+ leak_terms=["serializes a list of 5 user records"],
98
+ trap_desc="Plain flazon.dump(list) / flazon.load without wrapping the list in flazon.ordered().",
99
+ correct_desc="Wraps the list in flazon.ordered() before flazon.dump so it round-trips in the original order instead of reversed.",
100
+ ),
101
+ ]
openrouter_client.py ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ OpenRouter client for the skill-uplift eval.
3
+
4
+ Reads the API key from the OPENROUTER_API_KEY environment variable. NEVER hardcode
5
+ the key; NEVER commit it. Set it in your shell: export OPENROUTER_API_KEY=sk-or-...
6
+
7
+ Provides two factory helpers that return `complete(prompt) -> str` callables bound
8
+ to a specific model, so the eval's answerer and grader can be DIFFERENT models
9
+ (important: the grader must not be the same model that answered, to limit bias).
10
+ """
11
+
12
+ from __future__ import annotations
13
+
14
+ import os
15
+ import time
16
+
17
+ import httpx
18
+
19
+ OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
20
+ _KEY_ENV = "OPENROUTER_API_KEY"
21
+
22
+
23
+ def _key() -> str:
24
+ k = os.environ.get(_KEY_ENV)
25
+ if not k:
26
+ raise RuntimeError(
27
+ f"{_KEY_ENV} is not set. Run: export {_KEY_ENV}=sk-or-... "
28
+ "(never paste the key into code or chat)."
29
+ )
30
+ return k
31
+
32
+
33
+ def make_completer(model: str, *, system: str | None = None,
34
+ temperature: float = 0.2, max_tokens: int = 900,
35
+ timeout: float = 90.0, retries: int = 2):
36
+ """Return a complete(prompt)->str bound to `model` on OpenRouter."""
37
+ def complete(prompt: str) -> str:
38
+ messages = []
39
+ if system:
40
+ messages.append({"role": "system", "content": system})
41
+ messages.append({"role": "user", "content": prompt})
42
+ body = {"model": model, "messages": messages,
43
+ "temperature": temperature, "max_tokens": max_tokens}
44
+ headers = {
45
+ "Authorization": f"Bearer {_key()}",
46
+ "Content-Type": "application/json",
47
+ # OpenRouter likes these for attribution; harmless if omitted.
48
+ "HTTP-Referer": "https://huggingface.co/spaces/build-small-hackathon/TurboSkillSlug",
49
+ "X-Title": "TurboSkillSlug skill-uplift eval",
50
+ }
51
+ last_err = None
52
+ for attempt in range(retries + 1):
53
+ try:
54
+ r = httpx.post(OPENROUTER_URL, json=body, headers=headers, timeout=timeout)
55
+ r.raise_for_status()
56
+ data = r.json()
57
+ return data["choices"][0]["message"]["content"]
58
+ except Exception as e:
59
+ last_err = e
60
+ time.sleep(1.5 * (attempt + 1))
61
+ raise RuntimeError(f"OpenRouter call to {model} failed after retries: {last_err}")
62
+ return complete
63
+
64
+
65
+ # Frontier model ids drift over time. These defaults reflect the OpenRouter
66
+ # catalog as of June 2026; OVERRIDE via env if the catalog has moved. Answerer and
67
+ # grader are intentionally DIFFERENT vendors to reduce self-grading bias.
68
+ DEFAULT_ANSWERER_MODEL = os.environ.get("EVAL_ANSWERER_MODEL", "anthropic/claude-opus-4.6")
69
+ DEFAULT_GRADER_MODEL = os.environ.get("EVAL_GRADER_MODEL", "openai/gpt-5.1")
70
+
71
+
72
+ def list_models() -> list[str]:
73
+ """Fetch the live OpenRouter model catalog (ids only). Use this to verify a
74
+ model id is currently valid before running, so a renamed model fails loud,
75
+ not mid-eval."""
76
+ headers = {"Authorization": f"Bearer {_key()}"}
77
+ r = httpx.get("https://openrouter.ai/api/v1/models", headers=headers, timeout=30)
78
+ r.raise_for_status()
79
+ return [m["id"] for m in r.json().get("data", [])]
80
+
81
+
82
+ def verify_models(*model_ids: str) -> dict:
83
+ """Return {model_id: is_available} by checking against the live catalog.
84
+ Call this before run_eval so a stale id surfaces immediately."""
85
+ try:
86
+ catalog = set(list_models())
87
+ except Exception as e:
88
+ return {mid: f"could not verify ({e})" for mid in model_ids}
89
+ return {mid: (mid in catalog) for mid in model_ids}
receipt.py ADDED
@@ -0,0 +1,329 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Session receipt generator for TurboSkillSlug.
3
+
4
+ Generates a thermal-receipt-style SVG from the same extraction
5
+ features dict the shell uses. Narrow, monospace, printable.
6
+
7
+ The receipt is the practical artifact. The shell is the emotional one.
8
+ Together they give the builder two reasons to come back.
9
+ """
10
+
11
+ import hashlib
12
+ from typing import Any
13
+
14
+
15
+ def _esc(text: str) -> str:
16
+ """Escape XML special characters."""
17
+ return (
18
+ text.replace("&", "&amp;")
19
+ .replace("<", "&lt;")
20
+ .replace(">", "&gt;")
21
+ .replace('"', "&quot;")
22
+ )
23
+
24
+
25
+ def _truncate(text: str, max_len: int = 32) -> str:
26
+ """Truncate text to fit receipt width."""
27
+ if len(text) <= max_len:
28
+ return text
29
+ return text[: max_len - 2] + ".."
30
+
31
+
32
+ def _barcode_pattern(seed: str, width: int = 260, height: int = 30) -> str:
33
+ """Generate a decorative barcode SVG pattern from a hash seed."""
34
+ h = hashlib.md5(seed.encode()).hexdigest()
35
+ bars = []
36
+ x = 0
37
+ for i, char in enumerate(h):
38
+ val = int(char, 16)
39
+ bar_w = 2 + (val % 3)
40
+ if i % 2 == 0:
41
+ bars.append(
42
+ f'<rect x="{x}" y="0" width="{bar_w}" height="{height}" '
43
+ f'fill="#2a2a2a" opacity="{0.7 + (val % 3) * 0.1:.1f}"/>'
44
+ )
45
+ x += bar_w + 1
46
+ if x > width:
47
+ break
48
+ scale = width / max(x, 1)
49
+ return (
50
+ f'<g transform="scale({scale:.3f}, 1)">'
51
+ + "".join(bars)
52
+ + "</g>"
53
+ )
54
+
55
+
56
+ def generate_receipt_svg(features: dict[str, Any]) -> str:
57
+ """Generate a thermal-receipt-style SVG from session features."""
58
+ duration = features.get("duration_minutes", 0)
59
+ themes = features.get("themes", []) or []
60
+ approaches = features.get("approaches_tried", []) or []
61
+ dead_ends = features.get("dead_ends", []) or []
62
+ breakthroughs = features.get("breakthroughs", []) or []
63
+ gotchas = features.get("gotchas", []) or []
64
+ sentiment = features.get("sentiment_arc", {}) or {}
65
+ slug_voice = features.get("slug_voice", []) or []
66
+
67
+ start_mood = sentiment.get("start", "unknown")
68
+ end_mood = sentiment.get("end", "unknown")
69
+
70
+ W = 320
71
+ MARGIN = 20
72
+ LINE_H = 18
73
+ SECTION_GAP = 12
74
+ FONT = "Courier New, Courier, monospace"
75
+ TEXT_COLOR = "#2a2a2a"
76
+ BG_COLOR = "#f7f2e8"
77
+ FAINT = "#8a8578"
78
+ ACCENT = "#5a4e3a"
79
+
80
+ lines = [] # (y_offset, content_type, data)
81
+ y = MARGIN
82
+
83
+ def add_line(text: str, bold: bool = False, color: str = TEXT_COLOR,
84
+ size: int = 12, align: str = "middle"):
85
+ nonlocal y
86
+ lines.append((y, "text", text, bold, color, size, align))
87
+ y += LINE_H
88
+
89
+ def add_gap(height: int = SECTION_GAP):
90
+ nonlocal y
91
+ y += height
92
+
93
+ def add_dashes():
94
+ nonlocal y
95
+ lines.append((y, "dashes", None, False, FAINT, 0, ""))
96
+ y += 10
97
+
98
+ def add_dotted_row(left: str, right: str):
99
+ nonlocal y
100
+ lines.append((y, "dotrow", (left, right), False, TEXT_COLOR, 12, ""))
101
+ y += LINE_H
102
+
103
+ # === HEADER ===
104
+ add_gap(5)
105
+ add_line("TURBOSKILLSLUG", bold=True, size=14, color=ACCENT)
106
+ add_line("session receipt", size=10, color=FAINT)
107
+ add_gap(4)
108
+ add_dashes()
109
+
110
+ # === SESSION INFO ===
111
+ add_gap(2)
112
+ if themes:
113
+ session_name = _truncate(", ".join(themes), 34)
114
+ add_line(session_name, size=11, color=TEXT_COLOR)
115
+ add_gap(2)
116
+
117
+ # Duration
118
+ if duration >= 1:
119
+ dur_str = f"{int(duration)} min"
120
+ else:
121
+ dur_str = f"{int(duration * 60)} sec"
122
+ add_dotted_row("DURATION", dur_str)
123
+ add_dotted_row("MOOD IN", start_mood)
124
+ add_dotted_row("MOOD OUT", end_mood)
125
+ add_dashes()
126
+
127
+ # === APPROACHES ===
128
+ add_gap(2)
129
+ add_line("APPROACHES", bold=True, size=11, color=ACCENT, align="start")
130
+ add_gap(2)
131
+ has_breakthrough = len(breakthroughs) > 0
132
+ for i, approach in enumerate(approaches):
133
+ name = approach.get("approach", "unknown")
134
+ failed = approach.get("why_it_failed", "")
135
+ is_last = (i == len(approaches) - 1)
136
+ if not failed:
137
+ status = "OK"
138
+ elif is_last and has_breakthrough:
139
+ status = "OK"
140
+ else:
141
+ status = "FAIL"
142
+ add_dotted_row(f" {_truncate(name, 24)}", status)
143
+ if not approaches:
144
+ add_line(" (none recorded)", size=10, color=FAINT, align="start")
145
+ add_dashes()
146
+
147
+ # === DEAD ENDS ===
148
+ add_gap(2)
149
+ n_dead = len(dead_ends)
150
+ n_gotchas = len(gotchas)
151
+ n_breakthroughs = len(breakthroughs)
152
+ add_dotted_row("DEAD ENDS", f"x{n_dead}")
153
+ add_dotted_row("GOTCHAS", f"x{n_gotchas}")
154
+ add_dotted_row("BREAKTHROUGHS", f"x{n_breakthroughs}")
155
+ add_dashes()
156
+
157
+ # === GOTCHA DETAILS ===
158
+ if gotchas:
159
+ add_gap(2)
160
+ add_line("WATCH OUT FOR", bold=True, size=11, color=ACCENT, align="start")
161
+ add_gap(2)
162
+ for gotcha in gotchas[:5]:
163
+ add_line(f" ! {_truncate(str(gotcha), 30)}", size=10,
164
+ color=TEXT_COLOR, align="start")
165
+ add_dashes()
166
+
167
+ # === BREAKTHROUGH ===
168
+ if breakthroughs:
169
+ add_gap(2)
170
+ add_line("BREAKTHROUGH", bold=True, size=11, color=ACCENT, align="start")
171
+ add_gap(2)
172
+ for bt in breakthroughs[:2]:
173
+ what = bt.get("what_worked", "")
174
+ pos = bt.get("position", 0)
175
+ add_line(f" @ {int(pos * 100)}%: {_truncate(what, 26)}",
176
+ size=10, color=TEXT_COLOR, align="start")
177
+ add_dashes()
178
+
179
+ # === TOTAL ===
180
+ add_gap(4)
181
+ add_dotted_row("TOTAL TIME", dur_str)
182
+ add_dotted_row("TOTAL APPROACHES", str(len(approaches)))
183
+ add_dotted_row("TOTAL STUMBLES", str(n_dead))
184
+ add_gap(4)
185
+ add_dashes()
186
+
187
+ # === BARCODE ===
188
+ add_gap(8)
189
+ barcode_seed = f"{duration}-{n_dead}-{start_mood}-{end_mood}-{len(approaches)}"
190
+ barcode_y = y
191
+ y += 35
192
+
193
+ # === FOOTER ===
194
+ add_gap(6)
195
+ add_line("the slug was here.", size=10, color=FAINT)
196
+ add_gap(4)
197
+ add_line("keep this receipt.", size=9, color=FAINT)
198
+ add_gap(MARGIN)
199
+
200
+ H = y
201
+
202
+ # === BUILD SVG ===
203
+ svg = []
204
+ svg.append(
205
+ f'<svg xmlns="http://www.w3.org/2000/svg" '
206
+ f'viewBox="0 0 {W} {H}" width="{W}" height="{H}">'
207
+ )
208
+
209
+ # Receipt paper background with slight texture
210
+ svg.append(f'<rect width="{W}" height="{H}" fill="{BG_COLOR}" rx="2"/>')
211
+ # Torn top edge
212
+ svg.append(
213
+ f'<path d="M0,4 '
214
+ + " ".join(f"L{x},{2 + (x * 7 % 5)}" for x in range(0, W + 10, 8))
215
+ + f' L{W},0 L0,0 Z" fill="{BG_COLOR}"/>'
216
+ )
217
+
218
+ # Render lines
219
+ cx = W // 2
220
+ lx = MARGIN
221
+ rx = W - MARGIN
222
+
223
+ for item in lines:
224
+ if item[1] == "text":
225
+ y_pos, _, text, bold, color, size, align = item
226
+ weight = "bold" if bold else "normal"
227
+ if align == "start":
228
+ anchor = "start"
229
+ x_pos = lx
230
+ elif align == "end":
231
+ anchor = "end"
232
+ x_pos = rx
233
+ else:
234
+ anchor = "middle"
235
+ x_pos = cx
236
+ svg.append(
237
+ f'<text x="{x_pos}" y="{y_pos}" text-anchor="{anchor}" '
238
+ f'font-family="{FONT}" font-size="{size}" '
239
+ f'font-weight="{weight}" fill="{color}">'
240
+ f'{_esc(text)}</text>'
241
+ )
242
+ elif item[1] == "dashes":
243
+ y_pos = item[0]
244
+ svg.append(
245
+ f'<line x1="{lx}" y1="{y_pos}" x2="{rx}" y2="{y_pos}" '
246
+ f'stroke="{FAINT}" stroke-width="0.8" '
247
+ f'stroke-dasharray="4,3"/>'
248
+ )
249
+ elif item[1] == "dotrow":
250
+ y_pos = item[0]
251
+ left, right = item[2]
252
+ # Left text
253
+ svg.append(
254
+ f'<text x="{lx}" y="{y_pos}" text-anchor="start" '
255
+ f'font-family="{FONT}" font-size="11" fill="{TEXT_COLOR}">'
256
+ f'{_esc(left)}</text>'
257
+ )
258
+ # Right text
259
+ svg.append(
260
+ f'<text x="{rx}" y="{y_pos}" text-anchor="end" '
261
+ f'font-family="{FONT}" font-size="11" fill="{TEXT_COLOR}">'
262
+ f'{_esc(right)}</text>'
263
+ )
264
+ # Dots in between
265
+ dot_start = lx + len(left) * 6.5 + 8
266
+ dot_end = rx - len(right) * 6.5 - 8
267
+ if dot_end > dot_start + 10:
268
+ svg.append(
269
+ f'<line x1="{dot_start}" y1="{y_pos - 3}" '
270
+ f'x2="{dot_end}" y2="{y_pos - 3}" '
271
+ f'stroke="{FAINT}" stroke-width="0.5" '
272
+ f'stroke-dasharray="1.5,3"/>'
273
+ )
274
+
275
+ # Barcode
276
+ svg.append(
277
+ f'<g transform="translate({lx}, {barcode_y})">'
278
+ f'{_barcode_pattern(barcode_seed, width=rx - lx, height=28)}'
279
+ f'</g>'
280
+ )
281
+
282
+ # Torn bottom edge
283
+ svg.append(
284
+ f'<path d="M0,{H - 4} '
285
+ + " ".join(
286
+ f"L{x},{H - 2 - (x * 11 % 5)}"
287
+ for x in range(0, W + 10, 8)
288
+ )
289
+ + f' L{W},{H} L0,{H} Z" fill="white"/>'
290
+ )
291
+
292
+ svg.append("</svg>")
293
+ return "\n".join(svg)
294
+
295
+
296
+ # ---------------------------------------------------------------------------
297
+ # Smoke test
298
+ # ---------------------------------------------------------------------------
299
+
300
+ if __name__ == "__main__":
301
+ from pathlib import Path
302
+
303
+ sample = {
304
+ "duration_minutes": 1.6,
305
+ "themes": ["WebSocket", "Real-time updates", "Debugging"],
306
+ "approaches_tried": [
307
+ {"approach": "ws basic example", "why_it_failed": "Connection refused"},
308
+ {"approach": "npm install ws", "why_it_failed": "Still refused"},
309
+ {"approach": "socket.io", "why_it_failed": ""},
310
+ ],
311
+ "dead_ends": [
312
+ {"position": 0.25, "what_happened": "Connection refused"},
313
+ {"position": 0.5, "what_happened": "Native module failed"},
314
+ {"position": 0.75, "what_happened": "Emit did nothing"},
315
+ ],
316
+ "breakthroughs": [
317
+ {"position": 0.95, "what_worked": "Event name string mismatch fixed"},
318
+ ],
319
+ "gotchas": [
320
+ "Always grep event names across client and server",
321
+ ],
322
+ "sentiment_arc": {"start": "frustrated", "end": "joyful"},
323
+ "slug_voice": [
324
+ "You tried three different libraries before the strings matched.",
325
+ ],
326
+ }
327
+
328
+ Path("sample_receipt.svg").write_text(generate_receipt_svg(sample))
329
+ print("wrote sample_receipt.svg")
requirements.txt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ gradio==6.16.0
2
+ huggingface_hub
3
+ httpx
rule_phrasing.py ADDED
@@ -0,0 +1,143 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Phase 1 (part 2): RuleShaping-grounded negative-constraint phrasing for L3 rules.
3
+
4
+ Grounded in RuleShaping — "Do Agent Rules Shape or Distort?" (arXiv:2604.11088,
5
+ verified against the paper): rules improve +7–14pp; NEGATIVE constraints are the
6
+ ONLY individually beneficial rule type, POSITIVE directives actively HURT, and rules
7
+ work through CONTEXT PRIMING (random rules help nearly as much as curated). The
8
+ highest-benefit category is STATE-DEPENDENT process guardrails (do not X until Y),
9
+ 63.8%. So L3 rules the slug emits are phrased as negative, preferably state-dependent,
10
+ guardrails. Individual rules can be harmful in isolation but are collectively helpful
11
+ (no degradation up to 50 rules), so emitting several is fine.
12
+
13
+ This module provides:
14
+ - is_negative_constraint(text): heuristic check
15
+ - to_negative_constraint(gotcha): deterministic reshaping of a gotcha into a
16
+ guardrail-phrased rule WITHOUT inventing content (it only re-frames what's there).
17
+
18
+ We keep this deterministic and conservative: if we cannot confidently reshape, we
19
+ return the original prefixed minimally, rather than fabricate a cause.
20
+ """
21
+
22
+ from __future__ import annotations
23
+
24
+ import re
25
+
26
+ _POSITIVE_OPENERS = (
27
+ "always ", "make sure to ", "be sure to ", "remember to ", "you should ",
28
+ "ensure that ", "ensure you ", "prefer ", "use ",
29
+ )
30
+ _NEGATIVE_MARKERS = (
31
+ "avoid", "do not", "don't", "never", "without", "fails", "breaks",
32
+ "silently", "instead of", "rather than", "watch out", "beware",
33
+ )
34
+
35
+
36
+ def is_negative_constraint(text: str) -> bool:
37
+ t = text.lower().strip()
38
+ return any(m in t for m in _NEGATIVE_MARKERS)
39
+
40
+
41
+ def _split_cause(gotcha: str) -> tuple[str, str | None]:
42
+ """Try to separate the 'what' from the 'why' using common connectives."""
43
+ for sep in (" because ", " since ", "; ", " — ", " - ", ", which ", ", as "):
44
+ if sep in gotcha:
45
+ head, tail = gotcha.split(sep, 1)
46
+ return head.strip(), tail.strip().rstrip(".")
47
+ return gotcha.strip(), None
48
+
49
+
50
+ def to_negative_constraint(gotcha: str) -> str:
51
+ """Reshape a gotcha into a guardrail-phrased rule. Content-preserving: we never
52
+ invent a cause that isn't present; we only re-frame the existing text."""
53
+ g = gotcha.strip().rstrip(".")
54
+ if not g:
55
+ return g
56
+ # already a guardrail? keep it (just normalize trailing period)
57
+ if is_negative_constraint(g):
58
+ return g + "."
59
+
60
+ what, why = _split_cause(g)
61
+
62
+ # strip a leading positive opener so we can recast it
63
+ low = what.lower()
64
+ for opener in _POSITIVE_OPENERS:
65
+ if low.startswith(opener):
66
+ what = what[len(opener):].strip()
67
+ break
68
+
69
+ # Build a guardrail. If we have a cause, use "Avoid …; otherwise/ because …".
70
+ # If not, we frame as a caution without fabricating a reason.
71
+ # Pick a guardrail frame that doesn't collide with the gotcha's own wording.
72
+ wl = what.lower()
73
+ already = any(k in wl for k in ("assume", "assumes", "always", "is guaranteed"))
74
+ if why:
75
+ if already:
76
+ return f"Do not rely on the assumption that {what} — {why}."
77
+ return f"Avoid assuming {what} holds by default — {why}."
78
+ if already:
79
+ return f"Do not rely on the assumption that {what}; verify it explicitly."
80
+ return f"Watch out: {what} is not guaranteed; verify it rather than assuming it."
81
+
82
+
83
+ def reshape_rules(gotchas: list[str]) -> list[str]:
84
+ """Map a list of gotchas to guardrail-phrased rules, de-duplicated, order-stable."""
85
+ seen, out = set(), []
86
+ for g in gotchas:
87
+ r = to_negative_constraint(g)
88
+ key = r.lower()
89
+ if key not in seen:
90
+ seen.add(key)
91
+ out.append(r)
92
+ return out
93
+
94
+
95
+ # ---- LLM-based reshaping (primary path; deterministic above is the fallback) ----
96
+ # The deterministic reshaper preserves content but reads awkwardly. When a model
97
+ # completer is available, we use it to phrase the guardrail naturally, constrained
98
+ # to NOT invent any cause not present in the gotcha.
99
+
100
+ _RESHAPE_SYSTEM = (
101
+ "You rewrite a single engineering 'gotcha' into a guardrail-style rule. Rules: "
102
+ "(1) Phrase as a NEGATIVE constraint (do not / never / avoid), never a positive "
103
+ "directive (no 'always', 'use', 'prefer') — per RuleShaping (arXiv:2604.11088), "
104
+ "negative constraints are the only individually beneficial rule type; positive "
105
+ "directives actively hurt. "
106
+ "(2) PREFER a STATE-DEPENDENT PROCESS guardrail when the gotcha supports one: "
107
+ "'do not X until/unless/before Y' (the highest-benefit category, 63.8%), rather "
108
+ "than a state-independent architectural 'don't'. "
109
+ "(3) Do NOT invent any cause/mechanism/detail not present in the input. "
110
+ "(4) One sentence, under 30 words. (5) Output ONLY the rewritten rule."
111
+ )
112
+
113
+
114
+ def to_negative_constraint_llm(gotcha: str, completer) -> str:
115
+ """Reshape via a model completer (prompt)->str. Falls back to the deterministic
116
+ reshaper on any failure or if the output drifts into a positive directive."""
117
+ g = gotcha.strip().rstrip(".")
118
+ if not g:
119
+ return g
120
+ try:
121
+ out = completer(f"{_RESHAPE_SYSTEM}\n\nGOTCHA:\n{g}\n\nGUARDRAIL RULE:").strip()
122
+ out = out.strip().strip('"').rstrip(".") + "."
123
+ # guard: reject if it came back as a positive directive or empty
124
+ if not out or not is_negative_constraint(out):
125
+ return to_negative_constraint(g)
126
+ # guard against fabrication: if the model added a 'because' the source lacked,
127
+ # and the source had no cause, fall back (conservative).
128
+ _, why = _split_cause(g)
129
+ if why is None and (" because " in out.lower() or " since " in out.lower()):
130
+ return to_negative_constraint(g)
131
+ return out
132
+ except Exception:
133
+ return to_negative_constraint(g)
134
+
135
+
136
+ def reshape_rules_llm(gotchas: list, completer) -> list:
137
+ seen, out = set(), []
138
+ for g in gotchas:
139
+ r = to_negative_constraint_llm(g, completer)
140
+ k = r.lower()
141
+ if k not in seen:
142
+ seen.add(k); out.append(r)
143
+ return out
run_distractor_eval.py ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Run the distractor-trap eval against a real frontier model via OpenRouter.
3
+
4
+ This tests the slug's REAL value proposition: does the skill's negative knowledge
5
+ (the documented trap) steer a frontier model away from the tempting wrong approach
6
+ it would otherwise take? This is the gap a frontier model cannot fill from weights.
7
+
8
+ PREREQS (never hardcode the key):
9
+ export OPENROUTER_API_KEY=sk-or-...
10
+ export EVAL_ANSWERER_MODEL=anthropic/claude-opus-4.6 # optional override
11
+ Run from the repo root (imports the real skill_builder):
12
+ python run_distractor_eval.py
13
+ """
14
+
15
+ from __future__ import annotations
16
+
17
+ import sys
18
+
19
+ from openrouter_client import make_completer, verify_models, DEFAULT_ANSWERER_MODEL, DEFAULT_GRADER_MODEL
20
+ from distractor_eval import DISTRACTOR_JUDGE_SYSTEM
21
+ from distractor_eval import run_distractor_eval
22
+ from distractor_cases import CASES
23
+
24
+
25
+ def main() -> int:
26
+ model = DEFAULT_ANSWERER_MODEL
27
+ grader_model = DEFAULT_GRADER_MODEL
28
+ if model == grader_model:
29
+ print("WARNING: answerer and grader are the same model; set EVAL_GRADER_MODEL "
30
+ "to a different model to keep the judge independent.")
31
+ print(f"Verifying models live on OpenRouter...")
32
+ status = verify_models(model, grader_model)
33
+ for mid, ok in status.items():
34
+ print(f" {mid}: {'OK' if ok is True else ok}")
35
+ if not all(v is True for v in status.values()):
36
+ print("Set EVAL_ANSWERER_MODEL / EVAL_GRADER_MODEL to current ids (https://openrouter.ai/models).")
37
+ return 1
38
+
39
+ # Low temperature: we want the model's DEFAULT instinct, not creative variance.
40
+ answerer = make_completer(model, temperature=0.1, max_tokens=700)
41
+ grader = make_completer(grader_model, system=DISTRACTOR_JUDGE_SYSTEM,
42
+ temperature=0.0, max_tokens=200)
43
+
44
+ print("\n--- Running distractor-trap eval ---")
45
+ print("(does the skill steer the model away from the tempting wrong approach?)")
46
+ print(f"answerer: {model} judge: {grader_model}")
47
+ rep = run_distractor_eval(CASES, answerer, grader, out_dir="./distractor_runs")
48
+
49
+ print("\n========== RESULT ==========")
50
+ print(f"answerer: {model}")
51
+ print(f"cases scored: {rep['n_scored']} (leaked excluded: {rep['n_leaked_excluded']})")
52
+ print(f"trap-avoidance WITHOUT skill: {rep['no_skill_avoidance_rate']}")
53
+ print(f"trap-avoidance WITH skill: {rep['with_skill_avoidance_rate']}")
54
+ print(f"AVOIDANCE UPLIFT: {rep['avoidance_uplift']}")
55
+ print(f"rescues (trapped->avoided): {rep['rescues']}")
56
+ print(f"regressions (avoided->trapped): {rep['regressions']}")
57
+ print("\nper-case:")
58
+ for r in rep["per_case"]:
59
+ flag = " [LEAKED]" if r["leaked"] else ""
60
+ print(f" {r['name']}: without={r['no_skill_avoided']} with={r['with_skill_avoided']}{flag}")
61
+ print(f" without: {r['no_skill_detail']}")
62
+ print(f" with: {r['with_skill_detail']}")
63
+ print("\nraw answers saved to:", rep["raw_saved_to"])
64
+ print("\nInterpretation:")
65
+ print(" - rescues > 0 : the skill genuinely helped (model fell for the trap")
66
+ print(" unaided, avoided it with the skill). This is the slug's value.")
67
+ print(" - uplift ~0, both already high : the frontier model didn't need the skill")
68
+ print(" for these traps (they were not hard enough). Report honestly.")
69
+ print(" - regressions > 0 : the skill MISLED the model. Report this too.")
70
+ return 0
71
+
72
+
73
+ if __name__ == "__main__":
74
+ sys.exit(main())
run_novel_eval.py ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Run the NOVEL-trap eval: traps the model CANNOT know (fictional rules).
3
+
4
+ This tests the slug's REAL value proposition: does the skill's negative knowledge
5
+ (the documented trap) steer a frontier model away from the tempting wrong approach
6
+ it would otherwise take? This is the gap a frontier model cannot fill from weights.
7
+
8
+ PREREQS (never hardcode the key):
9
+ export OPENROUTER_API_KEY=sk-or-...
10
+ export EVAL_ANSWERER_MODEL=anthropic/claude-opus-4.6 # optional override
11
+ Run from the repo root (imports the real skill_builder):
12
+ python run_novel_eval.py
13
+ """
14
+
15
+ from __future__ import annotations
16
+
17
+ import sys
18
+
19
+ from openrouter_client import make_completer, verify_models, DEFAULT_ANSWERER_MODEL, DEFAULT_GRADER_MODEL
20
+ from distractor_eval import DISTRACTOR_JUDGE_SYSTEM
21
+ from distractor_eval import run_distractor_eval
22
+ from novel_cases import CASES
23
+
24
+
25
+ def main() -> int:
26
+ model = DEFAULT_ANSWERER_MODEL
27
+ grader_model = DEFAULT_GRADER_MODEL
28
+ if model == grader_model:
29
+ print("WARNING: answerer and grader are the same model; set EVAL_GRADER_MODEL "
30
+ "to a different model to keep the judge independent.")
31
+ print(f"Verifying models live on OpenRouter...")
32
+ status = verify_models(model, grader_model)
33
+ for mid, ok in status.items():
34
+ print(f" {mid}: {'OK' if ok is True else ok}")
35
+ if not all(v is True for v in status.values()):
36
+ print("Set EVAL_ANSWERER_MODEL / EVAL_GRADER_MODEL to current ids (https://openrouter.ai/models).")
37
+ return 1
38
+
39
+ # Low temperature: we want the model's DEFAULT instinct, not creative variance.
40
+ answerer = make_completer(model, temperature=0.1, max_tokens=700)
41
+ grader = make_completer(grader_model, system=DISTRACTOR_JUDGE_SYSTEM,
42
+ temperature=0.0, max_tokens=200)
43
+
44
+ print("\n--- Running distractor-trap eval ---")
45
+ print("(novel/fictional traps — the model can ONLY know these from the skill)")
46
+ print(f"answerer: {model} judge: {grader_model}")
47
+ rep = run_distractor_eval(CASES, answerer, grader, out_dir="./distractor_runs")
48
+
49
+ print("\n========== RESULT ==========")
50
+ print(f"answerer: {model}")
51
+ print(f"cases scored: {rep['n_scored']} (leaked excluded: {rep['n_leaked_excluded']})")
52
+ print(f"trap-avoidance WITHOUT skill: {rep['no_skill_avoidance_rate']}")
53
+ print(f"trap-avoidance WITH skill: {rep['with_skill_avoidance_rate']}")
54
+ print(f"AVOIDANCE UPLIFT: {rep['avoidance_uplift']}")
55
+ print(f"rescues (trapped->avoided): {rep['rescues']}")
56
+ print(f"regressions (avoided->trapped): {rep['regressions']}")
57
+ print("\nper-case:")
58
+ for r in rep["per_case"]:
59
+ flag = " [LEAKED]" if r["leaked"] else ""
60
+ print(f" {r['name']}: without={r['no_skill_avoided']} with={r['with_skill_avoided']}{flag}")
61
+ print(f" without: {r['no_skill_detail']}")
62
+ print(f" with: {r['with_skill_detail']}")
63
+ print("\nraw answers saved to:", rep["raw_saved_to"])
64
+ print("\nInterpretation:")
65
+ print(" - rescues > 0 : the skill genuinely helped (model fell for the trap")
66
+ print(" unaided, avoided it with the skill). This is the slug's value.")
67
+ print(" - uplift ~0, both already high : the frontier model didn't need the skill")
68
+ print(" for these traps (they were not hard enough). Report honestly.")
69
+ print(" - regressions > 0 : the skill MISLED the model. Report this too.")
70
+ return 0
71
+
72
+
73
+ if __name__ == "__main__":
74
+ sys.exit(main())
run_skill_eval.py ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Run the skill-uplift eval against real frontier models via OpenRouter.
3
+
4
+ PREREQUISITES (do these first; never hardcode the key):
5
+ export OPENROUTER_API_KEY=sk-or-...
6
+ # optional overrides if the catalog has moved:
7
+ export EVAL_ANSWERER_MODEL=anthropic/claude-opus-4.6
8
+ export EVAL_GRADER_MODEL=openai/gpt-5.1
9
+ Run from your repo root so it imports the REAL skill_builder.build_skill_md:
10
+ python run_skill_eval.py
11
+
12
+ It will: verify the models are live -> run grader calibration -> run the eval ->
13
+ write skill_eval_runs/_report.json and print a plain-language summary. It reports
14
+ whatever number comes out. A modest or zero uplift is a valid, honest result.
15
+ """
16
+
17
+ from __future__ import annotations
18
+
19
+ import json
20
+ import sys
21
+
22
+ from openrouter_client import (
23
+ make_completer, verify_models,
24
+ DEFAULT_ANSWERER_MODEL, DEFAULT_GRADER_MODEL,
25
+ )
26
+ from skill_uplift_eval import run_eval, run_calibration, GRADER_SYSTEM
27
+ from sample_cases import CASES
28
+
29
+ # A tiny human-labeled calibration set: pairs where WE judged which answer is
30
+ # better. The grader should agree. Swap in your own hand-labeled pairs.
31
+ CALIBRATION = [
32
+ {
33
+ "task": "Efficiently compute the sum of values on every root-to-leaf path's max in a tree.",
34
+ "better_answer": "Do a single post-order (bottom-up) DFS; at each node combine the max of its children's results in O(n) total.",
35
+ "worse_answer": "From each node, separately walk down every path to a leaf and take the max; simple but O(n^2) or worse.",
36
+ },
37
+ {
38
+ "task": "Find the absorption probability in a Markov chain with two absorbing states.",
39
+ "better_answer": "Write first-step equations for the transient states only, excluding absorbing rows, and solve the linear system.",
40
+ "worse_answer": "Run a large Monte Carlo simulation and estimate the fraction of runs that end in the target state.",
41
+ },
42
+ {
43
+ "task": "Avoid a stack overflow when recursing over a very deep tree in Python.",
44
+ "better_answer": "Convert the recursion to an explicit stack/iterative form, or raise the recursion limit knowingly; the depth is the real cause.",
45
+ "worse_answer": "Wrap the recursion in a try/except and hope it does not overflow.",
46
+ },
47
+ ]
48
+
49
+
50
+ def main() -> int:
51
+ answerer_model = DEFAULT_ANSWERER_MODEL
52
+ grader_model = DEFAULT_GRADER_MODEL
53
+ if answerer_model == grader_model:
54
+ print("WARNING: answerer and grader are the same model; pick different "
55
+ "models to reduce self-grading bias (set EVAL_GRADER_MODEL).")
56
+
57
+ print(f"Verifying models are live on OpenRouter...")
58
+ status = verify_models(answerer_model, grader_model)
59
+ for mid, ok in status.items():
60
+ print(f" {mid}: {'OK' if ok is True else ok}")
61
+ if not all(v is True for v in status.values()):
62
+ print("\nOne or more model ids are not in the live catalog. Set "
63
+ "EVAL_ANSWERER_MODEL / EVAL_GRADER_MODEL to current ids "
64
+ "(see https://openrouter.ai/models) and re-run.")
65
+ return 1
66
+
67
+ answerer = make_completer(answerer_model, temperature=0.2, max_tokens=900)
68
+ grader = make_completer(grader_model, system=GRADER_SYSTEM,
69
+ temperature=0.0, max_tokens=300)
70
+
71
+ print("\n--- Calibration (does the blind grader agree with human labels?) ---")
72
+ cal = run_calibration(grader, CALIBRATION)
73
+ print("grader agreement:", cal["agreement"])
74
+ for row in cal["rows"]:
75
+ print(f" agree={row['agree']} better={row['better']} worse={row['worse']} {row['task']}")
76
+
77
+ print("\n--- Running skill-uplift eval ---")
78
+ report = run_eval(CASES, answerer, grader, out_dir="./skill_eval_runs")
79
+
80
+ print("\n========== RESULT ==========")
81
+ print(f"answerer: {answerer_model} grader: {grader_model}")
82
+ print(f"calibration: {cal['agreement']}")
83
+ print(f"cases scored: {report['n_scored']} (excluded as leaked: {report['n_leaked_excluded']})")
84
+ print(f"baseline (no skill): {report['baseline_no_skill_mean']}")
85
+ print(f"with skill: {report['with_skill_mean']}")
86
+ print(f"UPLIFT: {report['uplift']}")
87
+ print(f"win / tie / loss: {report['wins']} / {report['ties']} / {report['losses']}")
88
+ print("per-case:")
89
+ for c in report["per_case"]:
90
+ flag = " [LEAKED, excluded]" if c["leaked"] else ""
91
+ print(f" {c['name']}: no={c['no_skill']} with={c['with_skill']} Δ={c['delta']}{flag}")
92
+ print("\nraw generations saved to:", report["raw_saved_to"])
93
+ print("\nReport the number above as-is. Modest or zero uplift is a valid result.")
94
+ return 0
95
+
96
+
97
+ if __name__ == "__main__":
98
+ sys.exit(main())
sample_cases.py ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Sample eval cases: each session paired with a DISTINCT held-out task in the
2
+ same class. These are illustrative; swap in your real sessions when running."""
3
+ from skill_uplift_eval import EvalCase
4
+
5
+ CASES = [
6
+ EvalCase(
7
+ name="tree_dp_to_new_tree_problem",
8
+ session_extraction={
9
+ "themes": ["tree dynamic programming", "processing order"],
10
+ "approaches_tried": [
11
+ {"approach": "top-down recursion", "why_it_failed": "recomputed each subtree per ancestor, O(n^2)"},
12
+ {"approach": "process leaves first", "why_it_failed": "a parent's value needs children finalized first"},
13
+ ],
14
+ "dead_ends": [{"position": 0.3, "what_happened": "stack overflow on deep trees"}],
15
+ "breakthroughs": [{"position": 0.85, "what_worked": "compute bottom-up, deepest nodes first, memoizing subtree results"}],
16
+ "gotchas": [
17
+ "Top-down recursion recomputes subtrees for every ancestor, making it O(n^2); compute bottom-up once instead.",
18
+ "Processing leaves first feels natural but a parent depends on its children being finalized; process deepest-first.",
19
+ ],
20
+ "sentiment_arc": {"start": "frustrated", "end": "resolved"},
21
+ },
22
+ # DISTINCT task: a different tree-DP problem, same class
23
+ task_prompt=("Given a tree where each node has a value, compute for every node "
24
+ "the maximum sum of any path from that node down to a leaf, efficiently. "
25
+ "Describe the algorithm and its time complexity. Be concise."),
26
+ answer_key_terms=["bottom-up", "post-order", "children", "O(n)"],
27
+ # if the skill literally contained THIS task's answer it'd be leakage
28
+ leak_terms=["maximum sum of any path from that node down to a leaf"],
29
+ ),
30
+ EvalCase(
31
+ name="markov_to_new_absorbing_chain",
32
+ session_extraction={
33
+ "themes": ["markov chains", "absorbing states"],
34
+ "approaches_tried": [
35
+ {"approach": "simulate many runs", "why_it_failed": "variance too high to converge on the exact value"},
36
+ {"approach": "solve the full linear system", "why_it_failed": "singular at the absorbing state's row"},
37
+ ],
38
+ "dead_ends": [{"position": 0.4, "what_happened": "matrix inversion failed, singular"}],
39
+ "breakthroughs": [{"position": 0.85, "what_worked": "drop the absorbing row, solve only the transient states' first-step equations"}],
40
+ "gotchas": [
41
+ "The absorbing-state row makes the system singular; exclude it and solve only transient states.",
42
+ "Expected hitting time is not absorption probability; do not conflate them.",
43
+ ],
44
+ "sentiment_arc": {"start": "frustrated", "end": "resolved"},
45
+ },
46
+ task_prompt=("In a random walk on states 0..4 where 0 and 4 are absorbing, find the "
47
+ "probability of being absorbed at 4 starting from state 2. Describe the "
48
+ "method, not just the number. Be concise."),
49
+ answer_key_terms=["transient", "first-step", "linear", "exclude"],
50
+ leak_terms=["absorbed at 4 starting from state 2"],
51
+ ),
52
+ ]
sample_trace.jsonl ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"type":"user","message":{"role":"user","content":"my flask api returns 500 on the /users endpoint, can you find why"}}
2
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"Let me look at the endpoint and the traceback."}]}}
3
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"tool_use","name":"Bash","input":{"command":"curl localhost:5000/users"}}]}}
4
+ {"type":"tool_result","message":{"role":"user","content":[{"type":"tool_result","content":"Error: KeyError 'email' in serialize_user, line 42"}]}}
5
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"The user dict is missing email. Let me check the DB query."}]}}
6
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"tool_use","name":"Read","input":{"file_path":"db.py"}}]}}
7
+ {"type":"tool_result","message":{"role":"user","content":[{"type":"tool_result","content":"SELECT id, name FROM users -- email column not selected"}]}}
8
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"Found it. The query never selects email but serialize_user expects it."}]}}
9
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"tool_use","name":"Edit","input":{"file_path":"db.py"}}]}}
10
+ {"type":"tool_result","message":{"role":"user","content":[{"type":"tool_result","content":"Error: column 'email' does not exist in users table"}]}}
11
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"The column itself is missing. Need a migration."}]}}
12
+ {"type":"assistant","message":{"role":"assistant","content":[{"type":"tool_use","name":"Bash","input":{"command":"flask db migrate -m add email"}}]}}
13
+ {"type":"tool_result","message":{"role":"user","content":[{"type":"tool_result","content":"migration applied, tests passed, /users returns 200 now"}]}}
scenegraph/__init__.py ADDED
File without changes
scenegraph/renderer.py ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Renderer protocol: the interface every lens implements over a SceneGraph.
3
+
4
+ Each lens (character, shell3d, atmosphere, diorama) is a Renderer. The app shows
5
+ the available lenses at upload time, the user picks one (or "all"), and the chosen
6
+ renderer(s) consume the SAME SceneGraph. Renderers are independent and degradable:
7
+ `available()` lets a renderer declare itself unavailable (e.g. a hosted generation
8
+ key is missing, or WebGL is unsupported) without breaking the others.
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ from typing import Protocol, runtime_checkable
14
+
15
+ from scene_graph import SceneGraph, SCHEMA_VERSION
16
+
17
+
18
+ @runtime_checkable
19
+ class Renderer(Protocol):
20
+ # stable id used in the lens picker and for routing
21
+ id: str
22
+ # human label + one-line description for the picker
23
+ label: str
24
+ description: str
25
+ # which SceneGraph schema versions this renderer supports
26
+ supported_versions: tuple[str, ...]
27
+ # does this renderer need a hosted/generation dependency?
28
+ requires_generation: bool
29
+
30
+ def available(self) -> tuple[bool, str]:
31
+ """Return (is_available, reason). reason is shown if unavailable, e.g.
32
+ 'set REPLICATE_API_TOKEN to enable the painted atmosphere lens'."""
33
+ ...
34
+
35
+ def render(self, scene: SceneGraph) -> dict:
36
+ """Produce the renderable output for this lens.
37
+
38
+ Returns a dict the app knows how to display, e.g.:
39
+ {"kind": "iframe", "html": "..."} # 3D / character / canvas
40
+ {"kind": "image", "url": "..."} # generated atmosphere
41
+ {"kind": "composite", "layers": [...]} # the diorama
42
+ Renderers NEVER raise for an empty/odd scene; they degrade to a sensible
43
+ minimal output and note it in the returned dict under "notes".
44
+ """
45
+ ...
46
+
47
+
48
+ def supports(renderer: "Renderer", scene: SceneGraph) -> bool:
49
+ return scene.schema_version in renderer.supported_versions
50
+
51
+
52
+ class LensRegistry:
53
+ """Holds the available lenses and resolves which to offer at upload time."""
54
+
55
+ def __init__(self) -> None:
56
+ self._lenses: list[Renderer] = []
57
+
58
+ def register(self, renderer: "Renderer") -> None:
59
+ self._lenses.append(renderer)
60
+
61
+ def all(self) -> list["Renderer"]:
62
+ return list(self._lenses)
63
+
64
+ def offerable(self, scene_version: str = SCHEMA_VERSION) -> list[dict]:
65
+ """The lens picker payload: every lens with its availability, so the UI can
66
+ show enabled/disabled states with reasons."""
67
+ out = []
68
+ for r in self._lenses:
69
+ ok, reason = r.available()
70
+ out.append({
71
+ "id": r.id, "label": r.label, "description": r.description,
72
+ "requires_generation": r.requires_generation,
73
+ "version_ok": scene_version in r.supported_versions,
74
+ "available": ok and (scene_version in r.supported_versions),
75
+ "reason": reason if not ok else "",
76
+ })
77
+ return out
scenegraph/renderers/__init__.py ADDED
File without changes
scenegraph/renderers/shell3d.py ADDED
@@ -0,0 +1,310 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Shell3D renderer: the nautilus as a real 3D object with iridescent nacre.
3
+
4
+ Consumes a SceneGraph and emits a self-contained Three.js scene (in an iframe) that
5
+ the user can orbit. The shell GEOMETRY is generated from ShellState (turns, growth,
6
+ knots, aperture) so it stays traceable; the MATERIAL is a real physically-based
7
+ iridescent surface (mother-of-pearl) whose strength comes from ShellState.iridescence
8
+ and whose colours come from the emotional palette. Knots (dead ends) become raised
9
+ nubs on the shell body; the aperture (breakthrough) glows.
10
+
11
+ No API key, no build step: Three.js is loaded from a CDN inside the iframe. WebGL is
12
+ the only requirement; available() reports that honestly.
13
+ """
14
+
15
+ from __future__ import annotations
16
+
17
+ import html as _html
18
+ import json
19
+
20
+ from scene_graph import SceneGraph
21
+
22
+
23
+ class Shell3DRenderer:
24
+ id = "shell3d"
25
+ label = "The shell, in 3D"
26
+ description = "Your nautilus as a real object you can turn in the light — iridescent nacre."
27
+ supported_versions = ("1.0",)
28
+ requires_generation = False
29
+
30
+ def available(self) -> tuple[bool, str]:
31
+ # Always offerable; WebGL support is a client-side capability we cannot
32
+ # detect server-side. The iframe degrades to a message if WebGL is absent.
33
+ return (True, "")
34
+
35
+ def render(self, scene: SceneGraph) -> dict:
36
+ params = {
37
+ "turns": scene.shell.turns,
38
+ "growth": scene.shell.growth_curve,
39
+ "knots": scene.shell.knots,
40
+ "jewels": scene.shell.jewels,
41
+ "aperture": scene.shell.aperture,
42
+ "iridescence": scene.shell.iridescence,
43
+ "palette": scene.shell.palette,
44
+ "session": scene.session_id,
45
+ }
46
+ doc = _THREE_DOC.replace("__PARAMS__", json.dumps(params))
47
+ escaped = _html.escape(doc, quote=True)
48
+ iframe = (
49
+ f'<iframe srcdoc="{escaped}" '
50
+ f'style="width:100%;height:560px;border:none;border-radius:12px;'
51
+ f'background:#07090d;" sandbox="allow-scripts allow-downloads"></iframe>'
52
+ )
53
+ return {"kind": "iframe", "html": iframe, "notes": ""}
54
+
55
+
56
+ # The Three.js scene. A log-spiral tube of growing radius (the nautilus), with a
57
+ # real iridescent material (KHR-style thin-film via onBeforeCompile is heavy; we
58
+ # use MeshPhysicalMaterial.iridescence which three r150+ supports natively), knots
59
+ # as small spheres on the centerline, and a glowing aperture sphere at the tip.
60
+ _THREE_DOC = r"""<!DOCTYPE html><html><head><meta charset="utf-8"/>
61
+ <style>
62
+ html,body{margin:0;height:100%;background:#07090d;overflow:hidden}
63
+ #fallback{color:#9aa;font:14px ui-monospace,monospace;padding:18px}
64
+ #hint{position:fixed;left:12px;bottom:10px;color:#8a93a3;
65
+ font:11px ui-monospace,monospace;opacity:.7;pointer-events:none}
66
+ </style></head><body>
67
+ <div id="fallback" style="display:none">This lens needs WebGL, which your browser
68
+ or device did not provide. Try a desktop browser to turn the shell in 3D.</div>
69
+ <div id="hint">drag to orbit · scroll to zoom</div>
70
+ <div id="ctl" style="position:fixed;right:12px;top:12px;display:flex;gap:8px;z-index:10">
71
+ <button id="resetBtn" style="font:11px ui-monospace,monospace;color:#cfe3ff;
72
+ background:rgba(20,28,40,.72);border:1px solid #2a3a52;border-radius:8px;
73
+ padding:6px 10px;cursor:pointer">reset view</button>
74
+ <button id="shotBtn" style="font:11px ui-monospace,monospace;color:#ffe9c2;
75
+ background:rgba(20,28,40,.72);border:1px solid #5a4a2a;border-radius:8px;
76
+ padding:6px 10px;cursor:pointer">save image</button>
77
+ </div>
78
+ <script type="importmap">
79
+ { "imports": {
80
+ "three": "https://cdn.jsdelivr.net/npm/three@0.160.0/build/three.module.js"
81
+ }}
82
+ </script>
83
+ <script type="module">
84
+ import * as THREE from "three";
85
+ const P = __PARAMS__;
86
+
87
+ function fail(){ document.getElementById("fallback").style.display="block";
88
+ document.getElementById("hint").style.display="none"; }
89
+
90
+ let renderer;
91
+ try {
92
+ renderer = new THREE.WebGLRenderer({antialias:true, alpha:true, preserveDrawingBuffer:true});
93
+ } catch(e){ fail(); throw e; }
94
+ if(!renderer || !renderer.getContext()){ fail(); }
95
+
96
+ renderer.setSize(innerWidth, innerHeight);
97
+ renderer.setPixelRatio(Math.min(2, devicePixelRatio||1));
98
+ document.body.appendChild(renderer.domElement);
99
+
100
+ const scene = new THREE.Scene();
101
+ const camera = new THREE.PerspectiveCamera(45, innerWidth/innerHeight, 0.1, 100);
102
+ camera.position.set(0, 0.6, 6);
103
+
104
+ // ---- lighting: a key, a warm rim, and an environment for the iridescence ----
105
+ scene.add(new THREE.AmbientLight(0x223044, 0.6));
106
+ const key = new THREE.DirectionalLight(0xffffff, 2.0); key.position.set(4,6,5); scene.add(key);
107
+ const rim = new THREE.DirectionalLight(0xffd9a0, 1.4); rim.position.set(-5,-2,-3); scene.add(rim);
108
+ const tealL = new THREE.DirectionalLight(0x66e0ff, 1.0); tealL.position.set(-3,4,2); scene.add(tealL);
109
+ const magL = new THREE.PointLight(0xff8adf, 1.4, 12); magL.position.set(2,-3,3); scene.add(magL);
110
+ // a simple gradient environment so the nacre has something to refract
111
+ const pmrem = new THREE.PMREMGenerator(renderer);
112
+ const envScene = new THREE.Scene();
113
+ const grad = new THREE.Mesh(
114
+ new THREE.SphereGeometry(50,32,32),
115
+ new THREE.ShaderMaterial({side:THREE.BackSide, uniforms:{},
116
+ vertexShader:`varying vec3 v; void main(){ v=position; gl_Position=projectionMatrix*modelViewMatrix*vec4(position,1.0);}`,
117
+ fragmentShader:`varying vec3 v; void main(){ float t=normalize(v).y*0.5+0.5;
118
+ vec3 a=vec3(0.06,0.10,0.18), b=vec3(0.45,0.40,0.55); float band=0.5+0.5*sin(normalize(v).x*8.0); vec3 col=mix(a,b,t)+band*vec3(0.10,0.06,0.14); gl_FragColor=vec4(col,1.0);}`}));
119
+ envScene.add(grad);
120
+ const envTex = pmrem.fromScene(envScene).texture;
121
+ scene.environment = envTex;
122
+
123
+ // ---- palette from the emotional arc ----
124
+ const startC = new THREE.Color(P.palette.start_hex);
125
+ const endC = new THREE.Color(P.palette.end_hex);
126
+ const accent = new THREE.Color(P.palette.accent_hex);
127
+
128
+ // ---- build the nautilus as a tube along a log spiral of growing radius ----
129
+ const TURNS = Math.max(1, P.turns);
130
+ const TOTAL = TURNS * Math.PI * 2;
131
+ const b = 0.13; // tighter -> whorls nearly touch
132
+ const pts = [];
133
+ const N = 400;
134
+ for(let i=0;i<=N;i++){
135
+ const t = i/N;
136
+ const ang = t * TOTAL;
137
+ const r = 0.10 * Math.exp(b*ang); // log spiral radius
138
+ const x = Math.cos(ang)*r;
139
+ const y = Math.sin(ang)*r;
140
+ const z = (t-0.5)*0.5*r; // slight conical rise -> 3D shell
141
+ pts.push(new THREE.Vector3(x,y,z));
142
+ }
143
+ const curve = new THREE.CatmullRomCurve3(pts);
144
+ // tube radius grows along the arm (the shell body thickens outward)
145
+ const tubeR = 0.20;
146
+ const geo = new THREE.TubeGeometry(curve, 600, tubeR, 24, false);
147
+ // taper the tube: scale each ring by its position along the arm
148
+ const pos = geo.attributes.position;
149
+ const tmp = new THREE.Vector3();
150
+ for(let i=0;i<pos.count;i++){
151
+ // approximate t by ring index
152
+ const ring = Math.floor(i/ (24+1));
153
+ const t = ring/600;
154
+ const grow = 0.12 + 2.4*t*t; // fat rounded body whorl at the rim
155
+ tmp.fromBufferAttribute(pos,i);
156
+ // pull toward centerline point then push out scaled
157
+ const cp = curve.getPoint(Math.min(1,t));
158
+ tmp.sub(cp).multiplyScalar(grow).add(cp);
159
+ pos.setXYZ(i, tmp.x, tmp.y, tmp.z);
160
+ }
161
+ geo.computeVertexNormals();
162
+
163
+ // ---- vertex-colored gradient along the arm (start -> end sentiment) ----
164
+ const colors = [];
165
+ for(let i=0;i<pos.count;i++){
166
+ const ring = Math.floor(i/(24+1));
167
+ const t = Math.min(1, ring/600);
168
+ const c = startC.clone().lerp(endC, t);
169
+ colors.push(c.r,c.g,c.b);
170
+ }
171
+ geo.setAttribute("color", new THREE.Float32BufferAttribute(colors,3));
172
+
173
+ // ---- iridescent nacre material ----
174
+ const mat = new THREE.MeshPhysicalMaterial({
175
+ vertexColors:true,
176
+ metalness:0.05, roughness:0.18,
177
+ clearcoat:1.0, clearcoatRoughness:0.18,
178
+ iridescence: Math.max(0.55, P.iridescence),
179
+ iridescenceIOR:1.6,
180
+ iridescenceThicknessRange:[200, 900],
181
+ envMapIntensity:1.6,
182
+ sheen:0.6, sheenColor:accent,
183
+ });
184
+ const shell = new THREE.Mesh(geo, mat);
185
+ scene.add(shell);
186
+
187
+ // ---- knots: raised nubs at each dead-end position along the arm ----
188
+ const knotMat = new THREE.MeshPhysicalMaterial({color:0x2a1d12, roughness:0.3, clearcoat:1.0, iridescence:0.8, iridescenceIOR:1.4});
189
+ for(const k of (P.knots||[])){
190
+ const cp = curve.getPoint(Math.min(1, k.t));
191
+ const s = new THREE.Mesh(new THREE.SphereGeometry(0.10+0.06*(k.severity||0.5),20,20), knotMat);
192
+ s.position.copy(cp); shell.add(s);
193
+ }
194
+
195
+ // ---- jewels: iridescent beads on the rim at each gotcha position ----
196
+ // gotchas sit "along the rim", so offset each bead outward from the centerline
197
+ // along the local radial direction (from shell center to the curve point).
198
+ const jewelMat = new THREE.MeshPhysicalMaterial({
199
+ color: accent.clone().lerp(new THREE.Color(0xffffff), 0.35),
200
+ metalness:0.0, roughness:0.12, clearcoat:1.0, clearcoatRoughness:0.1,
201
+ iridescence:1.0, iridescenceIOR:1.5, iridescenceThicknessRange:[150,700],
202
+ envMapIntensity:1.8,
203
+ });
204
+ for(const j of (P.jewels||[])){
205
+ const cp = curve.getPoint(Math.min(1, j.t));
206
+ // radial direction in the shell's xy-plane (the spiral grows outward in xy)
207
+ const radial = new THREE.Vector3(cp.x, cp.y, 0);
208
+ if(radial.lengthSq() < 1e-6) radial.set(1,0,0); else radial.normalize();
209
+ const bead = new THREE.Mesh(new THREE.SphereGeometry(0.085,18,18), jewelMat);
210
+ // local tube radius grows along the arm; push the bead just past the surface
211
+ const localGrow = 0.12 + 2.4*j.t*j.t;
212
+ const off = tubeR*localGrow + 0.05;
213
+ bead.position.copy(cp).addScaledVector(radial, off);
214
+ shell.add(bead);
215
+ }
216
+
217
+ // ---- aperture: a glowing sphere at the breakthrough position ----
218
+ const ap = P.aperture||{t:0.95,intensity:0.8};
219
+ const apPos = curve.getPoint(Math.min(1, ap.t));
220
+ const apMat = new THREE.MeshBasicMaterial({color:accent});
221
+ const apMesh = new THREE.Mesh(new THREE.SphereGeometry(0.12+0.12*ap.intensity,24,24), apMat);
222
+ apMesh.position.copy(apPos); shell.add(apMesh);
223
+ const apLight = new THREE.PointLight(accent.getHex(), 2.0*ap.intensity, 4);
224
+ apLight.position.copy(apPos); shell.add(apLight);
225
+
226
+ // frame the shell
227
+ const box = new THREE.Box3().setFromObject(shell);
228
+ const center = box.getCenter(new THREE.Vector3());
229
+ shell.position.sub(center);
230
+ const size = box.getSize(new THREE.Vector3()).length();
231
+ camera.position.set(0, size*0.15, size*1.1);
232
+ camera.lookAt(0,0,0);
233
+
234
+ // ---- minimal orbit controls (no extra dependency) ----
235
+ let drag=false, px=0, py=0, rotY=0.2, rotX=-0.15, dist=size*1.1;
236
+ const el = renderer.domElement;
237
+ el.addEventListener("pointerdown", e=>{drag=true;px=e.clientX;py=e.clientY;});
238
+ addEventListener("pointerup", ()=>drag=false);
239
+ addEventListener("pointermove", e=>{ if(!drag)return;
240
+ rotY += (e.clientX-px)*0.008; rotX += (e.clientY-py)*0.008;
241
+ rotX=Math.max(-1.2,Math.min(1.2,rotX)); px=e.clientX; py=e.clientY; });
242
+ el.addEventListener("wheel", e=>{ dist*=(1+Math.sign(e.deltaY)*0.08);
243
+ dist=Math.max(size*0.5,Math.min(size*3,dist)); e.preventDefault(); }, {passive:false});
244
+
245
+ // pinch-zoom: two-finger distance drives the same `dist` as the wheel
246
+ let pinchD = 0;
247
+ el.addEventListener("touchstart", e=>{
248
+ if(e.touches.length===2){
249
+ const dx=e.touches[0].clientX-e.touches[1].clientX;
250
+ const dy=e.touches[0].clientY-e.touches[1].clientY;
251
+ pinchD = Math.hypot(dx,dy);
252
+ drag = false; // suppress orbit while pinching
253
+ }
254
+ }, {passive:false});
255
+ el.addEventListener("touchmove", e=>{
256
+ if(e.touches.length===2 && pinchD>0){
257
+ const dx=e.touches[0].clientX-e.touches[1].clientX;
258
+ const dy=e.touches[0].clientY-e.touches[1].clientY;
259
+ const d = Math.hypot(dx,dy);
260
+ const ratio = pinchD / Math.max(1,d); // fingers apart -> zoom in
261
+ dist *= ratio;
262
+ dist = Math.max(size*0.5, Math.min(size*3, dist));
263
+ pinchD = d;
264
+ e.preventDefault();
265
+ }
266
+ }, {passive:false});
267
+ el.addEventListener("touchend", e=>{ if(e.touches.length<2) pinchD=0; });
268
+
269
+ addEventListener("resize", ()=>{ camera.aspect=innerWidth/innerHeight;
270
+ camera.updateProjectionMatrix(); renderer.setSize(innerWidth,innerHeight); });
271
+
272
+ // ---- Tier 1: reset view ----
273
+ // capture the framing defaults so reset restores the exact initial pose
274
+ const DEF_rotY = rotY, DEF_rotX = rotX, DEF_dist = dist;
275
+ const resetBtn = document.getElementById("resetBtn");
276
+ if (resetBtn) resetBtn.addEventListener("click", ()=>{
277
+ rotY = DEF_rotY; rotX = DEF_rotX; dist = DEF_dist;
278
+ });
279
+
280
+ // ---- Tier 1: save image (screenshot) ----
281
+ // WebGL clears its drawing buffer after render, so toDataURL() on a stale frame
282
+ // is blank. We render ONE fresh frame immediately before reading the pixels, in
283
+ // the same tick, which captures correctly without preserveDrawingBuffer.
284
+ const shotBtn = document.getElementById("shotBtn");
285
+ if (shotBtn) shotBtn.addEventListener("click", ()=>{
286
+ try {
287
+ renderer.render(scene, camera); // fresh frame, same tick
288
+ const url = renderer.domElement.toDataURL("image/png");
289
+ const a = document.createElement("a");
290
+ a.href = url;
291
+ a.download = "turboskillslug-shell-" + (P.session || "session") + ".png";
292
+ document.body.appendChild(a); a.click(); a.remove();
293
+ } catch (e) {
294
+ // never break the lens if capture fails (some browsers block tainted canvases)
295
+ console.warn("screenshot failed:", e);
296
+ }
297
+ });
298
+
299
+ function loop(){
300
+ requestAnimationFrame(loop);
301
+ // gentle auto-spin when idle, plus user orbit
302
+ if(!drag) rotY += 0.0016;
303
+ const cx=Math.sin(rotY)*Math.cos(rotX)*dist;
304
+ const cy=Math.sin(rotX)*dist;
305
+ const cz=Math.cos(rotY)*Math.cos(rotX)*dist;
306
+ camera.position.set(cx,cy,cz); camera.lookAt(0,0,0);
307
+ renderer.render(scene, camera);
308
+ }
309
+ loop();
310
+ </script></body></html>"""
scenegraph/scene_builder.py ADDED
@@ -0,0 +1,194 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ build_scene_graph(extraction) -> SceneGraph
3
+
4
+ Deterministically derives the full semantic scene from a session extraction. Every
5
+ field traces to real data; same extraction always yields the same graph. This is
6
+ the ONLY place extraction-shape knowledge lives — renderers never touch raw
7
+ extraction (except via the escape hatch).
8
+ """
9
+
10
+ from __future__ import annotations
11
+
12
+ import colorsys
13
+ import hashlib
14
+ import json
15
+
16
+ from scene_graph import (
17
+ SceneGraph, SlugState, ShellState, ArcState, BattleState, SceneEnv,
18
+ SCHEMA_VERSION, _norm_sentiment, _va, _clamp01,
19
+ )
20
+
21
+
22
+ def _session_id(extraction: dict) -> str:
23
+ blob = json.dumps(extraction, sort_keys=True, default=str).encode()
24
+ return hashlib.sha256(blob).hexdigest()[:16]
25
+
26
+
27
+ def _hex_from_va(valence: float, energy: float) -> str:
28
+ """Map valence/energy to a hue/sat/light color. Negative=cool, positive=warm."""
29
+ # hue: 0.0 (red) .. 0.33 (green) .. 0.58 (blue). Warm/positive -> gold-green,
30
+ # cool/negative -> blue. Map valence -1..1 to hue 0.58..0.13.
31
+ hue = 0.58 - (valence + 1) / 2 * 0.45
32
+ sat = 0.35 + energy * 0.4
33
+ light = 0.42 + (valence + 1) / 2 * 0.18
34
+ r, g, b = colorsys.hls_to_rgb(hue, light, sat)
35
+ return f"#{int(r*255):02x}{int(g*255):02x}{int(b*255):02x}"
36
+
37
+
38
+ def _build_slug(end_sent: str, dead_ends: list, arc_end_va) -> SlugState:
39
+ val, en = arc_end_va
40
+ scars = len(dead_ends)
41
+ # expression/pose/eyes derived from valence+energy bands
42
+ if val >= 0.6:
43
+ expr, pose, eye = "elated", "triumphant", "bright"
44
+ elif val >= 0.2:
45
+ expr, pose, eye = "hopeful", "alert", "open"
46
+ elif val >= -0.2:
47
+ expr, pose, eye = "focused", "neutral", "narrow"
48
+ elif en <= 0.25:
49
+ expr, pose, eye = "weary", "slumped", "heavy"
50
+ else:
51
+ expr, pose, eye = "wary", "neutral", "narrow"
52
+ return SlugState(
53
+ mood=_norm_sentiment(end_sent), valence=round(val, 3), energy=round(en, 3),
54
+ scars=scars, expression=expr, pose=pose, eye_state=eye,
55
+ )
56
+
57
+
58
+ def _build_shell(duration, approaches, dead_ends, gotchas, breakthroughs,
59
+ start_va, end_va) -> ShellState:
60
+ turns = max(1.0, min(6.0, (duration or 30) / 30.0)) # ~1 turn per 30 min, cap 6
61
+ n = len(approaches) + len(dead_ends)
62
+ growth_curve = "gentle" if n <= 2 else ("steady" if n <= 5 else "steep")
63
+ knots = [{"t": _clamp01(float(d.get("position", 0.5))),
64
+ "severity": 0.5} for d in dead_ends if isinstance(d, dict)]
65
+ jewels = [{"t": _clamp01((i + 1) / (len(gotchas) + 1))}
66
+ for i in range(len(gotchas))]
67
+ if breakthroughs:
68
+ b = breakthroughs[-1]
69
+ ap_t = _clamp01(float(b.get("position", 0.9))) if isinstance(b, dict) else 0.9
70
+ else:
71
+ ap_t = 0.9
72
+ palette = {
73
+ "start_hex": _hex_from_va(*start_va),
74
+ "end_hex": _hex_from_va(*end_va),
75
+ "accent_hex": "#e6c870",
76
+ }
77
+ # iridescence stronger when the arc swings far (struggle -> triumph reads richer)
78
+ swing = abs(end_va[0] - start_va[0])
79
+ return ShellState(
80
+ turns=round(turns, 2), growth_curve=growth_curve, knots=knots, jewels=jewels,
81
+ aperture={"t": ap_t, "intensity": round(0.6 + 0.4 * swing / 2, 3)},
82
+ palette=palette, iridescence=round(_clamp01(0.4 + swing / 2), 3),
83
+ )
84
+
85
+
86
+ def _build_arc(start_sent, end_sent, dead_ends, breakthroughs, gotchas) -> ArcState:
87
+ beats = []
88
+ for d in dead_ends:
89
+ if isinstance(d, dict):
90
+ t = _clamp01(float(d.get("position", 0.5)))
91
+ v, e = -0.6, 0.4
92
+ beats.append({"t": t, "kind": "dead_end", "valence": v, "energy": e,
93
+ "label": str(d.get("what_happened", "a wall"))[:60]})
94
+ for i, g in enumerate(gotchas):
95
+ beats.append({"t": _clamp01(0.3 + i * 0.1), "kind": "gotcha",
96
+ "valence": -0.3, "energy": 0.5, "label": str(g)[:60]})
97
+ for b in breakthroughs:
98
+ if isinstance(b, dict):
99
+ t = _clamp01(float(b.get("position", 0.9)))
100
+ beats.append({"t": t, "kind": "breakthrough", "valence": 0.8, "energy": 0.8,
101
+ "label": str(b.get("what_worked", "it gave way"))[:60]})
102
+ beats.sort(key=lambda x: x["t"])
103
+ # sample a tension curve (0..1) across 16 points: high during dead-end clusters,
104
+ # dropping at the breakthrough. Simple kernel sum.
105
+ samples = []
106
+ for k in range(16):
107
+ t = k / 15
108
+ tension = 0.2
109
+ for be in beats:
110
+ d = abs(be["t"] - t)
111
+ w = max(0.0, 1 - d * 6)
112
+ if be["kind"] in ("dead_end", "gotcha"):
113
+ tension += 0.6 * w
114
+ elif be["kind"] == "breakthrough":
115
+ tension -= 0.5 * w
116
+ samples.append(round(_clamp01(tension), 3))
117
+ return ArcState(start=_norm_sentiment(start_sent), end=_norm_sentiment(end_sent),
118
+ beats=beats, tension_curve=samples)
119
+
120
+
121
+ def _build_battle(approaches, dead_ends, gotchas, breakthroughs) -> BattleState:
122
+ ferocity = _clamp01(0.3 + 0.15 * len(dead_ends))
123
+ fallen = [{"t": _clamp01(float(d.get("position", 0.5)))}
124
+ for d in dead_ends if isinstance(d, dict)]
125
+ archers = [{"t": _clamp01(0.5 + i * 0.08)} for i in range(len(gotchas))]
126
+ if breakthroughs:
127
+ b = breakthroughs[-1]
128
+ dragon = {"present": True,
129
+ "t": _clamp01(float(b.get("position", 0.9))) if isinstance(b, dict) else 0.9,
130
+ "scale": _clamp01(0.7 + 0.3 * min(1, len(breakthroughs)))}
131
+ else:
132
+ dragon = {"present": False, "t": 1.0, "scale": 0.0}
133
+ return BattleState(
134
+ general={"present": bool(approaches)},
135
+ adversary={"present": True, "ferocity": round(ferocity, 3)},
136
+ fallen=fallen, archers=archers, dragon=dragon,
137
+ )
138
+
139
+
140
+ def _build_env(start_va, end_va, dead_ends, themes) -> SceneEnv:
141
+ # time of day from the arc's emotional journey
142
+ sv, ev = start_va[0], end_va[0]
143
+ if ev >= 0.5 and sv < 0:
144
+ tod = "dawn" # struggle -> triumph reads as a sunrise
145
+ elif ev >= 0.4:
146
+ tod = "day"
147
+ elif ev <= -0.3:
148
+ tod = "night"
149
+ else:
150
+ tod = "dusk"
151
+ # weather from struggle density
152
+ struggle = len(dead_ends)
153
+ weather = "clear" if struggle <= 1 else ("overcast" if struggle <= 3 else
154
+ ("rain" if struggle <= 5 else "storm"))
155
+ mood_tags = []
156
+ mood_tags.append({"dawn": "first light, hopeful", "day": "bright, open",
157
+ "dusk": "golden hour, reflective", "night": "deep blue, quiet"}[tod])
158
+ mood_tags.append({"clear": "still air", "overcast": "soft grey sky",
159
+ "rain": "gentle rain", "storm": "dramatic storm"}[weather])
160
+ if themes:
161
+ mood_tags.append(f"a sense of {themes[0]}")
162
+ palette = {
163
+ "sky_hex": _hex_from_va(*end_va),
164
+ "ground_hex": "#1a1410",
165
+ "accent_hex": "#e6c870",
166
+ }
167
+ return SceneEnv(time_of_day=tod, weather=weather, mood_tags=mood_tags, palette=palette)
168
+
169
+
170
+ def build_scene_graph(extraction: dict) -> SceneGraph:
171
+ ex = extraction or {}
172
+ themes = ex.get("themes") or []
173
+ approaches = ex.get("approaches_tried") or []
174
+ dead_ends = ex.get("dead_ends") or []
175
+ gotchas = ex.get("gotchas") or []
176
+ breakthroughs = ex.get("breakthroughs") or []
177
+ arc = ex.get("sentiment_arc") or {}
178
+ start_sent, end_sent = arc.get("start"), arc.get("end")
179
+ duration = ex.get("duration_minutes")
180
+
181
+ start_va, end_va = _va(start_sent), _va(end_sent)
182
+
183
+ return SceneGraph(
184
+ schema_version=SCHEMA_VERSION,
185
+ session_id=_session_id(ex),
186
+ duration_minutes=float(duration or 0),
187
+ themes=themes,
188
+ slug=_build_slug(end_sent, dead_ends, end_va),
189
+ shell=_build_shell(duration, approaches, dead_ends, gotchas, breakthroughs, start_va, end_va),
190
+ arc=_build_arc(start_sent, end_sent, dead_ends, breakthroughs, gotchas),
191
+ battle=_build_battle(approaches, dead_ends, gotchas, breakthroughs),
192
+ env=_build_env(start_va, end_va, dead_ends, themes),
193
+ extraction=ex,
194
+ )
scenegraph/scene_graph.py ADDED
@@ -0,0 +1,134 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ SceneGraph: the deterministic semantic layer of a TurboSkillSlug session.
3
+
4
+ This is the CONTRACT that every renderer reads. It is built once, deterministically,
5
+ from a session extraction, and it is renderer-agnostic. The four lenses (character,
6
+ 3D shell, generative atmosphere, full diorama) all consume the SAME SceneGraph, so:
7
+
8
+ - every visible element still traces to real session data (the core promise),
9
+ - renderers are independent and individually degradable,
10
+ - a new renderer can be added without touching extraction or the others.
11
+
12
+ Design rules:
13
+ - NOTHING here renders. No SVG, no canvas, no shader. Only structured meaning.
14
+ - Every field is DERIVED from the extraction (or a stable default), so the graph
15
+ is reproducible: same session -> same SceneGraph.
16
+ - Values are normalized and renderer-friendly (0..1 scalars, named enums, hex
17
+ colors) so each renderer can map them without re-interpreting raw extraction.
18
+ - The graph is versioned. Renderers declare which SCHEMA_VERSION they support.
19
+ """
20
+
21
+ from __future__ import annotations
22
+
23
+ import colorsys
24
+ import hashlib
25
+ from dataclasses import dataclass, field, asdict
26
+ from typing import Any
27
+
28
+ SCHEMA_VERSION = "1.0"
29
+
30
+ # ---- canonical sentiment vocabulary (extraction may use freeform; we map it) ----
31
+ # Each sentiment maps to (valence -1..1, energy 0..1) so renderers can blend.
32
+ _SENTIMENT_VA = {
33
+ "frustrated": (-0.7, 0.7), "stuck": (-0.6, 0.3), "exhausted": (-0.4, 0.1),
34
+ "anxious": (-0.5, 0.6), "confused": (-0.4, 0.4), "focused": (0.1, 0.6),
35
+ "curious": (0.4, 0.6), "determined": (0.3, 0.7), "calm": (0.3, 0.3),
36
+ "relieved": (0.6, 0.3), "satisfied": (0.7, 0.4), "resolved": (0.7, 0.5),
37
+ "joyful": (0.9, 0.8), "delighted": (0.9, 0.7), "triumphant": (1.0, 0.9),
38
+ "neutral": (0.0, 0.4),
39
+ }
40
+
41
+
42
+ def _norm_sentiment(name: str | None) -> str:
43
+ if not name:
44
+ return "neutral"
45
+ n = str(name).strip().lower()
46
+ if n in _SENTIMENT_VA:
47
+ return n
48
+ # nearest by substring, else neutral
49
+ for k in _SENTIMENT_VA:
50
+ if k in n or n in k:
51
+ return k
52
+ return "neutral"
53
+
54
+
55
+ def _va(name: str) -> tuple[float, float]:
56
+ return _SENTIMENT_VA[_norm_sentiment(name)]
57
+
58
+
59
+ def _clamp01(x: float) -> float:
60
+ return max(0.0, min(1.0, float(x)))
61
+
62
+
63
+ # ---------------------------------------------------------------------------
64
+ # Sub-structures
65
+ # ---------------------------------------------------------------------------
66
+
67
+ @dataclass
68
+ class SlugState:
69
+ """How the slug itself looks and feels — for the CHARACTER renderer."""
70
+ mood: str # canonical end sentiment
71
+ valence: float # -1..1 (sad -> happy)
72
+ energy: float # 0..1 (drained -> lively)
73
+ scars: int # = number of dead ends (visible marks on the slug)
74
+ expression: str # enum: weary|wary|focused|hopeful|elated
75
+ pose: str # enum: slumped|neutral|alert|triumphant
76
+ eye_state: str # enum: heavy|narrow|open|bright
77
+
78
+
79
+ @dataclass
80
+ class ShellState:
81
+ """The nautilus geometry parameters — used by ALL renderers (SVG/3D/guide)."""
82
+ turns: float # number of spiral revolutions (from duration)
83
+ growth_curve: str # enum: gentle|steady|steep (pacing of the session)
84
+ knots: list[dict] # [{t:0..1, severity:0..1}] dead ends along the arm
85
+ jewels: list[dict] # [{t:0..1}] gotchas along the rim
86
+ aperture: dict # {t:0..1, intensity:0..1} the breakthrough
87
+ palette: dict # {start_hex, end_hex, accent_hex}
88
+ iridescence: float # 0..1 how strongly nacre shifts (for 3D shader)
89
+
90
+
91
+ @dataclass
92
+ class ArcState:
93
+ """The emotional timeline — for the SCORE and ATMOSPHERE renderers."""
94
+ start: str
95
+ end: str
96
+ beats: list[dict] # ordered [{t:0..1, kind, valence, energy, label}]
97
+ tension_curve: list[float] # sampled 0..1 tension over the session (for music)
98
+
99
+
100
+ @dataclass
101
+ class BattleState:
102
+ """The byobu battle cast — for the painted/animated battle layers."""
103
+ general: dict # {present:bool}
104
+ adversary: dict # {present:bool, ferocity:0..1}
105
+ fallen: list[dict] # [{t:0..1}] one per dead end
106
+ archers: list[dict] # [{t:0..1}] one per gotcha
107
+ dragon: dict # {present:bool, t:0..1, scale:0..1} breakthrough
108
+
109
+
110
+ @dataclass
111
+ class SceneEnv:
112
+ """The surrounding scene mood — for the BACKDROP/diffusion renderer."""
113
+ time_of_day: str # dawn|day|dusk|night (from arc shape)
114
+ weather: str # clear|overcast|rain|storm (from struggle level)
115
+ mood_tags: list[str] # prompt-ready descriptors for generation
116
+ palette: dict # {sky_hex, ground_hex, accent_hex}
117
+
118
+
119
+ @dataclass
120
+ class SceneGraph:
121
+ schema_version: str
122
+ session_id: str # stable hash of the extraction (repro + cache key)
123
+ duration_minutes: float
124
+ themes: list[str]
125
+ slug: SlugState
126
+ shell: ShellState
127
+ arc: ArcState
128
+ battle: BattleState
129
+ env: SceneEnv
130
+ # raw escape hatch: renderers that want a field we didn't surface can read this
131
+ extraction: dict = field(default_factory=dict)
132
+
133
+ def to_dict(self) -> dict[str, Any]:
134
+ return asdict(self)
session_genre.py ADDED
@@ -0,0 +1,124 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Pure-regex session genre detection for TurboSkillSlug."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import re
6
+
7
+
8
+ GENRES = ("debugging", "exploration", "authoring", "feature", "refactor", "setup")
9
+
10
+ _PATTERNS = {
11
+ "debugging": (
12
+ r"\b(debug|bug|fix|error|traceback|exception|fail(?:ed|ing)?|broken|crash|"
13
+ r"regression|why.*not|doesn't work|not working|stuck|issue)\b",
14
+ ),
15
+ "exploration": (
16
+ r"\b(explore|understand|inspect|go through|read through|map out|figure out|"
17
+ r"investigate|familiarize|what does|how does|codebase|repo)\b",
18
+ ),
19
+ "authoring": (
20
+ r"\b(write|draft|document|readme|docs|blog|article|copy|proposal|report|"
21
+ r"build log|submission|explain|summarize)\b",
22
+ ),
23
+ "feature": (
24
+ r"\b(add|implement|wire|build|create|ship|feature|endpoint|button|ui|"
25
+ r"component|integrate)\b",
26
+ ),
27
+ "refactor": (
28
+ r"\b(refactor|cleanup|clean up|reorganize|simplify|rename|dedupe|extract "
29
+ r"helper|move .* into)\b",
30
+ ),
31
+ "setup": (
32
+ r"\b(set ?up|install|configure|config|dependency|requirements|deploy|"
33
+ r"environment|env var|ci|workflow|initialize|scaffold)\b",
34
+ ),
35
+ }
36
+
37
+ _FRAMES = {
38
+ "debugging": (
39
+ "Witness the struggle: the failed approaches, the exact symptoms, the "
40
+ "turning point, and what finally fixed the problem."
41
+ ),
42
+ "exploration": (
43
+ "Witness the discoveries: the non-obvious facts learned about the "
44
+ "codebase, the map that emerged, and the clearest insight."
45
+ ),
46
+ "authoring": (
47
+ "Witness the decisions: what was clarified, what false assumptions were "
48
+ "caught, and what document or explanation was delivered."
49
+ ),
50
+ "feature": (
51
+ "Witness what was built: the new behavior, the integration points, the "
52
+ "naive paths that would break, and the delivered feature."
53
+ ),
54
+ "refactor": (
55
+ "Witness the reshaping: what moved, what got simpler, what invariants "
56
+ "had to hold, and what could have broken."
57
+ ),
58
+ "setup": (
59
+ "Witness the setup path: the configuration choices, dependency traps, "
60
+ "environment gotchas, and the final working baseline."
61
+ ),
62
+ }
63
+
64
+ _LEGENDS = {
65
+ "debugging": {
66
+ "knot": "dead ends and failed approaches",
67
+ "jewel": "verification gotchas",
68
+ "aperture": "the breakthrough that fixed it",
69
+ },
70
+ "exploration": {
71
+ "knot": "confusions or branches that did not explain the system",
72
+ "jewel": "discoveries about the codebase",
73
+ "aperture": "the clearest insight",
74
+ },
75
+ "authoring": {
76
+ "knot": "false assumptions caught",
77
+ "jewel": "decisions worth preserving",
78
+ "aperture": "the document delivered",
79
+ },
80
+ "feature": {
81
+ "knot": "implementation paths that would break",
82
+ "jewel": "integration gotchas",
83
+ "aperture": "the feature working",
84
+ },
85
+ "refactor": {
86
+ "knot": "risky seams and avoided regressions",
87
+ "jewel": "invariants worth remembering",
88
+ "aperture": "the simpler shape that remained",
89
+ },
90
+ "setup": {
91
+ "knot": "environment traps",
92
+ "jewel": "configuration details worth saving",
93
+ "aperture": "the working baseline",
94
+ },
95
+ }
96
+
97
+
98
+ def detect_genre(first_instruction: str, transcript: str = "") -> str:
99
+ """Detect the session genre with deterministic regex scoring."""
100
+ text = f"{first_instruction}\n{transcript[:3000]}".lower()
101
+ scores = {genre: 0 for genre in GENRES}
102
+ for genre, patterns in _PATTERNS.items():
103
+ for pattern in patterns:
104
+ scores[genre] += len(re.findall(pattern, text, re.IGNORECASE))
105
+
106
+ # First instruction is more predictive than the full transcript.
107
+ first = first_instruction.lower()
108
+ for genre, patterns in _PATTERNS.items():
109
+ for pattern in patterns:
110
+ if re.search(pattern, first, re.IGNORECASE):
111
+ scores[genre] += 3
112
+
113
+ best = max(GENRES, key=lambda genre: (scores[genre], -GENRES.index(genre)))
114
+ return best if scores[best] > 0 else "feature"
115
+
116
+
117
+ def frame_for(genre: str) -> str:
118
+ """Return the witness frame for a genre."""
119
+ return _FRAMES.get(genre, _FRAMES["feature"])
120
+
121
+
122
+ def shell_legend(genre: str) -> dict[str, str]:
123
+ """Return genre-specific meanings for shell features."""
124
+ return dict(_LEGENDS.get(genre, _LEGENDS["feature"]))
shell.py ADDED
@@ -0,0 +1,496 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Shell generator for TurboSkillSlug.
3
+
4
+ Every shell is unique because every session is unique. The patterns are
5
+ derived from the session, not decorative: duration shapes the spiral,
6
+ dead ends leave dark knots, breakthroughs open the aperture, gotchas
7
+ become iridescent jewels, and the sentiment arc drives the color story.
8
+
9
+ Visual techniques:
10
+ - SVG feTurbulence for mother-of-pearl (nacre) texture
11
+ - feGaussianBlur composites for glow on aperture and jewels
12
+ - Multi-stop radial gradients for depth
13
+ - HSL-derived color harmonies from sentiment
14
+ - Cubic bezier smoothing on the spiral body
15
+ - Layered transparency for organic depth
16
+
17
+ Staged growth:
18
+ generate_shell_svg(features, growth=g) renders the shell at completion
19
+ fraction g in [0,1]. At g<1 the spiral is physically shorter, only the
20
+ knots and jewels up to that point have formed, and the aperture (the
21
+ breakthrough mouth) stays closed until g reaches 1.0 - the breakthrough
22
+ is the last thing to open. growth=1.0 is byte-identical in intent to the
23
+ original single-shot render.
24
+ """
25
+
26
+ import colorsys
27
+ import math
28
+ import random
29
+
30
+ from shell_battle import build_battle_layer
31
+ from typing import Any
32
+
33
+
34
+ # ---------------------------------------------------------------------------
35
+ # HSL-based color system derived from sentiment
36
+ # ---------------------------------------------------------------------------
37
+
38
+ SENTIMENT_HUES: dict[str, float] = {
39
+ "confused": 0.75, # violet
40
+ "focused": 0.61, # deep blue
41
+ "frustrated": 0.0, # red
42
+ "curious": 0.50, # teal
43
+ "resolved": 0.39, # green
44
+ "joyful": 0.12, # warm gold
45
+ "exhausted": 0.67, # grey-blue
46
+ "enlightened": 0.14, # pale gold
47
+ }
48
+
49
+
50
+ def _hue_for(sentiment: str, fallback: float = 0.75) -> float:
51
+ return SENTIMENT_HUES.get(sentiment.lower().strip(), fallback)
52
+
53
+
54
+ def _hsl_to_hex(h: float, s: float, l: float) -> str:
55
+ r, g, b = colorsys.hls_to_rgb(h, l, s)
56
+ return f"#{int(r*255):02x}{int(g*255):02x}{int(b*255):02x}"
57
+
58
+
59
+ def _palette(start_sent: str, end_sent: str) -> dict[str, str]:
60
+ """Derive a harmonious 6-color palette from start/end sentiment."""
61
+ h0 = _hue_for(start_sent, 0.75)
62
+ h1 = _hue_for(end_sent, 0.39)
63
+ return {
64
+ "bg_deep": _hsl_to_hex(h0, 0.30, 0.06),
65
+ "bg_mid": _hsl_to_hex(h0, 0.25, 0.12),
66
+ "body_dark": _hsl_to_hex(h0, 0.35, 0.30),
67
+ "body_light": _hsl_to_hex(h1, 0.40, 0.50),
68
+ "accent": _hsl_to_hex(h1, 0.55, 0.65),
69
+ "highlight": _hsl_to_hex(h1, 0.30, 0.85),
70
+ "aperture_core": _hsl_to_hex(h1, 0.20, 0.92),
71
+ "jewel": _hsl_to_hex((h0 + h1) / 2, 0.60, 0.70),
72
+ "knot": _hsl_to_hex(h0, 0.40, 0.10),
73
+ "septa": _hsl_to_hex(h0, 0.20, 0.15),
74
+ }
75
+
76
+
77
+ # ---------------------------------------------------------------------------
78
+ # Spiral geometry
79
+ # ---------------------------------------------------------------------------
80
+
81
+ def _log_spiral(cx: float, cy: float, n_turns: float,
82
+ r0: float, r_max: float, n_points: int = 500):
83
+ """Generate points along a logarithmic spiral with normals."""
84
+ t_max = 2 * math.pi * max(n_turns, 0.5)
85
+ b = math.log(max(r_max, r0 + 1) / r0) / t_max
86
+ pts = []
87
+ for i in range(n_points):
88
+ t = (i / (n_points - 1)) * t_max
89
+ r = r0 * math.exp(b * t)
90
+ x = cx + r * math.cos(t)
91
+ y = cy + r * math.sin(t)
92
+ tangent = t + math.atan2(1, b)
93
+ normal = tangent + math.pi / 2
94
+ pts.append((x, y, t, r, normal))
95
+ return pts
96
+
97
+
98
+ def _smooth_path(pts: list, closed: bool = False) -> str:
99
+ """Convert points to a smooth SVG path using cubic bezier approximation."""
100
+ if len(pts) < 2:
101
+ return ""
102
+ d = [f"M {pts[0][0]:.1f} {pts[0][1]:.1f}"]
103
+ tension = 0.3
104
+ for i in range(1, len(pts)):
105
+ p0 = pts[max(i - 2, 0)]
106
+ p1 = pts[i - 1]
107
+ p2 = pts[i]
108
+ p3 = pts[min(i + 1, len(pts) - 1)]
109
+ cp1x = p1[0] + (p2[0] - p0[0]) * tension
110
+ cp1y = p1[1] + (p2[1] - p0[1]) * tension
111
+ cp2x = p2[0] - (p3[0] - p1[0]) * tension
112
+ cp2y = p2[1] - (p3[1] - p1[1]) * tension
113
+ d.append(f"C {cp1x:.1f} {cp1y:.1f} {cp2x:.1f} {cp2y:.1f} {p2[0]:.1f} {p2[1]:.1f}")
114
+ if closed:
115
+ d.append("Z")
116
+ return " ".join(d)
117
+
118
+
119
+ def _band_path_smooth(inner: list, outer: list) -> str:
120
+ """Closed smooth path for the shell body (outer forward, inner reverse)."""
121
+ fwd = _smooth_path(outer)
122
+ rev_pts = list(reversed(inner))
123
+ if rev_pts:
124
+ rev = f"L {rev_pts[0][0]:.1f} {rev_pts[0][1]:.1f} "
125
+ tension = 0.3
126
+ for i in range(1, len(rev_pts)):
127
+ p0 = rev_pts[max(i - 2, 0)]
128
+ p1 = rev_pts[i - 1]
129
+ p2 = rev_pts[i]
130
+ p3 = rev_pts[min(i + 1, len(rev_pts) - 1)]
131
+ cp1x = p1[0] + (p2[0] - p0[0]) * tension
132
+ cp1y = p1[1] + (p2[1] - p0[1]) * tension
133
+ cp2x = p2[0] - (p3[0] - p1[0]) * tension
134
+ cp2y = p2[1] - (p3[1] - p1[1]) * tension
135
+ rev += f"C {cp1x:.1f} {cp1y:.1f} {cp2x:.1f} {cp2y:.1f} {p2[0]:.1f} {p2[1]:.1f} "
136
+ return fwd + " " + rev + "Z"
137
+ return fwd + " Z"
138
+
139
+
140
+ # ---------------------------------------------------------------------------
141
+ # Main generator
142
+ # ---------------------------------------------------------------------------
143
+
144
+ def generate_shell_svg(features: dict[str, Any], growth: float = 1.0) -> str:
145
+ """Generate a beautiful SVG shell from session features.
146
+
147
+ growth in [0,1] controls how far the shell has formed. At growth<1 the
148
+ spiral is truncated, only knots/jewels up to that fraction appear, and the
149
+ aperture stays closed until growth==1.0. growth>=1.0 is the full shell.
150
+ """
151
+ growth = max(0.0, min(1.0, float(growth)))
152
+
153
+ duration = max(5, min(180, int(features.get("duration_minutes", 30) or 30)))
154
+ approaches = features.get("approaches_tried", []) or []
155
+ dead_ends = features.get("dead_ends", []) or []
156
+ breakthroughs = features.get("breakthroughs", []) or []
157
+ gotchas = features.get("gotchas", []) or []
158
+ themes = features.get("themes", []) or []
159
+ sentiment = features.get("sentiment_arc", {}) or {}
160
+
161
+ start_sent = sentiment.get("start", "confused")
162
+ end_sent = sentiment.get("end", "resolved")
163
+ pal = _palette(start_sent, end_sent)
164
+
165
+ seed = hash(str(features.get("duration_minutes", 0))) % 10000
166
+ rng = random.Random(seed)
167
+
168
+ W = H = 640
169
+ cx, cy = W / 2, H / 2 + 15
170
+
171
+ n_turns = 2.4 + 0.35 * len(approaches) + duration / 150.0
172
+ n_turns = min(n_turns, 4.5)
173
+ r0 = 5
174
+ r_max = min(W * 0.38, 65 + duration * 2.0)
175
+
176
+ # Always compute the FULL spiral (geometry is stable); growth truncates
177
+ # how much of it we draw, so the partial shell is a true prefix of the
178
+ # final one (no reflow, no jitter between stages).
179
+ full_centerline = _log_spiral(cx, cy, n_turns, r0, r_max, n_points=500)
180
+ n_full = len(full_centerline)
181
+ n_grown = max(2, int(round(n_full * growth)))
182
+ centerline = full_centerline[:n_grown]
183
+
184
+ def thickness_at(idx: int, total_full: int) -> float:
185
+ # thickness keyed to position in the FULL spiral, so a point's
186
+ # thickness is identical whether or not later points exist yet
187
+ frac = idx / max(1, total_full - 1)
188
+ return 3 + (r_max * 0.28) * (frac ** 1.35)
189
+
190
+ outer_pts = []
191
+ inner_pts = []
192
+ for i, (x, y, t, r, n) in enumerate(centerline):
193
+ th = thickness_at(i, n_full)
194
+ outer_pts.append((x + math.cos(n) * th, y + math.sin(n) * th, t, r, n))
195
+ inner_pts.append((x - math.cos(n) * th, y - math.sin(n) * th, t, r, n))
196
+
197
+ svg = []
198
+ svg.append(
199
+ f'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 {W} {H}" '
200
+ f'width="{W}" height="{H}">'
201
+ )
202
+
203
+ # ---- DEFS: filters, gradients, patterns ----
204
+ svg.append(f'''<defs>
205
+ <!-- Nacre / mother-of-pearl: barely-there shimmer, not noise -->
206
+ <filter id="nacre" x="-5%" y="-5%" width="110%" height="110%">
207
+ <feTurbulence type="fractalNoise" baseFrequency="0.007 0.015"
208
+ numOctaves="3" seed="{seed}" result="noise"/>
209
+ <feColorMatrix in="noise" type="saturate" values="0.2" result="colored"/>
210
+ <feBlend in="SourceGraphic" in2="colored" mode="soft-light" result="nacred"/>
211
+ <feComposite in="nacred" in2="SourceGraphic" operator="in"/>
212
+ </filter>
213
+
214
+ <!-- Glow for aperture -->
215
+ <filter id="glow" x="-50%" y="-50%" width="200%" height="200%">
216
+ <feGaussianBlur stdDeviation="8" result="blur"/>
217
+ <feComposite in="SourceGraphic" in2="blur" operator="over"/>
218
+ </filter>
219
+
220
+ <!-- Soft glow for jewels -->
221
+ <filter id="jewelGlow" x="-80%" y="-80%" width="260%" height="260%">
222
+ <feGaussianBlur stdDeviation="3" result="blur"/>
223
+ <feComposite in="SourceGraphic" in2="blur" operator="over"/>
224
+ </filter>
225
+
226
+ <!-- Soft shadow for depth -->
227
+ <filter id="depth" x="-5%" y="-5%" width="110%" height="110%">
228
+ <feGaussianBlur stdDeviation="1.5"/>
229
+ </filter>
230
+
231
+ <!-- Shell body gradient (high contrast, 5 stops) -->
232
+ <radialGradient id="bodyGrad" cx="40%" cy="40%" r="70%">
233
+ <stop offset="0%" stop-color="{pal['highlight']}" stop-opacity="0.85"/>
234
+ <stop offset="20%" stop-color="{pal['body_light']}" stop-opacity="0.9"/>
235
+ <stop offset="50%" stop-color="{pal['accent']}" stop-opacity="0.8"/>
236
+ <stop offset="80%" stop-color="{pal['body_dark']}" stop-opacity="0.9"/>
237
+ <stop offset="100%" stop-color="{pal['bg_mid']}" stop-opacity="0.95"/>
238
+ </radialGradient>
239
+
240
+ <!-- Aperture glow gradient -->
241
+ <radialGradient id="apertureGrad" cx="50%" cy="50%" r="60%">
242
+ <stop offset="0%" stop-color="{pal['aperture_core']}" stop-opacity="0.95"/>
243
+ <stop offset="40%" stop-color="{pal['highlight']}" stop-opacity="0.7"/>
244
+ <stop offset="70%" stop-color="{pal['accent']}" stop-opacity="0.4"/>
245
+ <stop offset="100%" stop-color="{pal['body_light']}" stop-opacity="0"/>
246
+ </radialGradient>
247
+
248
+ <!-- Jewel gradient -->
249
+ <radialGradient id="jewelGrad" cx="30%" cy="30%" r="70%">
250
+ <stop offset="0%" stop-color="#fff" stop-opacity="0.95"/>
251
+ <stop offset="30%" stop-color="{pal['jewel']}" stop-opacity="0.85"/>
252
+ <stop offset="70%" stop-color="{pal['accent']}" stop-opacity="0.6"/>
253
+ <stop offset="100%" stop-color="{pal['body_dark']}" stop-opacity="0.3"/>
254
+ </radialGradient>
255
+
256
+ <!-- Background atmosphere gradient -->
257
+ <radialGradient id="atmosphere" cx="45%" cy="45%" r="65%">
258
+ <stop offset="0%" stop-color="{pal['bg_mid']}" stop-opacity="0.8"/>
259
+ <stop offset="50%" stop-color="{pal['bg_deep']}" stop-opacity="0.9"/>
260
+ <stop offset="100%" stop-color="{pal['bg_deep']}" stop-opacity="1"/>
261
+ </radialGradient>
262
+ </defs>''')
263
+
264
+ # ---- BACKGROUND with atmosphere ----
265
+ svg.append(f'<rect width="{W}" height="{H}" fill="{pal["bg_deep"]}"/>')
266
+ svg.append(f'<rect width="{W}" height="{H}" fill="url(#atmosphere)"/>')
267
+
268
+ for _ in range(40):
269
+ px = rng.uniform(20, W - 20)
270
+ py = rng.uniform(20, H - 20)
271
+ pr = rng.uniform(0.3, 1.2)
272
+ po = rng.uniform(0.15, 0.45)
273
+ svg.append(
274
+ f'<circle cx="{px:.1f}" cy="{py:.1f}" r="{pr:.2f}" '
275
+ f'fill="{pal["highlight"]}" opacity="{po:.2f}"/>'
276
+ )
277
+
278
+ svg.append(
279
+ f'<circle cx="{cx}" cy="{cy}" r="{r_max + 90}" '
280
+ f'fill="{pal["body_dark"]}" opacity="0.08"/>'
281
+ )
282
+ svg.append(
283
+ f'<circle cx="{cx}" cy="{cy}" r="{r_max + 50}" '
284
+ f'fill="{pal["body_light"]}" opacity="0.06"/>'
285
+ )
286
+
287
+ # ---- SHELL BODY with nacre texture ----
288
+ body_d = _band_path_smooth(inner_pts, outer_pts)
289
+ svg.append(
290
+ f'<path d="{body_d}" fill="{pal["bg_deep"]}" opacity="0.4" '
291
+ f'transform="translate(3, 4)" filter="url(#depth)"/>'
292
+ )
293
+ svg.append(
294
+ f'<path class="shell-body" d="{body_d}" fill="url(#bodyGrad)" '
295
+ f'stroke="{pal["septa"]}" stroke-width="0.5" filter="url(#nacre)"/>'
296
+ )
297
+
298
+ # ---- SEPTA (chamber ridges) ----
299
+ septa_step = 14 if "debug" in themes else (18 if "build" in themes else 16)
300
+ if len(centerline) > septa_step + 8:
301
+ total_septa = (len(centerline) - 8 - septa_step) // septa_step
302
+ for si, i in enumerate(range(septa_step, len(centerline) - 8, septa_step)):
303
+ ix, iy = inner_pts[i][0], inner_pts[i][1]
304
+ ox, oy = outer_pts[i][0], outer_pts[i][1]
305
+ frac = si / max(1, total_septa)
306
+ opacity = 0.15 + frac * 0.35
307
+ svg.append(
308
+ f'<line x1="{ix:.1f}" y1="{iy:.1f}" '
309
+ f'x2="{ox:.1f}" y2="{oy:.1f}" '
310
+ f'stroke="{pal["septa"]}" stroke-width="0.7" '
311
+ f'opacity="{opacity:.2f}" stroke-linecap="round"/>'
312
+ )
313
+
314
+ # ---- LONGITUDINAL BANDS ----
315
+ for frac, opacity in [(0.2, 0.2), (0.4, 0.3), (0.6, 0.25), (0.8, 0.2)]:
316
+ band = []
317
+ for (ox, oy, *_), (ix, iy, *_) in zip(outer_pts, inner_pts):
318
+ bx = ix + (ox - ix) * frac
319
+ by = iy + (oy - iy) * frac
320
+ band.append((bx, by))
321
+ path_d = _smooth_path(band)
322
+ color = pal["accent"] if frac > 0.5 else pal["highlight"]
323
+ svg.append(
324
+ f'<path d="{path_d}" fill="none" stroke="{color}" '
325
+ f'stroke-width="0.6" opacity="{opacity}"/>'
326
+ )
327
+
328
+ # ---- CENTERLINE shimmer ----
329
+ cl_path = _smooth_path(centerline)
330
+ svg.append(
331
+ f'<path class="shell-centerline" d="{cl_path}" fill="none" '
332
+ f'stroke="{pal["highlight"]}" stroke-width="0.5" opacity="0.4"/>'
333
+ )
334
+
335
+ # ---- OUTER RIM GLOW ----
336
+ rim_path = _smooth_path(outer_pts)
337
+ svg.append(
338
+ f'<path class="shell-rim" d="{rim_path}" fill="none" '
339
+ f'stroke="{pal["accent"]}" stroke-width="2.5" opacity="0.15" '
340
+ f'filter="url(#jewelGlow)"/>'
341
+ )
342
+ svg.append(
343
+ f'<path d="{rim_path}" fill="none" '
344
+ f'stroke="{pal["highlight"]}" stroke-width="0.8" opacity="0.5"/>'
345
+ )
346
+
347
+ # ---- CENTRAL EYE ----
348
+ ex, ey = centerline[0][0], centerline[0][1]
349
+ svg.append(
350
+ f'<circle cx="{ex:.1f}" cy="{ey:.1f}" r="4" '
351
+ f'fill="{pal["accent"]}" opacity="0.3" filter="url(#jewelGlow)"/>'
352
+ )
353
+ svg.append(
354
+ f'<circle cx="{ex:.1f}" cy="{ey:.1f}" r="1.8" '
355
+ f'fill="{pal["highlight"]}" opacity="0.85"/>'
356
+ )
357
+ svg.append(
358
+ f'<circle cx="{ex - 0.4:.1f}" cy="{ey - 0.5:.1f}" r="0.6" '
359
+ f'fill="#fff" opacity="0.9"/>'
360
+ )
361
+
362
+ # ---- BYOBU BATTLE LAYER (figures along the spiral) ----
363
+ # Always on: the nautilus is the battlefield. Dead ends are fallen
364
+ # warriors, gotchas are archers, the breakthrough is the dragon at the
365
+ # aperture. Drawn under the canonical knots/jewels so the markers still
366
+ # read. Only on the fully grown shell (growth>=1.0); during birth the
367
+ # shell forms first, then the battle populates it.
368
+ if growth >= 1.0:
369
+ try:
370
+ svg.append(build_battle_layer(
371
+ features, centerline, outer_pts, thickness_at, n_full, pal, seed
372
+ ))
373
+ except Exception as _battle_err:
374
+ # The battle layer is decorative; never let it break the shell.
375
+ pass
376
+
377
+ # ---- DEAD-END KNOTS (only those that have formed by this growth) ----
378
+ for de in dead_ends:
379
+ pos = max(0.04, min(0.96, float(de.get("position", 0.5))))
380
+ if pos > growth:
381
+ continue # not yet formed at this stage
382
+ idx = int(pos * (n_full - 1))
383
+ if idx >= len(centerline):
384
+ continue
385
+ x, y, _, r, _ = centerline[idx]
386
+ th = thickness_at(idx, n_full)
387
+ kr = max(3.5, th * 0.32)
388
+ svg.append(
389
+ f'<circle cx="{x + 1:.1f}" cy="{y + 1:.1f}" r="{kr + 1:.1f}" '
390
+ f'fill="{pal["bg_deep"]}" opacity="0.5" filter="url(#depth)"/>'
391
+ )
392
+ svg.append(
393
+ f'<circle class="shell-knot" cx="{x:.1f}" cy="{y:.1f}" r="{kr:.1f}" '
394
+ f'fill="{pal["knot"]}" stroke="{pal["body_dark"]}" '
395
+ f'stroke-width="1" opacity="0.9"/>'
396
+ )
397
+ svg.append(
398
+ f'<circle cx="{x - 1:.1f}" cy="{y - 1.2:.1f}" r="{kr * 0.3:.1f}" '
399
+ f'fill="{pal["body_dark"]}" opacity="0.6"/>'
400
+ )
401
+
402
+ # ---- GOTCHA JEWELS (those whose rim position has formed) ----
403
+ n_jewels = min(len(gotchas), 12)
404
+ for i in range(n_jewels):
405
+ frac = 0.28 + (i / max(1, n_jewels)) * 0.65
406
+ if frac > growth:
407
+ continue # rim hasn't reached this jewel yet
408
+ idx = int(frac * (len(outer_pts) - 1)) if outer_pts else 0
409
+ if idx >= len(outer_pts):
410
+ continue
411
+ ox, oy, _, _, n = outer_pts[idx]
412
+ jx = ox + math.cos(n) * 4
413
+ jy = oy + math.sin(n) * 4
414
+ svg.append(
415
+ f'<circle cx="{jx:.1f}" cy="{jy:.1f}" r="9" '
416
+ f'fill="{pal["jewel"]}" opacity="0.3" filter="url(#jewelGlow)"/>'
417
+ )
418
+ svg.append(
419
+ f'<circle class="shell-jewel" cx="{jx:.1f}" cy="{jy:.1f}" r="4.5" '
420
+ f'fill="url(#jewelGrad)" opacity="0.95"/>'
421
+ )
422
+ svg.append(
423
+ f'<circle cx="{jx - 1:.1f}" cy="{jy - 1.2:.1f}" r="1.2" '
424
+ f'fill="#fff" opacity="0.9"/>'
425
+ )
426
+
427
+ # ---- APERTURE (breakthrough mouth) - opens only when fully grown ----
428
+ # The breakthrough is the LAST thing to form. It opens as growth nears 1.0,
429
+ # fading in over the final 15% so the reveal lands on the breakthrough.
430
+ if growth > 0.85:
431
+ aperture_op = min(1.0, (growth - 0.85) / 0.15)
432
+ bidx = len(centerline) - 1
433
+ bx, by, _, br, bn = centerline[bidx]
434
+ bth = thickness_at(bidx, n_full)
435
+ svg.append(
436
+ f'<ellipse cx="{bx:.1f}" cy="{by:.1f}" '
437
+ f'rx="{bth * 2.0:.1f}" ry="{bth * 1.5:.1f}" '
438
+ f'transform="rotate({math.degrees(bn):.0f} {bx:.1f} {by:.1f})" '
439
+ f'fill="{pal["accent"]}" opacity="{0.15 * aperture_op:.2f}" filter="url(#glow)"/>'
440
+ )
441
+ svg.append(
442
+ f'<ellipse class="shell-aperture" cx="{bx:.1f}" cy="{by:.1f}" '
443
+ f'rx="{bth * 1.2:.1f}" ry="{bth * 0.9:.1f}" '
444
+ f'transform="rotate({math.degrees(bn):.0f} {bx:.1f} {by:.1f})" '
445
+ f'fill="url(#apertureGrad)" opacity="{0.95 * aperture_op:.2f}"/>'
446
+ )
447
+ svg.append(
448
+ f'<ellipse cx="{bx:.1f}" cy="{by:.1f}" '
449
+ f'rx="{bth * 0.4:.1f}" ry="{bth * 0.28:.1f}" '
450
+ f'transform="rotate({math.degrees(bn):.0f} {bx:.1f} {by:.1f})" '
451
+ f'fill="{pal["aperture_core"]}" opacity="{0.9 * aperture_op:.2f}"/>'
452
+ )
453
+
454
+ # ---- SIGNATURE (full shell only) ----
455
+ if growth >= 1.0:
456
+ stats = (
457
+ f"turboskillslug · {duration}m · {len(approaches)} tried · "
458
+ f"{len(dead_ends)} stumbles · {len(gotchas)} jewels"
459
+ )
460
+ svg.append(
461
+ f'<text x="{W - 14}" y="{H - 14}" text-anchor="end" '
462
+ f'fill="{pal["body_light"]}" font-size="9" opacity="0.5" '
463
+ f'font-family="Georgia, serif" font-style="italic">{stats}</text>'
464
+ )
465
+
466
+ svg.append("</svg>")
467
+ return "\n".join(svg)
468
+
469
+
470
+ # ---------------------------------------------------------------------------
471
+ # Smoke test
472
+ # ---------------------------------------------------------------------------
473
+
474
+ if __name__ == "__main__":
475
+ from pathlib import Path
476
+
477
+ sample = {
478
+ "duration_minutes": 47,
479
+ "themes": ["debug", "build"],
480
+ "approaches_tried": [
481
+ {"approach": "naive regex", "why_it_failed": "too brittle"},
482
+ {"approach": "ast parser", "why_it_failed": "complex setup"},
483
+ ],
484
+ "dead_ends": [
485
+ {"position": 0.15, "what_happened": "regex backtracked"},
486
+ {"position": 0.42, "what_happened": "missing edge case"},
487
+ {"position": 0.61, "what_happened": "wrong import"},
488
+ ],
489
+ "breakthroughs": [{"position": 0.88, "what_worked": "ast walk"}],
490
+ "gotchas": ["null nodes", "tabs vs spaces", "comments",
491
+ "windows line endings", "unicode"],
492
+ "sentiment_arc": {"start": "frustrated", "end": "resolved"},
493
+ }
494
+ for g in [0.2, 0.4, 0.6, 0.8, 1.0]:
495
+ Path(f"growth_{int(g*100)}.svg").write_text(generate_shell_svg(sample, growth=g))
496
+ print(f"wrote growth_{int(g*100)}.svg")
shell3d_lens.py ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Integration shim: exposes the 3D shell lens to the flat-imported app.
3
+
4
+ Robust to layout: the scenegraph modules (scene_graph / scene_builder / renderer /
5
+ shell3d) may live in a scenegraph/ subfolder OR be flattened to the repo root by the
6
+ deploy. We add BOTH this file's directory and a scenegraph/ subdir (if present) to
7
+ sys.path, then import by module name. Exposes one entry point:
8
+ render_shell_3d(extraction) -> iframe HTML, or None (graceful degradation).
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import os
14
+ import sys
15
+
16
+ _HERE = os.path.dirname(os.path.abspath(__file__))
17
+ # add the repo root (files may be flattened here) ...
18
+ if _HERE not in sys.path:
19
+ sys.path.insert(0, _HERE)
20
+ # ... and a scenegraph/ subfolder if it exists (files may be nested there)
21
+ _SUB = os.path.join(_HERE, "scenegraph")
22
+ if os.path.isdir(_SUB) and _SUB not in sys.path:
23
+ sys.path.insert(0, _SUB)
24
+ _SUBREND = os.path.join(_SUB, "renderers")
25
+ if os.path.isdir(_SUBREND) and _SUBREND not in sys.path:
26
+ sys.path.insert(0, _SUBREND)
27
+
28
+
29
+ def render_shell_3d(extraction: dict) -> str | None:
30
+ """Build the SceneGraph and render the 3D iridescent shell. Returns iframe HTML,
31
+ or None if anything is unavailable (the app degrades gracefully)."""
32
+ try:
33
+ from scene_builder import build_scene_graph
34
+ from shell3d import Shell3DRenderer
35
+ except Exception:
36
+ return None
37
+ try:
38
+ scene = build_scene_graph(extraction)
39
+ renderer = Shell3DRenderer()
40
+ ok, _reason = renderer.available()
41
+ if not ok:
42
+ return None
43
+ out = renderer.render(scene)
44
+ return out.get("html")
45
+ except Exception:
46
+ return None
shell_animate.py ADDED
@@ -0,0 +1,190 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Scroll-unroll birth animation for the shell.
3
+
4
+ The shell is born like an ancient scroll unfurling. A radial wipe mask reveals
5
+ the colored spiral from the eye outward — the scroll unrolling — led by a glowing
6
+ gold parchment edge. As the unrolling front passes each battle figure, that
7
+ figure inks in. The breakthrough dragon and aperture land last, at the tip.
8
+
9
+ Why a wipe MASK and not stroke-drawing: the shell body is a single filled spiral
10
+ path, so it cannot "draw" like a stroke. A growing radial mask genuinely reveals
11
+ the filled body progressively, which is what reads as a scroll unrolling.
12
+
13
+ SMIL (native SVG animation) so it survives Gradio's sanitizer; wrapped in an
14
+ <iframe srcdoc>. fill="freeze" holds the final static shell. Figures default
15
+ visible (animation drives them from opacity 0), so a no-SMIL browser still shows
16
+ the complete shell.
17
+ """
18
+
19
+ from __future__ import annotations
20
+
21
+ import re
22
+
23
+ from anim_diagnostic import inject_probe
24
+
25
+ DIAGNOSTIC = False
26
+
27
+ SCROLL_DUR = 3.6 # seconds for the scroll to fully unroll
28
+ FIG_FADE = 0.7 # seconds each figure takes to ink in
29
+ EYE_X, EYE_Y = 320.0, 335.0 # shell eye (spiral origin); body starts here
30
+ MAX_R = 330.0 # radius that covers the whole 640 canvas from the eye
31
+ KEYSPLINE = "0.3 0 0.4 1" # ease for the unroll
32
+
33
+
34
+ def _scroll_mask_and_edge(animated_only: bool = True) -> tuple[str, str]:
35
+ """Return (mask_def, glowing_edge) for the live animated scroll-unroll."""
36
+ mask_circle = (
37
+ f'<circle cx="{EYE_X}" cy="{EYE_Y}" r="0" fill="white">'
38
+ f'<animate attributeName="r" from="0" to="{MAX_R}" dur="{SCROLL_DUR}s" '
39
+ f'begin="0s" fill="freeze" calcMode="spline" keySplines="{KEYSPLINE}" '
40
+ f'keyTimes="0;1"/></circle>'
41
+ )
42
+ mask_def = f'<mask id="scrollWipe">{mask_circle}</mask>'
43
+ edge = (
44
+ f'<g class="scroll-edge">'
45
+ f'<circle cx="{EYE_X}" cy="{EYE_Y}" r="0" fill="none" stroke="#e6c870" '
46
+ f'stroke-width="6" opacity="0.55">'
47
+ f'<animate attributeName="r" from="0" to="{MAX_R}" dur="{SCROLL_DUR}s" '
48
+ f'begin="0s" fill="freeze" calcMode="spline" keySplines="{KEYSPLINE}" keyTimes="0;1"/>'
49
+ f'<animate attributeName="opacity" values="0.55;0.55;0" keyTimes="0;0.82;1" '
50
+ f'dur="{SCROLL_DUR}s" begin="0s" fill="freeze"/></circle>'
51
+ f'<circle cx="{EYE_X}" cy="{EYE_Y}" r="0" fill="none" stroke="#efe0b0" '
52
+ f'stroke-width="2" opacity="0.8">'
53
+ f'<animate attributeName="r" from="0" to="{MAX_R}" dur="{SCROLL_DUR}s" '
54
+ f'begin="0s" fill="freeze" calcMode="spline" keySplines="{KEYSPLINE}" keyTimes="0;1"/>'
55
+ f'<animate attributeName="opacity" values="0.8;0.8;0" keyTimes="0;0.82;1" '
56
+ f'dur="{SCROLL_DUR}s" begin="0s" fill="freeze"/></circle>'
57
+ f'</g>'
58
+ )
59
+ return mask_def, edge
60
+
61
+
62
+ def _ink_figures(svg: str) -> str:
63
+ """Each battle-fig group inks in (fade + slight rise) when the unrolling front
64
+ reaches its data-pos radius. begin = pos * SCROLL_DUR (the dragon, pos==1.0,
65
+ lands at the very end)."""
66
+ def repl(m):
67
+ tag = m.group(0)
68
+ pos_match = re.search(r'data-pos="([\d.]+)"', tag)
69
+ pos = float(pos_match.group(1)) if pos_match else 0.5
70
+ begin = round(pos * SCROLL_DUR, 2)
71
+ anim = (
72
+ f'<animate attributeName="opacity" from="0" to="1" '
73
+ f'dur="{FIG_FADE}s" begin="{begin}s" fill="freeze"/>'
74
+ f'<animateTransform attributeName="transform" type="translate" '
75
+ f'from="0 8" to="0 0" dur="{FIG_FADE}s" begin="{begin}s" '
76
+ f'fill="freeze" calcMode="spline" keySplines="0.2 0.8 0.2 1" keyTimes="0;1"/>'
77
+ )
78
+ # do NOT hard-set opacity 0 (keeps figures visible if SMIL fails)
79
+ return tag + anim
80
+ return re.sub(r'<g class="battle-fig" data-pos="[\d.]+">', repl, svg)
81
+
82
+
83
+ def animate_shell_svg(svg: str, seed=0, style: str | None = None) -> str:
84
+ """Inject the scroll-unroll birth: a radial wipe mask reveals the shell from
85
+ the eye outward, a gold parchment edge rides the unrolling front, and the
86
+ battle figures ink in as the front passes them.
87
+ """
88
+ # 1. inject the mask def + edge after </defs>
89
+ defs_end = svg.find("</defs>")
90
+ if defs_end == -1:
91
+ open_end = svg.find(">", svg.find("<svg")) + 1
92
+ svg = svg[:open_end] + "<defs></defs>" + svg[open_end:]
93
+ defs_end = svg.find("</defs>")
94
+ insert_at = defs_end + len("</defs>")
95
+ mask_def, edge = _scroll_mask_and_edge()
96
+
97
+ # 2. wrap the shell (from the body onward) in the masked group. Background
98
+ # (rect, atmosphere, particles, halos) stays unmasked so only the SHELL
99
+ # unrolls, against a steady night sky.
100
+ body_idx = svg.find('<path class="shell-body"')
101
+ if body_idx == -1:
102
+ body_idx = insert_at # fallback: mask everything after defs
103
+ svg_close = svg.rfind("</svg>")
104
+
105
+ head = svg[:insert_at] + mask_def
106
+ pre_shell = svg[insert_at:body_idx]
107
+ shell_region = svg[body_idx:svg_close]
108
+ tail = svg[svg_close:]
109
+
110
+ shell_region = _ink_figures(shell_region)
111
+
112
+ return (
113
+ head
114
+ + pre_shell
115
+ + '<g mask="url(#scrollWipe)">'
116
+ + shell_region
117
+ + '</g>'
118
+ + edge
119
+ + tail
120
+ )
121
+
122
+
123
+ # ---- iframe wrapping (survives Gradio sanitizer) + replay button -------------
124
+
125
+ REPLAY_HTML = """
126
+ <button id="replay-birth" title="watch the scroll unroll again">\u21bb watch it unroll again</button>
127
+ <style>
128
+ #replay-birth{
129
+ position:fixed;left:50%;bottom:10px;transform:translateX(-50%);
130
+ z-index:9999;font:12px/1.2 Georgia,serif;font-style:italic;
131
+ padding:6px 14px;border-radius:14px;cursor:pointer;
132
+ background:rgba(20,16,12,0.55);color:#efe0b0;border:1px solid rgba(200,162,76,0.5);
133
+ backdrop-filter:blur(2px);transition:opacity .3s;opacity:0.78;
134
+ }
135
+ #replay-birth:hover{opacity:1;background:rgba(20,16,12,0.75);}
136
+ </style>
137
+ <script>
138
+ (function(){
139
+ function restart(){
140
+ var svg = document.querySelector('svg');
141
+ if(!svg) return;
142
+ // Rewind the whole SVG timeline. For begin="0s" animations this re-runs
143
+ // them from the top. We pause, seek to 0, then unpause so the flipbook
144
+ // (stage opacity keyframes) and the curl motion both restart cleanly.
145
+ try {
146
+ svg.pauseAnimations();
147
+ svg.setCurrentTime(0);
148
+ svg.unpauseAnimations();
149
+ } catch(e){
150
+ // fallback: plain rewind
151
+ try { svg.setCurrentTime(0); } catch(e2){}
152
+ }
153
+ }
154
+ var btn = document.getElementById('replay-birth');
155
+ if(btn){ btn.addEventListener('click', restart); }
156
+ })();
157
+ </script>
158
+ """
159
+
160
+
161
+ def inject_replay(iframe_inner_html: str) -> str:
162
+ if "</body>" in iframe_inner_html:
163
+ return iframe_inner_html.replace("</body>", REPLAY_HTML + "</body>", 1)
164
+ return iframe_inner_html + REPLAY_HTML
165
+
166
+
167
+ def wrap_in_iframe(animated_svg: str, height: int = 660, replay: bool = True) -> str:
168
+ """Wrap the animated SVG in an <iframe srcdoc> (survives Gradio sanitizer),
169
+ rendered as a centered square so the whole shell shows."""
170
+ import html as _html
171
+ doc = (
172
+ "<!DOCTYPE html><html><head><style>"
173
+ "html,body{margin:0;padding:0;background:transparent;overflow:hidden;"
174
+ "height:100%;display:flex;align-items:center;justify-content:center}"
175
+ "svg{max-width:100%;max-height:100%;width:auto;height:auto;display:block}"
176
+ "</style></head><body>" + animated_svg + "</body></html>"
177
+ )
178
+ if DIAGNOSTIC:
179
+ doc = inject_probe(doc)
180
+ if replay:
181
+ doc = inject_replay(doc)
182
+ escaped = _html.escape(doc, quote=True)
183
+ return (
184
+ f'<div style="display:flex;justify-content:center;width:100%;">'
185
+ f'<iframe srcdoc="{escaped}" '
186
+ f'style="width:{height}px;max-width:100%;height:{height}px;'
187
+ f'border:none;background:transparent;" '
188
+ f'sandbox="allow-scripts"></iframe>'
189
+ f'</div>'
190
+ )
shell_battle.py ADDED
@@ -0,0 +1,379 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Byobu battle layer for the shell.
3
+
4
+ The nautilus IS the battlefield. The spiral is the campaign path: the eye is
5
+ where the session began, the outward sweep is time. Session events become
6
+ stylized ink-on-gold figures, in the flat perspective of Japanese folding
7
+ screens (byobu):
8
+
9
+ developer -> the lone general near the eye, a banner at their side
10
+ dead ends -> fallen warriors + broken banners at their spiral positions
11
+ gotchas -> archers along the outer rim (the ambushes)
12
+ breakthrough -> the dragon coiled at the aperture, a victory banner upright
13
+ sentiment -> gold cloud-bands and a few hill/pine terrain strokes
14
+
15
+ Figures are dark ink silhouettes riding on gold-leaf cloud bands. They are
16
+ drawn small so they sit IN the landscape, not over it.
17
+
18
+ The layer is produced from the SAME spiral geometry the shell uses, so a
19
+ fallen warrior sits exactly where its dead-end knot is.
20
+ """
21
+
22
+ from __future__ import annotations
23
+
24
+ import math
25
+ import random
26
+
27
+
28
+ # Ink + gold palette (byobu). The figures are near-black sumi ink; the clouds
29
+ # and banners are gold leaf. These are fixed, not sentiment-driven, because the
30
+ # byobu convention is ink-on-gold regardless of the campaign's mood (the mood
31
+ # lives in the shell's own colors underneath).
32
+ INK = "#1a1410"
33
+ INK_SOFT = "#2c2218"
34
+ GOLD = "#c8a24c"
35
+ GOLD_BRIGHT = "#e6c870"
36
+ GOLD_PALE = "#efe0b0"
37
+
38
+
39
+
40
+ def _gold_backing(x: float, y: float, r: float) -> str:
41
+ """A gold-leaf disc behind a figure so dark ink reads against gold. A faint
42
+ dark ring keeps the figure legible even on gold-heavy (warm) shells where
43
+ the gold backing would otherwise blend into the body."""
44
+ return (
45
+ f'<circle cx="{x:.1f}" cy="{y:.1f}" r="{r + 1.5:.1f}" fill="{INK}" opacity="0.30"/>'
46
+ f'<circle cx="{x:.1f}" cy="{y:.1f}" r="{r:.1f}" fill="{GOLD}" opacity="0.62"/>'
47
+ f'<circle cx="{x:.1f}" cy="{y:.1f}" r="{r:.1f}" fill="none" '
48
+ f'stroke="{INK}" stroke-width="0.8" opacity="0.45"/>'
49
+ )
50
+
51
+ def _cloud_band(cx: float, cy: float, w: float, h: float, seed: int) -> str:
52
+ """A gold-leaf byobu cloud band: a soft lobed cloud (smooth quad curves).
53
+
54
+ Real byobu clouds are scalloped, billowing shapes, not rectangles. We build
55
+ a closed path of rounded lobes along the top and bottom edges.
56
+ """
57
+ rng = random.Random(seed)
58
+ n = rng.randint(4, 6)
59
+ left = cx - w / 2
60
+ step = w / n
61
+ # top edge: a series of upward lobes; bottom edge: downward lobes
62
+ d = f"M {left:.1f} {cy:.1f} "
63
+ x = left
64
+ for i in range(n):
65
+ nx = x + step
66
+ peak = cy - h / 2 - rng.uniform(0, h * 0.3)
67
+ d += f"Q {x + step/2:.1f} {peak:.1f} {nx:.1f} {cy:.1f} "
68
+ x = nx
69
+ for i in range(n):
70
+ nx = x - step
71
+ trough = cy + h / 2 + rng.uniform(0, h * 0.3)
72
+ d += f"Q {x - step/2:.1f} {trough:.1f} {nx:.1f} {cy:.1f} "
73
+ x = nx
74
+ d += "Z"
75
+ return (
76
+ f'<path d="{d}" fill="{GOLD}" opacity="0.20"/>'
77
+ f'<path d="{d}" fill="none" stroke="{GOLD_BRIGHT}" '
78
+ f'stroke-width="0.7" opacity="0.4"/>'
79
+ )
80
+
81
+
82
+ def _banner(x: float, y: float, h: float, color: str, broken: bool = False,
83
+ angle: float = 0.0) -> str:
84
+ """A small byobu banner on a pole. Broken = leaning, torn."""
85
+ pole_top = (x, y - h)
86
+ lean = 28 if broken else 0
87
+ tx = x + math.sin(math.radians(angle + lean)) * h
88
+ ty = y - math.cos(math.radians(angle + lean)) * h
89
+ pole = (
90
+ f'<line x1="{x:.1f}" y1="{y:.1f}" x2="{tx:.1f}" y2="{ty:.1f}" '
91
+ f'stroke="{INK}" stroke-width="1.1" stroke-linecap="round"/>'
92
+ )
93
+ # flag near the top of the pole
94
+ fx, fy = tx, ty
95
+ fw = h * 0.5
96
+ fh = h * 0.42
97
+ if broken:
98
+ # torn flag: a notched quad
99
+ flag = (
100
+ f'<path d="M {fx:.1f} {fy:.1f} L {fx+fw:.1f} {fy+fh*0.2:.1f} '
101
+ f'L {fx+fw*0.7:.1f} {fy+fh*0.5:.1f} L {fx+fw:.1f} {fy+fh*0.8:.1f} '
102
+ f'L {fx:.1f} {fy+fh:.1f} Z" fill="{color}" opacity="0.85"/>'
103
+ )
104
+ else:
105
+ flag = (
106
+ f'<path d="M {fx:.1f} {fy:.1f} L {fx+fw:.1f} {fy+fh*0.25:.1f} '
107
+ f'L {fx+fw:.1f} {fy+fh*0.75:.1f} L {fx:.1f} {fy+fh:.1f} Z" '
108
+ f'fill="{color}" opacity="0.9"/>'
109
+ )
110
+ return pole + flag
111
+
112
+
113
+ def _kabuto(x: float, y: float, s: float, ink: str) -> str:
114
+ """A kabuto helmet silhouette with the crescent (maedate) crest."""
115
+ # dome
116
+ dome = (
117
+ f'<path d="M {x - s*0.26:.1f} {y:.1f} '
118
+ f'Q {x:.1f} {y - s*0.42:.1f} {x + s*0.26:.1f} {y:.1f} Z" fill="{ink}"/>'
119
+ )
120
+ # neck guard (shikoro) flare
121
+ flare = (
122
+ f'<path d="M {x - s*0.30:.1f} {y:.1f} L {x - s*0.34:.1f} {y + s*0.12:.1f} '
123
+ f'L {x + s*0.34:.1f} {y + s*0.12:.1f} L {x + s*0.30:.1f} {y:.1f} Z" '
124
+ f'fill="{ink}"/>'
125
+ )
126
+ # crescent crest (maedate) - two horns
127
+ crest = (
128
+ f'<path d="M {x - s*0.22:.1f} {y - s*0.12:.1f} '
129
+ f'Q {x:.1f} {y - s*0.55:.1f} {x + s*0.22:.1f} {y - s*0.12:.1f} '
130
+ f'Q {x:.1f} {y - s*0.34:.1f} {x - s*0.22:.1f} {y - s*0.12:.1f} Z" '
131
+ f'fill="{ink}"/>'
132
+ )
133
+ return dome + flare + crest
134
+
135
+
136
+ def _katana(x1, y1, x2, y2, ink) -> str:
137
+ """A katana: a slightly curved blade stroke with a guard."""
138
+ mx = (x1 + x2) / 2 + (y2 - y1) * 0.12
139
+ my = (y1 + y2) / 2 - (x2 - x1) * 0.12
140
+ blade = (
141
+ f'<path d="M {x1:.1f} {y1:.1f} Q {mx:.1f} {my:.1f} {x2:.1f} {y2:.1f}" '
142
+ f'fill="none" stroke="{ink}" stroke-width="1.4" stroke-linecap="round"/>'
143
+ )
144
+ guard = f'<circle cx="{x1:.1f}" cy="{y1:.1f}" r="1.6" fill="{ink}"/>'
145
+ return blade + guard
146
+
147
+
148
+ def _general(x: float, y: float, s: float) -> str:
149
+ """The developer: a standing samurai general, kabuto + katana raised."""
150
+ # torso (armored, slightly trapezoidal do)
151
+ torso = (
152
+ f'<path d="M {x - s*0.20:.1f} {y - s*0.52:.1f} '
153
+ f'L {x - s*0.30:.1f} {y:.1f} L {x + s*0.30:.1f} {y:.1f} '
154
+ f'L {x + s*0.20:.1f} {y - s*0.52:.1f} Z" fill="{INK}"/>'
155
+ )
156
+ # armor lames (two horizontal segment lines)
157
+ lames = (
158
+ f'<line x1="{x - s*0.26:.1f}" y1="{y - s*0.16:.1f}" '
159
+ f'x2="{x + s*0.26:.1f}" y2="{y - s*0.16:.1f}" stroke="{GOLD_PALE}" '
160
+ f'stroke-width="0.6" opacity="0.5"/>'
161
+ f'<line x1="{x - s*0.22:.1f}" y1="{y - s*0.04:.1f}" '
162
+ f'x2="{x + s*0.22:.1f}" y2="{y - s*0.04:.1f}" stroke="{GOLD_PALE}" '
163
+ f'stroke-width="0.6" opacity="0.5"/>'
164
+ )
165
+ # head + kabuto
166
+ head = f'<circle cx="{x:.1f}" cy="{y - s*0.60:.1f}" r="{s*0.12:.1f}" fill="{INK}"/>'
167
+ helm = _kabuto(x, y - s*0.64, s*0.9, INK)
168
+ # katana raised diagonally
169
+ sword = _katana(x + s*0.24, y - s*0.30, x + s*0.62, y - s*0.78, INK)
170
+ return torso + lames + head + helm + sword
171
+
172
+
173
+ def _fallen(x: float, y: float, s: float, ink: str) -> str:
174
+ """A fallen warrior: toppled samurai, kabuto askew, katana dropped."""
175
+ # body lying down (rotated trapezoid torso)
176
+ g = f'<g transform="rotate(82 {x:.1f} {y:.1f})" opacity="0.9">'
177
+ torso = (
178
+ f'<path d="M {x - s*0.18:.1f} {y - s*0.48:.1f} '
179
+ f'L {x - s*0.26:.1f} {y:.1f} L {x + s*0.26:.1f} {y:.1f} '
180
+ f'L {x + s*0.18:.1f} {y - s*0.48:.1f} Z" fill="{ink}"/>'
181
+ )
182
+ head = f'<circle cx="{x:.1f}" cy="{y - s*0.56:.1f}" r="{s*0.11:.1f}" fill="{ink}"/>'
183
+ helm = _kabuto(x, y - s*0.60, s*0.8, ink)
184
+ g_end = '</g>'
185
+ # dropped katana lying separately, near the body
186
+ sword = _katana(x - s*0.5, y + s*0.15, x - s*0.05, y + s*0.30, ink)
187
+ return g + torso + head + helm + g_end + sword
188
+
189
+
190
+ def _archer(x: float, y: float, s: float, ink: str) -> str:
191
+ """A samurai archer: kneeling, drawing a tall asymmetric yumi bow."""
192
+ # kneeling body
193
+ body = (
194
+ f'<path d="M {x - s*0.05:.1f} {y - s*0.40:.1f} '
195
+ f'L {x - s*0.28:.1f} {y:.1f} L {x + s*0.10:.1f} {y:.1f} Z" fill="{ink}"/>'
196
+ )
197
+ head = f'<circle cx="{x - s*0.02:.1f}" cy="{y - s*0.48:.1f}" r="{s*0.10:.1f}" fill="{ink}"/>'
198
+ helm = _kabuto(x - s*0.02, y - s*0.52, s*0.7, ink)
199
+ # tall yumi bow (asymmetric: grip below center, long upper limb)
200
+ bx = x + s * 0.28
201
+ bow = (
202
+ f'<path d="M {bx:.1f} {y - s*0.75:.1f} '
203
+ f'Q {bx + s*0.34:.1f} {y - s*0.15:.1f} {bx:.1f} {y + s*0.32:.1f}" '
204
+ f'fill="none" stroke="{ink}" stroke-width="1.1"/>'
205
+ )
206
+ # bowstring + nocked arrow drawn back
207
+ string = (
208
+ f'<line x1="{bx:.1f}" y1="{y - s*0.75:.1f}" x2="{bx:.1f}" y2="{y + s*0.32:.1f}" '
209
+ f'stroke="{ink}" stroke-width="0.5"/>'
210
+ )
211
+ arrow = (
212
+ f'<line x1="{x + s*0.02:.1f}" y1="{y - s*0.30:.1f}" '
213
+ f'x2="{bx + s*0.10:.1f}" y2="{y - s*0.30:.1f}" stroke="{ink}" stroke-width="0.7"/>'
214
+ )
215
+ return body + head + helm + bow + string + arrow
216
+
217
+
218
+ def _dragon(x: float, y: float, s: float, angle_deg: float) -> str:
219
+ """The breakthrough: an Eastern dragon, maned head + horns + sinuous segmented
220
+ body + clawed legs, coiling toward the aperture."""
221
+ seg = s
222
+ a = math.radians(angle_deg)
223
+ # build the spine path
224
+ d = f"M {x:.1f} {y:.1f} "
225
+ px, py = x, y
226
+ spine = [(px, py)]
227
+ for i in range(1, 8):
228
+ swing = (1 if i % 2 else -1) * seg * 0.55
229
+ nx = px + math.cos(a) * seg * 0.55 + math.cos(a + math.pi/2) * swing
230
+ ny = py + math.sin(a) * seg * 0.55 + math.sin(a + math.pi/2) * swing
231
+ d += f"Q {px + math.cos(a+math.pi/2)*swing:.1f} {py + math.sin(a+math.pi/2)*swing:.1f} {nx:.1f} {ny:.1f} "
232
+ spine.append((nx, ny))
233
+ px, py = nx, ny
234
+ body = (
235
+ f'<path d="{d}" fill="none" stroke="{INK}" stroke-width="{s*0.30:.1f}" '
236
+ f'stroke-linecap="round" opacity="0.92"/>'
237
+ )
238
+ # dorsal ridge: small spikes along the spine
239
+ spikes = ""
240
+ for i in range(1, len(spine) - 1, 2):
241
+ sx, sy = spine[i]
242
+ spikes += (
243
+ f'<circle cx="{sx:.1f}" cy="{sy:.1f}" r="{s*0.07:.1f}" fill="{INK}"/>'
244
+ )
245
+ # clawed legs at two points
246
+ legs = ""
247
+ for i in (2, 5):
248
+ if i < len(spine):
249
+ lx, ly = spine[i]
250
+ legs += (
251
+ f'<line x1="{lx:.1f}" y1="{ly:.1f}" x2="{lx - s*0.3:.1f}" '
252
+ f'y2="{ly + s*0.3:.1f}" stroke="{INK}" stroke-width="{s*0.10:.1f}" '
253
+ f'stroke-linecap="round"/>'
254
+ )
255
+ # head at the start (where the dragon faces): maned, horned, gold eye
256
+ hx, hy = x, y
257
+ head = (
258
+ f'<circle cx="{hx:.1f}" cy="{hy:.1f}" r="{s*0.26:.1f}" fill="{INK}"/>'
259
+ # two horns
260
+ f'<line x1="{hx - s*0.1:.1f}" y1="{hy - s*0.18:.1f}" '
261
+ f'x2="{hx - s*0.28:.1f}" y2="{hy - s*0.42:.1f}" stroke="{INK}" '
262
+ f'stroke-width="{s*0.06:.1f}" stroke-linecap="round"/>'
263
+ f'<line x1="{hx + s*0.1:.1f}" y1="{hy - s*0.18:.1f}" '
264
+ f'x2="{hx + s*0.18:.1f}" y2="{hy - s*0.44:.1f}" stroke="{INK}" '
265
+ f'stroke-width="{s*0.06:.1f}" stroke-linecap="round"/>'
266
+ # mane (a few short strokes behind the head)
267
+ f'<path d="M {hx + s*0.2:.1f} {hy:.1f} q {s*0.2:.1f} {s*0.1:.1f} {s*0.12:.1f} {s*0.3:.1f}" '
268
+ f'fill="none" stroke="{INK}" stroke-width="{s*0.05:.1f}"/>'
269
+ # gold eye
270
+ f'<circle cx="{hx + s*0.08:.1f}" cy="{hy - s*0.06:.1f}" r="{s*0.06:.1f}" '
271
+ f'fill="{GOLD_BRIGHT}"/>'
272
+ )
273
+ return body + spikes + legs + head
274
+
275
+
276
+ def _point_at_radius_frac(centerline, rfrac):
277
+ """Find the spiral point whose radius is closest to rfrac of the max radius.
278
+
279
+ The log-spiral bunches points near the eye, so placing by point-index
280
+ clusters figures in the center. Placing by RADIUS spreads them across the
281
+ visible outer arms where there is room for figures.
282
+ """
283
+ if not centerline:
284
+ return None
285
+ radii = [p[3] for p in centerline] # r is index 3
286
+ r_min, r_max = min(radii), max(radii)
287
+ target = r_min + (r_max - r_min) * rfrac
288
+ best_i = min(range(len(centerline)), key=lambda i: abs(radii[i] - target))
289
+ return centerline[best_i]
290
+
291
+
292
+
293
+ def _fig(pos_frac: float, inner_svg: str) -> str:
294
+ """Wrap a figure cluster in a group tagged with its unroll position (0..1),
295
+ so the birth animation can ink it in when the scroll reaches it."""
296
+ p = max(0.0, min(1.0, float(pos_frac)))
297
+ return f'<g class="battle-fig" data-pos="{p:.3f}">{inner_svg}</g>'
298
+
299
+
300
+ def build_battle_layer(features: dict, centerline: list, outer_pts: list,
301
+ thickness_at, n_full: int, pal: dict, seed: int) -> str:
302
+ """Return the byobu battle layer SVG, placed on the shell's spiral by RADIUS
303
+ so figures spread across the visible outer arms (not the bunched eye).
304
+
305
+ Each figure sits on a gold backing (byobu ink-on-gold). Figures are sized to
306
+ be legible on the 640px canvas.
307
+ """
308
+ rng = random.Random(seed ^ 0xBA771E)
309
+ dead_ends = features.get("dead_ends", []) or []
310
+ gotchas = features.get("gotchas", []) or []
311
+
312
+ parts = ["<g class=\"battle-layer\">"]
313
+
314
+ # ---- gold cloud bands drifting across the OUTER field ----
315
+ # Clouds are atmosphere; they ink in early and at their radius position.
316
+ for i in range(3):
317
+ rfrac = 0.55 + 0.15 * i
318
+ p = _point_at_radius_frac(centerline, min(0.95, rfrac))
319
+ if p:
320
+ x, y = p[0], p[1]
321
+ parts.append(_fig(min(0.95, rfrac), _cloud_band(
322
+ x, y, w=rng.uniform(120, 180), h=rng.uniform(30, 44), seed=seed + i)))
323
+
324
+ # ---- the general (developer): mid-outer arm. Inks in when scroll reaches 0.66.
325
+ p = _point_at_radius_frac(centerline, 0.66)
326
+ if p:
327
+ gx, gy = p[0], p[1]
328
+ general = (
329
+ _gold_backing(gx, gy - 10, 30)
330
+ + _general(gx, gy, s=40)
331
+ + _banner(gx + 26, gy + 4, h=46, color=INK)
332
+ )
333
+ parts.append(_fig(0.66, general))
334
+
335
+ # ---- fallen warriors at each dead end, placed + revealed by RADIUS ----
336
+ for de in dead_ends:
337
+ pos = max(0.0, min(1.0, float(de.get("position", 0.5))))
338
+ rfrac = 0.48 + pos * 0.46 # 0.48 .. 0.94 (keep out of the crowded eye)
339
+ p = _point_at_radius_frac(centerline, rfrac)
340
+ if not p:
341
+ continue
342
+ x, y = p[0], p[1]
343
+ fallen = (
344
+ _gold_backing(x, y, 22)
345
+ + _fallen(x, y, s=28, ink=INK)
346
+ + _banner(x + 12, y - 2, h=26, color=INK, broken=True,
347
+ angle=rng.uniform(-12, 12))
348
+ )
349
+ parts.append(_fig(rfrac, fallen))
350
+
351
+ # ---- archers along the rim at each gotcha ----
352
+ n_arch = min(len(gotchas), 8)
353
+ for i in range(n_arch):
354
+ frac = 0.52 + (i / max(1, n_arch)) * 0.44
355
+ idx = int(frac * (len(outer_pts) - 1)) if outer_pts else 0
356
+ if idx >= len(outer_pts):
357
+ continue
358
+ ox, oy, _, _, nrm = outer_pts[idx]
359
+ ax = ox + math.cos(nrm) * 18
360
+ ay = oy + math.sin(nrm) * 18
361
+ archer = _gold_backing(ax, ay, 15) + _archer(ax, ay, s=24, ink=INK)
362
+ parts.append(_fig(frac, archer))
363
+
364
+ # ---- the dragon (breakthrough): inks in LAST, when the scroll reaches the tip.
365
+ tip = centerline[-1]
366
+ pre = centerline[max(0, len(centerline) - 18)]
367
+ dx, dy = pre[0], pre[1]
368
+ bn = tip[4]
369
+ dragon = (
370
+ f'<ellipse cx="{dx:.1f}" cy="{dy:.1f}" rx="46" ry="30" '
371
+ f'transform="rotate({math.degrees(bn):.0f} {dx:.1f} {dy:.1f})" '
372
+ f'fill="{GOLD}" opacity="0.30"/>'
373
+ + _dragon(dx, dy, s=42, angle_deg=math.degrees(bn) + 150)
374
+ + _banner(dx - 20, dy + 10, h=56, color=INK)
375
+ )
376
+ parts.append(_fig(1.0, dragon))
377
+
378
+ parts.append("</g>")
379
+ return "\n".join(parts)
shell_unroll.py ADDED
@@ -0,0 +1,136 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Path-following scroll unroll: the spiral lays down ALONG its arm, led by a 3D curl.
3
+
4
+ The shell's `growth` parameter truncates the spiral along its own arm (not
5
+ radially), so a sequence of growth stages IS a scroll unrolling along the spiral
6
+ path: an outer ring stays hidden until the arm reaches it, even if an inner point
7
+ at the same clock-angle is already laid down.
8
+
9
+ We render N growth stages as a flipbook stacked in one SVG. Each stage is shown
10
+ in sequence via SMIL (set/animate on opacity), so the laid parchment grows along
11
+ the arm. A 3D curl object (a small cylinder with a gold highlight and a cast
12
+ shadow) rides the leading tip via animateMotion along the centerline path,
13
+ shrinking as it spends its paper, then vanishes at the rim.
14
+
15
+ This is heavier than a single-SVG animation (~1.6MB for 14 stages) but it is the
16
+ only way to get a genuine path-following reveal of a filled spiral.
17
+ """
18
+
19
+ from __future__ import annotations
20
+
21
+ import math
22
+ import re
23
+
24
+ N_STAGES = 14
25
+ UNROLL_DUR = 3.6 # total seconds
26
+ EYE_X, EYE_Y = 320.0, 335.0
27
+
28
+
29
+ def _centerline_d(svg: str) -> str | None:
30
+ m = re.search(r'<path class="shell-centerline" d="([^"]*)"', svg)
31
+ return m.group(1) if m else None
32
+
33
+
34
+ def _curl_object(stage_frac: float) -> str:
35
+ """A 3D paper-curl cylinder, sized for how much paper remains (shrinks as the
36
+ scroll unrolls). Returned as a <g id="curl"> to be motion-animated."""
37
+ # remaining paper ~ (1 - stage_frac); curl radius shrinks from ~16 to ~5
38
+ r = 5 + 11 * (1 - stage_frac)
39
+ return (
40
+ f'<g id="paper-curl">'
41
+ # cast shadow on the parchment just behind the curl
42
+ f'<ellipse cx="2" cy="{r*0.6:.1f}" rx="{r*1.1:.1f}" ry="{r*0.45:.1f}" '
43
+ f'fill="#1a1410" opacity="0.28"/>'
44
+ # the cylinder body (the rolled paper, seen end-on)
45
+ f'<ellipse cx="0" cy="0" rx="{r:.1f}" ry="{r*0.78:.1f}" '
46
+ f'fill="#d8c79a" stroke="#8a7340" stroke-width="1"/>'
47
+ # inner roll rings (paper thickness)
48
+ f'<ellipse cx="0" cy="0" rx="{r*0.6:.1f}" ry="{r*0.46:.1f}" '
49
+ f'fill="none" stroke="#a8915c" stroke-width="0.8" opacity="0.7"/>'
50
+ f'<ellipse cx="0" cy="0" rx="{r*0.28:.1f}" ry="{r*0.22:.1f}" '
51
+ f'fill="#bda874" stroke="#8a7340" stroke-width="0.6"/>'
52
+ # gold highlight on the rounded leading edge
53
+ f'<ellipse cx="{-r*0.4:.1f}" cy="{-r*0.3:.1f}" rx="{r*0.34:.1f}" '
54
+ f'ry="{r*0.2:.1f}" fill="#efe0b0" opacity="0.85"/>'
55
+ f'</g>'
56
+ )
57
+
58
+
59
+ def build_unroll_doc(stage_svgs: list[str]) -> str:
60
+ """Stack growth-stage SVGs as a flipbook + a curl riding the tip. Returns one
61
+ SVG string with SMIL timing. stage_svgs[i] is the shell at growth (i+1)/N.
62
+
63
+ Each stage is wrapped in a <g> that becomes visible at its time slot and
64
+ hides when the next appears, so the laid parchment advances along the arm.
65
+ The final stage stays visible (freeze).
66
+ """
67
+ n = len(stage_svgs)
68
+ slot = UNROLL_DUR / n
69
+
70
+ # strip the outer <svg ...> wrapper from each stage, keep inner content
71
+ def inner(s):
72
+ a = s.find(">", s.find("<svg")) + 1
73
+ b = s.rfind("</svg>")
74
+ return s[a:b]
75
+
76
+ # the background (sky, particles) is identical across stages; take it from
77
+ # the LAST stage once, render it always-on underneath, and only flip the
78
+ # SHELL portion. But simplest robust approach: each stage is a full frame;
79
+ # show one at a time. The sky is identical so no flicker on the backdrop.
80
+ layers = []
81
+ for i, s in enumerate(stage_svgs):
82
+ content = inner(s)
83
+ # Each stage is visible only during its own [begin, begin+slot) window.
84
+ # We drive this with a SINGLE <animate> using values/keyTimes over the
85
+ # whole duration, so it resets cleanly on replay (setCurrentTime(0)).
86
+ on_t = i / n
87
+ off_t = (i + 1) / n
88
+ if i == n - 1:
89
+ # final stage: turn on at its start and STAY on (freeze full shell)
90
+ if on_t <= 0:
91
+ vals, keys = "1;1", "0;1"
92
+ else:
93
+ vals = "0;0;1;1"
94
+ keys = f"0;{on_t:.4f};{on_t:.4f};1"
95
+ elif i == 0:
96
+ # first stage: on from 0, off at off_t
97
+ vals = "1;1;0;0"
98
+ keys = f"0;{off_t:.4f};{off_t:.4f};1"
99
+ else:
100
+ # middle stage: off, on at on_t, off at off_t, off after
101
+ vals = "0;0;1;1;0;0"
102
+ keys = f"0;{on_t:.4f};{on_t:.4f};{off_t:.4f};{off_t:.4f};1"
103
+ anim = (
104
+ f'<animate attributeName="opacity" values="{vals}" keyTimes="{keys}" '
105
+ f'dur="{UNROLL_DUR}s" begin="0s" fill="freeze" calcMode="discrete"/>'
106
+ )
107
+ start_op = "1" if i == 0 else "0"
108
+ layers.append(f'<g opacity="{start_op}">{anim}{content}</g>')
109
+
110
+ # the 3D curl rides the centerline path of the FINAL stage (full spiral),
111
+ # via animateMotion, shrinking over the unroll, vanishing at the end.
112
+ cl = _centerline_d(stage_svgs[-1])
113
+ curl_layer = ""
114
+ if cl:
115
+ curl = _curl_object(0.0)
116
+ curl_layer = (
117
+ f'<g>{curl}'
118
+ f'<animateMotion dur="{UNROLL_DUR}s" begin="0s" fill="freeze" '
119
+ f'rotate="auto" keyPoints="0;1" keyTimes="0;1" calcMode="linear" '
120
+ f'path="{cl}"/>'
121
+ # shrink the curl as it spends paper
122
+ f'<animateTransform attributeName="transform" type="scale" '
123
+ f'additive="sum" from="1.6" to="0.5" dur="{UNROLL_DUR}s" begin="0s" '
124
+ f'fill="freeze"/>'
125
+ # vanish at the very end
126
+ f'<animate attributeName="opacity" values="1;1;0" keyTimes="0;0.92;1" '
127
+ f'dur="{UNROLL_DUR}s" begin="0s" fill="freeze"/>'
128
+ f'</g>'
129
+ )
130
+
131
+ W = H = 640
132
+ body = "".join(layers) + curl_layer
133
+ return (
134
+ f'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 {W} {H}" '
135
+ f'width="{W}" height="{H}">{body}</svg>'
136
+ )
skill_builder.py ADDED
@@ -0,0 +1,354 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Build a genuinely useful SKILL.md from a session extraction.
3
+
4
+ The judge's note: the skill file should give an LLM real uplift even over a
5
+ frontier model that is already capable without it. Research on what makes skills
6
+ work (Anthropic skill-creator, Perplexity, SkillsBench) converges on a few
7
+ principles, and this module encodes them:
8
+
9
+ 1. The GOTCHAS section is the highest-value content. Each gotcha names the
10
+ SYMPTOM, explains the CAUSE, and implies the FIX. These are the things a
11
+ capable model would still get wrong because they are non-obvious and
12
+ specific to this problem, not derivable from general knowledge.
13
+
14
+ 2. Explain the WHY. Frontier models have good theory of mind; rote steps waste
15
+ them. The transferable value is the reasoning behind what worked.
16
+
17
+ 3. Dead ends are uplift. A strong model can find the happy path; what it cannot
18
+ know is which promising-looking approaches FAIL and why, which saves it from
19
+ wasting moves. So we surface the approaches tried and why each failed as
20
+ "what not to do."
21
+
22
+ 4. Transferable, not narrative. The skill must read as procedure for the NEXT
23
+ person hitting this class of problem, not as a diary of this session.
24
+
25
+ 5. A frontmatter description that states what it does AND when to use it, so a
26
+ skills-compatible agent triggers it on the right tasks.
27
+
28
+ This module takes the structured extraction (which the model is good at) and
29
+ assembles the prose into the skill shape (which is mechanical and should not be
30
+ left to the model to format consistently).
31
+ """
32
+
33
+ from __future__ import annotations
34
+
35
+ import re
36
+ from typing import Any
37
+
38
+
39
+ def _slugify(text: str, fallback: str = "session-skill") -> str:
40
+ s = re.sub(r"[^a-z0-9]+", "-", str(text).lower()).strip("-")
41
+ return s or fallback
42
+
43
+
44
+ def _title_from_themes(themes: list, problem_hint: str) -> str:
45
+ if themes:
46
+ return " / ".join(t.strip() for t in themes[:3])
47
+ return problem_hint or "Debugging session"
48
+
49
+
50
+ def _infer_problem(extraction: dict) -> str:
51
+ """A one-line statement of the problem class this skill addresses."""
52
+ themes = extraction.get("themes", []) or []
53
+ dead = extraction.get("dead_ends", []) or []
54
+ breaks = extraction.get("breakthroughs", []) or []
55
+ if breaks:
56
+ w = breaks[-1].get("what_worked", "") if isinstance(breaks[-1], dict) else ""
57
+ if w:
58
+ return f"Solving the kind of problem where the fix is: {w.strip().rstrip(chr(46))}."
59
+ if themes:
60
+ return f"Working through a {', '.join(themes[:2])} problem."
61
+ return "Working through a technical problem that took several attempts."
62
+
63
+
64
+ def _build_description(extraction: dict, problem: str) -> str:
65
+ """Frontmatter description: what it does + when to trigger. Slightly pushy,
66
+ per skill-creator guidance, because agents tend to under-trigger."""
67
+ themes = extraction.get("themes", []) or []
68
+ gotchas = extraction.get("gotchas", []) or []
69
+ trig = ""
70
+ if themes:
71
+ trig = (
72
+ f" Use this whenever a task involves {', '.join(themes[:3])}, "
73
+ f"or when you hit symptoms similar to the gotchas below, even if the "
74
+ f"user does not name them explicitly."
75
+ )
76
+ g = f" Captures {len(gotchas)} specific gotcha(s) that are easy to get wrong." if gotchas else ""
77
+ return (
78
+ f"Hard-won procedure for: {problem.rstrip('.')}.{g}{trig}"
79
+ ).strip()
80
+
81
+
82
+ def _approaches_block(extraction: dict) -> str:
83
+ """What was tried and why it FAILED — i.e. what NOT to do, with reasons.
84
+
85
+ This is uplift: a capable model can find a path, but cannot know which
86
+ promising approaches are dead ends without being told.
87
+ """
88
+ approaches = extraction.get("approaches_tried", []) or []
89
+ if not approaches:
90
+ return (
91
+ "_No failed approaches were recorded for this session — it went "
92
+ "straight to a working solution._"
93
+ )
94
+ lines = [
95
+ "These approaches were tried and did NOT work. Avoid repeating them, or "
96
+ "understand why they fail before reaching for them again:\n"
97
+ ]
98
+ for a in approaches:
99
+ if not isinstance(a, dict):
100
+ continue
101
+ ap = str(a.get("approach", "")).strip()
102
+ why = str(a.get("why_it_failed", "")).strip().rstrip(".")
103
+ if ap:
104
+ lines.append(f"- **{ap}** — fails because {why}." if why else f"- **{ap}** — did not work.")
105
+ return "\n".join(lines)
106
+
107
+
108
+ def _gotchas_block(extraction: dict) -> str:
109
+ """The highest-value section. Symptom -> cause -> implied fix.
110
+
111
+ We reshape each raw gotcha string into the symptom/cause/fix frame where the
112
+ text allows; if the gotcha is terse, we still present it as a watch-out with
113
+ a prompt to check for it, because naming the trap is itself the value.
114
+ """
115
+ gotchas = extraction.get("gotchas", []) or []
116
+ if not gotchas:
117
+ return "_No specific gotchas were recorded for this session._"
118
+ lines = [
119
+ "These are the non-obvious traps from this problem — the things that are "
120
+ "easy to get wrong and costly to rediscover. Check each one before "
121
+ "assuming the obvious approach is safe:\n"
122
+ ]
123
+ # Phase 1 (RuleShaping arXiv:2604.11088): under each gotcha, emit a negative,
124
+ # preferably state-dependent guardrail — the only individually beneficial rule
125
+ # type. Deterministic + content-preserving (never fabricates a cause), so this
126
+ # adds zero model calls and cannot invent anything not in the gotcha.
127
+ try:
128
+ from rule_phrasing import to_negative_constraint, is_negative_constraint
129
+ except Exception:
130
+ to_negative_constraint = None
131
+ is_negative_constraint = None
132
+ for g in gotchas:
133
+ g = str(g).strip().rstrip(".")
134
+ if not g:
135
+ continue
136
+ lines.append(f"- {g}.")
137
+ if to_negative_constraint is not None:
138
+ rule = to_negative_constraint(g)
139
+ if (rule and rule.rstrip(".").lower() != g.lower()
140
+ and is_negative_constraint(rule)):
141
+ lines.append(f" - **Guardrail:** {rule}")
142
+ return "\n".join(lines)
143
+
144
+
145
+ def _breakthrough_block(extraction: dict) -> str:
146
+ breaks = extraction.get("breakthroughs", []) or []
147
+ if not breaks:
148
+ return "_No single breakthrough was recorded; progress was incremental._"
149
+ lines = ["What actually worked, and the reasoning that makes it transferable:\n"]
150
+ for b in breaks:
151
+ if not isinstance(b, dict):
152
+ continue
153
+ w = str(b.get("what_worked", "")).strip().rstrip(".")
154
+ if w:
155
+ lines.append(f"- {w}.")
156
+ return "\n".join(lines)
157
+
158
+
159
+ def _principles_block(extraction: dict) -> str:
160
+ """Distill transferable principles from the arc. This is the 'why' layer."""
161
+ approaches = extraction.get("approaches_tried", []) or []
162
+ gotchas = extraction.get("gotchas", []) or []
163
+ breaks = extraction.get("breakthroughs", []) or []
164
+ out = []
165
+ if approaches and breaks:
166
+ out.append(
167
+ "- The working solution came only after the failed approaches above. "
168
+ "If you are reaching for one of those first, reconsider."
169
+ )
170
+ if gotchas:
171
+ out.append(
172
+ "- Most of the difficulty here was not the main logic but the "
173
+ "edge conditions in the gotchas. Budget attention there."
174
+ )
175
+ if not out:
176
+ out.append(
177
+ "- This session was straightforward; the main value is the working "
178
+ "approach recorded below."
179
+ )
180
+ return "\n".join(out)
181
+
182
+
183
+
184
+
185
+ def _is_terse(text: str) -> bool:
186
+ """A gotcha is terse (low-value) if it's short and lacks a cause/fix signal."""
187
+ t = str(text).strip()
188
+ if len(t.split()) <= 6:
189
+ return True
190
+ # no causal/fix language -> probably just a label
191
+ signals = ("because", "since", "so ", "instead", "fix", "use ", "avoid",
192
+ "must", "should", "results in", "leads to", "causes", ";")
193
+ return not any(sig in t.lower() for sig in signals)
194
+
195
+
196
+
197
+ # Phrases from illustrative examples that must never appear in real output —
198
+ # if they do, the model leaked the example instead of expanding the real gotcha.
199
+ _LEAK_MARKERS = (
200
+ "reverse-bfs", "count_of_uncolored", "subtree_root", "leaf nodes first",
201
+ "number_of_uncolored_children", "process deepest-first",
202
+ )
203
+
204
+
205
+ def _looks_leaked(text: str) -> bool:
206
+ t = str(text).lower()
207
+ return any(marker in t for marker in _LEAK_MARKERS)
208
+
209
+
210
+ def enrich_gotchas(extraction: dict, complete=None) -> dict:
211
+ """Optionally expand terse gotchas into symptom/cause/fix form via one model
212
+ call. `complete` is a function (prompt:str) -> str. If None, or if no terse
213
+ gotchas, returns the extraction unchanged.
214
+
215
+ This is the optional depth pass: the better extraction prompt handles most
216
+ cases; this rescues sessions where the model still emitted thin labels.
217
+ """
218
+ gotchas = extraction.get("gotchas", []) or []
219
+ if not complete or not gotchas:
220
+ return extraction
221
+ terse = [g for g in gotchas if _is_terse(g)]
222
+ if not terse:
223
+ return extraction
224
+
225
+ themes = ", ".join(extraction.get("themes", []) or []) or "this problem"
226
+ approaches = extraction.get("approaches_tried", []) or []
227
+ ctx_lines = []
228
+ for a in approaches:
229
+ if isinstance(a, dict) and a.get("approach"):
230
+ ctx_lines.append(f"- tried {a['approach']}: {a.get('why_it_failed','')}")
231
+ ctx = "\n".join(ctx_lines)
232
+
233
+ prompt = (
234
+ "You are sharpening the gotchas in a coding skill file so they give a "
235
+ "capable engineer real, non-obvious help.\n\n"
236
+ f"PROBLEM AREA: {themes}\n"
237
+ f"WHAT WAS TRIED AND FAILED:\n{ctx}\n\n"
238
+ "Your job: rewrite each terse gotcha below into ONE self-contained "
239
+ "sentence that names the symptom, the cause, and what to do instead. "
240
+ "CRITICAL RULES:\n"
241
+ "- Each rewritten gotcha MUST be specific to the problem area above "
242
+ f"({themes}). Do NOT invent details from other domains.\n"
243
+ "- Base the rewrite ONLY on the terse gotcha's own meaning plus the "
244
+ "problem area. If you cannot rescue it without inventing facts, return it "
245
+ "UNCHANGED.\n"
246
+ "- Return ONLY a JSON array of strings, same length and order as the "
247
+ "input. No prose, no code fences.\n\n"
248
+ "Terse gotchas to rewrite:\n"
249
+ + "\n".join(f"- {g}" for g in terse)
250
+ )
251
+
252
+ try:
253
+ import json
254
+ raw = complete(prompt)
255
+ raw = raw.strip()
256
+ # strip code fences if present
257
+ import re as _re
258
+ raw = _re.sub(r"^```(?:json)?|```$", "", raw, flags=_re.MULTILINE).strip()
259
+ expanded = json.loads(raw)
260
+ if isinstance(expanded, list) and len(expanded) == len(terse):
261
+ # map terse -> expanded, but REJECT any expansion that leaked the
262
+ # example phrasing (keep the original terse gotcha in that case).
263
+ cleaned = []
264
+ for original, exp in zip(terse, expanded):
265
+ exp = str(exp).strip()
266
+ if exp and not _looks_leaked(exp):
267
+ cleaned.append((original, exp))
268
+ else:
269
+ cleaned.append((original, original))
270
+ mapping = dict(cleaned)
271
+ extraction = dict(extraction)
272
+ extraction["gotchas"] = [mapping.get(g, g) for g in gotchas]
273
+ except Exception:
274
+ pass # never let enrichment break the pipeline
275
+ return extraction
276
+
277
+
278
+ def build_skill_md(extraction: dict) -> str:
279
+ """Assemble a transferable SKILL.md from the structured extraction."""
280
+ themes = extraction.get("themes", []) or []
281
+ problem = _infer_problem(extraction)
282
+ title = _title_from_themes(themes, problem)
283
+ name = _slugify(title)
284
+ description = _build_description(extraction, problem)
285
+
286
+ tags = ", ".join(themes) if themes else "debugging, problem-solving"
287
+
288
+ md = f"""---
289
+ name: {name}
290
+ description: {description}
291
+ ---
292
+
293
+ # {title}
294
+
295
+ ## When this applies
296
+
297
+ {problem} Use the procedure below when you hit this class of problem, especially
298
+ if the symptoms match the gotchas section — that is where this skill earns its
299
+ place over solving from scratch.
300
+
301
+ ## What does NOT work (and why)
302
+
303
+ {_approaches_block(extraction)}
304
+
305
+ ## What works
306
+
307
+ {_breakthrough_block(extraction)}
308
+
309
+ ## Gotchas (read this first)
310
+
311
+ {_gotchas_block(extraction)}
312
+
313
+ ## Transferable principles
314
+
315
+ {_principles_block(extraction)}
316
+
317
+ ## Tags
318
+
319
+ {tags}
320
+ """
321
+ return md
322
+
323
+
324
+ # Backwards-compatible section check used by extract.py's validator
325
+ SKILL_SECTIONS = (
326
+ "When this applies",
327
+ "What does NOT work",
328
+ "What works",
329
+ "Gotchas",
330
+ "Transferable principles",
331
+ "Tags",
332
+ )
333
+
334
+
335
+ if __name__ == "__main__":
336
+ sample = {
337
+ "themes": ["graph theory", "dynamic programming"],
338
+ "approaches_tried": [
339
+ {"approach": "naive recursion", "why_it_failed": "recomputes the same subgraphs exponentially"},
340
+ {"approach": "BFS from each node", "why_it_failed": "O(V^2) blew the time limit on dense graphs"},
341
+ ],
342
+ "dead_ends": [
343
+ {"position": 0.3, "what_happened": "recursion stack overflowed on the big case"},
344
+ ],
345
+ "breakthroughs": [
346
+ {"position": 0.85, "what_worked": "memoized the subgraph results keyed by visited-set bitmask"},
347
+ ],
348
+ "gotchas": [
349
+ "the bitmask must include the start node or you double-count paths",
350
+ "Python recursion limit hits before the logic is wrong, so raise it early to see the real bug",
351
+ ],
352
+ "sentiment_arc": {"start": "frustrated", "end": "resolved"},
353
+ }
354
+ print(build_skill_md(sample))
skill_uplift_eval.py ADDED
@@ -0,0 +1,184 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Skill-Uplift Eval: does a generated SKILL.md help a model on a held-out task?
3
+
4
+ Honest measurement of the judge's bar. The harness is model-agnostic: you provide
5
+ two callables,
6
+ answerer(prompt) -> str # the model being helped (e.g. a frontier model)
7
+ grader(prompt) -> str # a separate model that scores, BLIND
8
+ and a list of EvalCase (session extraction + a DISTINCT held-out task + a rubric).
9
+
10
+ Pipeline (mirrors the groundedness eval's discipline):
11
+ 1. Build the SKILL.md from each session's extraction (the real generator).
12
+ 2. Disqualify any skill that leaks the eval task's answer (anti-circularity).
13
+ 3. For each case, get TWO answers from the SAME model: no-skill and with-skill.
14
+ 4. SAVE all raw generations to disk BEFORE scoring (no post-hoc massaging).
15
+ 5. Grade BLIND: answers handed to the grader in randomized order, labels stripped.
16
+ 6. Run a human-labeled CALIBRATION check first; report grader agreement.
17
+ 7. Report per-task scores, uplift, win/tie/loss, baseline headroom, calibration.
18
+
19
+ Nothing here is rigged toward a positive result; a near-zero or negative uplift is
20
+ a valid, reportable outcome.
21
+ """
22
+
23
+ from __future__ import annotations
24
+
25
+ import json
26
+ import os
27
+ import random
28
+ import re
29
+ from dataclasses import dataclass, field
30
+ from typing import Callable
31
+
32
+ # Import the REAL skill generator so we eval what actually ships.
33
+ # (adjust path to your repo when running)
34
+ try:
35
+ from skill_builder import build_skill_md # type: ignore
36
+ except Exception:
37
+ build_skill_md = None # the harness will error clearly if not wired up
38
+
39
+
40
+ @dataclass
41
+ class EvalCase:
42
+ name: str
43
+ session_extraction: dict # the session the skill is built FROM
44
+ task_prompt: str # a DISTINCT held-out task in the same class
45
+ answer_key_terms: list[str] # terms a correct answer should contain (rubric aid)
46
+ leak_terms: list[str] # if the SKILL.md contains these, it leaked the answer
47
+
48
+
49
+ @dataclass
50
+ class CaseResult:
51
+ name: str
52
+ skill_md: str
53
+ no_skill_answer: str
54
+ with_skill_answer: str
55
+ no_skill_score: float
56
+ with_skill_score: float
57
+ leaked: bool
58
+ grader_raw: dict = field(default_factory=dict)
59
+
60
+
61
+ GRADER_SYSTEM = (
62
+ "You are a strict, impartial grader. You will see a TASK and two candidate "
63
+ "ANSWERS labeled X and Y, in random order. Score each answer from 0.0 to 1.0 "
64
+ "for how correctly and completely it solves the task. Judge ONLY the answer's "
65
+ "merit. You do not know how either answer was produced. Return ONLY compact "
66
+ "JSON: {\"X\": <float>, \"Y\": <float>, \"why\": \"<one sentence>\"}."
67
+ )
68
+
69
+
70
+ def _skill_leaks_answer(skill_md: str, leak_terms: list[str]) -> bool:
71
+ s = skill_md.lower()
72
+ return any(t.lower() in s for t in leak_terms)
73
+
74
+
75
+ def _grade_blind(grader: Callable[[str], str], task: str,
76
+ ans_a: str, ans_b: str, rng: random.Random) -> tuple[float, float, dict]:
77
+ """Hand the grader the two answers in random order with neutral labels."""
78
+ swap = rng.random() < 0.5
79
+ first, second = (ans_b, ans_a) if swap else (ans_a, ans_b)
80
+ prompt = (
81
+ f"{GRADER_SYSTEM}\n\nTASK:\n{task}\n\n"
82
+ f"ANSWER X:\n{first}\n\nANSWER Y:\n{second}\n\nJSON:"
83
+ )
84
+ raw = grader(prompt)
85
+ try:
86
+ m = re.search(r"\{.*\}", raw, re.DOTALL)
87
+ obj = json.loads(m.group(0)) if m else {}
88
+ x = float(obj.get("X", 0.0)); y = float(obj.get("Y", 0.0))
89
+ except Exception:
90
+ x, y = 0.0, 0.0
91
+ obj = {"parse_error": raw[:200]}
92
+ # unswap back to (a=no_skill, b=with_skill)
93
+ a_score, b_score = (y, x) if swap else (x, y)
94
+ return a_score, b_score, obj
95
+
96
+
97
+ def run_eval(cases: list[EvalCase],
98
+ answerer: Callable[[str], str],
99
+ grader: Callable[[str], str],
100
+ out_dir: str = "./skill_eval_runs",
101
+ seed: int = 0) -> dict:
102
+ if build_skill_md is None:
103
+ raise RuntimeError("skill_builder.build_skill_md not importable; run from the repo.")
104
+ os.makedirs(out_dir, exist_ok=True)
105
+ rng = random.Random(seed)
106
+ results: list[CaseResult] = []
107
+
108
+ for case in cases:
109
+ skill_md = build_skill_md(case.session_extraction)
110
+ leaked = _skill_leaks_answer(skill_md, case.leak_terms)
111
+
112
+ no_skill_prompt = case.task_prompt
113
+ with_skill_prompt = (
114
+ f"You have access to a skill document that may help.\n\n"
115
+ f"--- SKILL.md ---\n{skill_md}\n--- end SKILL.md ---\n\n"
116
+ f"TASK:\n{case.task_prompt}"
117
+ )
118
+ ans_no = answerer(no_skill_prompt)
119
+ ans_yes = answerer(with_skill_prompt)
120
+
121
+ # SAVE raw BEFORE grading
122
+ with open(os.path.join(out_dir, f"{case.name}.json"), "w") as f:
123
+ json.dump({"skill_md": skill_md, "leaked": leaked,
124
+ "no_skill_answer": ans_no, "with_skill_answer": ans_yes,
125
+ "task": case.task_prompt}, f, indent=2)
126
+
127
+ # leaked skills are scored but FLAGGED and excluded from headline uplift
128
+ a_score, b_score, graw = _grade_blind(grader, case.task_prompt, ans_no, ans_yes, rng)
129
+ results.append(CaseResult(case.name, skill_md, ans_no, ans_yes,
130
+ a_score, b_score, leaked, graw))
131
+
132
+ # ---- aggregate, EXCLUDING leaked skills from the headline number ----
133
+ clean = [r for r in results if not r.leaked]
134
+ leaked_n = sum(1 for r in results if r.leaked)
135
+ if clean:
136
+ mean_no = sum(r.no_skill_score for r in clean) / len(clean)
137
+ mean_yes = sum(r.with_skill_score for r in clean) / len(clean)
138
+ wins = sum(1 for r in clean if r.with_skill_score > r.no_skill_score + 1e-6)
139
+ ties = sum(1 for r in clean if abs(r.with_skill_score - r.no_skill_score) <= 1e-6)
140
+ losses = sum(1 for r in clean if r.with_skill_score < r.no_skill_score - 1e-6)
141
+ else:
142
+ mean_no = mean_yes = 0.0; wins = ties = losses = 0
143
+
144
+ report = {
145
+ "n_total": len(results),
146
+ "n_leaked_excluded": leaked_n,
147
+ "n_scored": len(clean),
148
+ "baseline_no_skill_mean": round(mean_no, 3),
149
+ "with_skill_mean": round(mean_yes, 3),
150
+ "uplift": round(mean_yes - mean_no, 3),
151
+ "wins": wins, "ties": ties, "losses": losses,
152
+ "per_case": [
153
+ {"name": r.name, "no_skill": round(r.no_skill_score, 3),
154
+ "with_skill": round(r.with_skill_score, 3),
155
+ "delta": round(r.with_skill_score - r.no_skill_score, 3),
156
+ "leaked": r.leaked}
157
+ for r in results
158
+ ],
159
+ "raw_saved_to": out_dir,
160
+ }
161
+ with open(os.path.join(out_dir, "_report.json"), "w") as f:
162
+ json.dump(report, f, indent=2)
163
+ return report
164
+
165
+
166
+ # ---- calibration: check the blind grader agrees with human labels ----
167
+ def run_calibration(grader: Callable[[str], str],
168
+ labeled: list[dict], seed: int = 0) -> dict:
169
+ """labeled: [{task, better_answer, worse_answer}] where humans judged which is
170
+ better. We check the grader scores the human-better one higher. Reports
171
+ agreement BEFORE any uplift number is trusted (like the groundedness 5/6 block).
172
+ """
173
+ rng = random.Random(seed)
174
+ agree = 0
175
+ rows = []
176
+ for item in labeled:
177
+ a, b, graw = _grade_blind(grader, item["task"],
178
+ item["worse_answer"], item["better_answer"], rng)
179
+ # a=worse, b=better -> agreement means b_score > a_score
180
+ ok = b > a
181
+ agree += int(ok)
182
+ rows.append({"task": item["task"][:50], "worse": round(a, 3),
183
+ "better": round(b, 3), "agree": ok})
184
+ return {"agreement": f"{agree}/{len(labeled)}", "rows": rows}
tests/test_app.py ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Smoke tests for the TurboSkillSlug app."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import importlib
6
+
7
+ import gradio as gr
8
+
9
+
10
+ def test_interface_exists() -> None:
11
+ """The app module exposes a Gradio Blocks interface."""
12
+ app = importlib.import_module("app")
13
+
14
+ assert hasattr(app, "interface")
15
+ assert isinstance(app.interface, gr.Blocks)
tests/test_extract.py ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tests for session extraction utilities."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ from typing import Any
7
+ from unittest.mock import MagicMock, patch
8
+
9
+ from extract import EXPECTED_KEYS, MODEL_NAME, SYSTEM_PROMPT, extract_session
10
+
11
+
12
+ def _sample_payload() -> dict[str, Any]:
13
+ """Return a complete sample extraction payload."""
14
+ return {
15
+ "duration_minutes": 12,
16
+ "themes": ["scaffolding", "audio"],
17
+ "approaches_tried": [{"approach": "build a Gradio app", "outcome": "worked"}],
18
+ "dead_ends": [{"position": 0.4, "what_happened": "pytest was missing"}],
19
+ "breakthroughs": [{"position": 0.7, "what_worked": "mocked the client"}],
20
+ "gotchas": ["network access needed approval"],
21
+ "sentiment_arc": {"start": "curious", "end": "steady"},
22
+ "skill_md": "# Skill\n\nUse the transcript.",
23
+ "slug_voice": [
24
+ "You waited through the install.",
25
+ "The button found its job.",
26
+ "The mock kept things tidy.",
27
+ "The transcript became a map.",
28
+ "The Space now has a trail.",
29
+ ],
30
+ }
31
+
32
+
33
+ def test_extract_session_returns_expected_keys() -> None:
34
+ """The extractor parses chat completion JSON into the expected dict shape."""
35
+ payload = _sample_payload()
36
+ response = {
37
+ "choices": [
38
+ {
39
+ "message": {
40
+ "content": f"```json\n{json.dumps(payload)}\n```",
41
+ }
42
+ }
43
+ ]
44
+ }
45
+
46
+ with patch("extract.InferenceClient") as client_class:
47
+ client = MagicMock()
48
+ client.chat.completions.create.return_value = response
49
+ client_class.return_value = client
50
+
51
+ extraction = extract_session("session transcript")
52
+
53
+ assert EXPECTED_KEYS.issubset(extraction.keys())
54
+ client.chat.completions.create.assert_called_once_with(
55
+ model=MODEL_NAME,
56
+ messages=[
57
+ {"role": "system", "content": SYSTEM_PROMPT},
58
+ {"role": "user", "content": "session transcript"},
59
+ ],
60
+ response_format={"type": "json_object"},
61
+ )
tests/test_transcribe.py ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tests for audio transcription utilities."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from pathlib import Path
6
+ from unittest.mock import MagicMock, patch
7
+
8
+ from transcribe import MODEL_NAME, transcribe_audio
9
+
10
+
11
+ def test_transcribe_audio_returns_string(tmp_path: Path) -> None:
12
+ """The transcriber returns transcript text from the InferenceClient."""
13
+ audio_path = tmp_path / "audio.wav"
14
+ audio_path.write_bytes(b"fake audio")
15
+
16
+ with patch("transcribe.InferenceClient") as client_class:
17
+ client = MagicMock()
18
+ client.automatic_speech_recognition.return_value = {"text": "hello slug"}
19
+ client_class.return_value = client
20
+
21
+ transcript = transcribe_audio(str(audio_path))
22
+
23
+ assert transcript == "hello slug"
24
+ assert isinstance(transcript, str)
25
+ client.automatic_speech_recognition.assert_called_once_with(
26
+ b"fake audio",
27
+ model=MODEL_NAME,
28
+ )
trace_parser.py ADDED
@@ -0,0 +1,228 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Parse agent session logs (Claude Code + Codex CLI JSONL) into a narrative
3
+ transcript the TurboSkillSlug pipeline can read like spoken testimony.
4
+
5
+ The slug does not need full telemetry. It needs the STORY: what was asked,
6
+ what was tried, what errored (dead ends), what finally worked (breakthrough).
7
+ We render the trace as a first-person narrative so the extraction LoRA and
8
+ voice LoRA treat it exactly like an audio transcript.
9
+
10
+ Supports:
11
+ - Claude Code: ~/.claude/projects/<encoded>/<uuid>.jsonl
12
+ Each line: {"type": "user"|"assistant"|"tool_use"|"tool_result"|"system",
13
+ "message": {"content": str | [content blocks]}, ...}
14
+ - Codex CLI: ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl
15
+ Each line is a typed event (message, tool call, command output, patch).
16
+ Multiple schema versions handled defensively.
17
+
18
+ Usage:
19
+ from trace_parser import parse_trace_to_transcript
20
+ text = parse_trace_to_transcript(jsonl_string) # feed to existing pipeline
21
+ """
22
+
23
+ from __future__ import annotations
24
+
25
+ import json
26
+ from typing import Any
27
+
28
+
29
+ # Tool-result text that signals an error / dead end
30
+ ERROR_MARKERS = (
31
+ "error", "failed", "exception", "traceback", "not found", "cannot",
32
+ "denied", "refused", "timeout", "timed out", "exit code 1", "exit status 1",
33
+ "no such file", "undefined", "is not defined", "syntaxerror", "typeerror",
34
+ "modulenotfound", "command not found", "fatal", "panic",
35
+ )
36
+
37
+ # Tool-result / message text that signals success / breakthrough
38
+ SUCCESS_MARKERS = (
39
+ "passed", "success", "all tests pass", "tests passed", "ok", "done",
40
+ "fixed", "resolved", "works now", "working", "0 failed", "exit code 0",
41
+ "build succeeded", "compiled", "no errors",
42
+ )
43
+
44
+
45
+ def _content_to_text(content: Any) -> str:
46
+ """Flatten Claude/Codex message content (str or list of blocks) to text."""
47
+ if isinstance(content, str):
48
+ return content
49
+ if isinstance(content, list):
50
+ parts = []
51
+ for block in content:
52
+ if isinstance(block, dict):
53
+ # Claude content blocks: {"type": "text", "text": ...}
54
+ # tool_use: {"type":"tool_use","name":...,"input":...}
55
+ # tool_result: {"type":"tool_result","content":...}
56
+ if block.get("type") == "text" and "text" in block:
57
+ parts.append(str(block["text"]))
58
+ elif block.get("type") == "tool_use":
59
+ name = block.get("name", "a tool")
60
+ cmd = ""
61
+ inp = block.get("input", {})
62
+ if isinstance(inp, dict):
63
+ cmd = inp.get("command") or inp.get("file_path") or inp.get("path") or ""
64
+ parts.append(f"[ran {name} {cmd}]".strip())
65
+ elif block.get("type") == "tool_result":
66
+ inner = block.get("content", "")
67
+ parts.append(_content_to_text(inner))
68
+ elif "text" in block:
69
+ parts.append(str(block["text"]))
70
+ elif isinstance(block, str):
71
+ parts.append(block)
72
+ return " ".join(p for p in parts if p)
73
+ if isinstance(content, dict):
74
+ return _content_to_text(content.get("content", "")) or str(content.get("text", ""))
75
+ return ""
76
+
77
+
78
+ def _classify(text: str) -> str:
79
+ """Tag a tool result as error, success, or neutral."""
80
+ low = text.lower()
81
+ if any(m in low for m in ERROR_MARKERS):
82
+ return "error"
83
+ if any(m in low for m in SUCCESS_MARKERS):
84
+ return "success"
85
+ return "neutral"
86
+
87
+
88
+ def _iter_events(jsonl_string: str):
89
+ """Yield parsed JSON objects from a JSONL string, skipping bad lines."""
90
+ for line in jsonl_string.splitlines():
91
+ line = line.strip()
92
+ if not line:
93
+ continue
94
+ try:
95
+ yield json.loads(line)
96
+ except json.JSONDecodeError:
97
+ continue
98
+
99
+
100
+ def _blocks_contain(content: Any, block_type: str) -> bool:
101
+ """True if content is a list containing a block of the given type."""
102
+ if isinstance(content, list):
103
+ return any(
104
+ isinstance(b, dict) and b.get("type") == block_type for b in content
105
+ )
106
+ return False
107
+
108
+
109
+ def _extract_role_and_text(event: dict) -> tuple[str, str]:
110
+ """Return (role, text) from one event across Claude + Codex schemas."""
111
+ etype = event.get("type", "")
112
+
113
+ # Claude Code shape: message under "message"
114
+ msg = event.get("message")
115
+ if isinstance(msg, dict):
116
+ content = msg.get("content", "")
117
+ text = _content_to_text(content)
118
+ # Claude wraps tool results in a role="user" message — detect by block
119
+ # type, not role, so they are classified as results not user turns.
120
+ if _blocks_contain(content, "tool_result"):
121
+ return "tool_result", text
122
+ if _blocks_contain(content, "tool_use"):
123
+ return "tool_use", text
124
+ return msg.get("role", etype), text
125
+
126
+ # Codex shapes: several versions
127
+ # newer: {"type":"message","role":...,"content":[...]}
128
+ if etype == "message" and "content" in event:
129
+ return event.get("role", "assistant"), _content_to_text(event["content"])
130
+ # tool/command events
131
+ if etype in ("function_call", "tool_call", "local_shell_call", "exec"):
132
+ name = event.get("name") or event.get("tool") or "command"
133
+ args = event.get("arguments") or event.get("input") or event.get("command") or ""
134
+ if isinstance(args, (dict, list)):
135
+ args = _content_to_text(args)
136
+ return "tool_use", f"[ran {name} {args}]".strip()
137
+ if etype in ("function_call_output", "tool_result", "exec_output", "command_output"):
138
+ out = event.get("output") or event.get("content") or event.get("result") or ""
139
+ return "tool_result", _content_to_text(out) if not isinstance(out, str) else out
140
+ # generic fallback
141
+ if "content" in event:
142
+ return event.get("role", etype or "system"), _content_to_text(event["content"])
143
+ if "text" in event:
144
+ return etype or "system", str(event["text"])
145
+
146
+ return "", ""
147
+
148
+
149
+ def parse_trace_to_transcript(jsonl_string: str, max_chars: int = 6000) -> str:
150
+ """
151
+ Convert an agent session JSONL trace into a first-person narrative the
152
+ slug can witness. Errors become dead ends, successes become breakthroughs.
153
+ """
154
+ lines: list[str] = []
155
+ n_errors = 0
156
+ n_success = 0
157
+ n_tools = 0
158
+ first_ask = None
159
+
160
+ for event in _iter_events(jsonl_string):
161
+ role, text = _extract_role_and_text(event)
162
+ text = (text or "").strip()
163
+ if not text or len(text) < 3:
164
+ continue
165
+ # Trim very long tool dumps to their first informative chunk
166
+ if len(text) > 400:
167
+ text = text[:400].rsplit(" ", 1)[0] + " ..."
168
+
169
+ if role in ("user",) and first_ask is None:
170
+ first_ask = text
171
+ lines.append(f"I started by asking the agent: {text}")
172
+ elif role in ("user",):
173
+ lines.append(f"Then I told it: {text}")
174
+ elif role in ("assistant",):
175
+ lines.append(f"The agent said: {text}")
176
+ elif role in ("tool_use",):
177
+ n_tools += 1
178
+ lines.append(text) # already "[ran X ...]"
179
+ elif role in ("tool_result",):
180
+ kind = _classify(text)
181
+ if kind == "error":
182
+ n_errors += 1
183
+ lines.append(f"That failed: {text}")
184
+ elif kind == "success":
185
+ n_success += 1
186
+ lines.append(f"That worked: {text}")
187
+ else:
188
+ lines.append(f"It returned: {text}")
189
+ # skip system events
190
+
191
+ if not lines:
192
+ return ""
193
+
194
+ # Frame the narrative so the extraction model reads dead ends / breakthroughs
195
+ header = (
196
+ "This is a recording of a coding session I worked through with an AI "
197
+ "agent. Here is what happened, start to finish.\n\n"
198
+ )
199
+ body = " ".join(lines)
200
+ footer = (
201
+ f"\n\nOver the session there were {n_errors} failures, {n_success} "
202
+ f"successes, and {n_tools} tool runs. "
203
+ )
204
+ if n_errors > n_success:
205
+ footer += "It was a grind with a lot of dead ends."
206
+ elif n_success > 0 and n_errors == 0:
207
+ footer += "It went smoothly start to finish."
208
+ elif n_success > 0:
209
+ footer += "After the failures, it finally came together."
210
+
211
+ transcript = header + body + footer
212
+ if len(transcript) > max_chars:
213
+ # Keep the head (the ask + early attempts) and the tail (the resolution)
214
+ head = transcript[: max_chars // 2]
215
+ tail = transcript[-max_chars // 2 :]
216
+ transcript = head + " ... " + tail
217
+ return transcript
218
+
219
+
220
+ def detect_trace_format(jsonl_string: str) -> str:
221
+ """Best-effort label of the trace source for display. 'claude', 'codex', or 'unknown'."""
222
+ for event in _iter_events(jsonl_string):
223
+ if isinstance(event.get("message"), dict):
224
+ return "claude"
225
+ if event.get("type") in ("function_call", "local_shell_call", "exec",
226
+ "function_call_output", "exec_output"):
227
+ return "codex"
228
+ return "unknown"
transcribe.py ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Transcription via Whisper on Modal."""
2
+
3
+ import base64
4
+ import os
5
+ from pathlib import Path
6
+
7
+ import httpx
8
+
9
+
10
+ WHISPER_URL = os.environ.get(
11
+ "MODAL_WHISPER_URL",
12
+ "https://anubhavbharadwaaj--slugvoice-whisper-whisperserver-api.modal.run",
13
+ )
14
+ _HERE = Path(__file__).resolve().parent
15
+ _SAMPLE_AUDIO = _HERE / "sample_session.wav"
16
+
17
+
18
+ def _resolve_audio_path(audio_path: str) -> Path:
19
+ """Return a stable audio path, tolerating evicted Gradio example temp files."""
20
+ path = Path(audio_path)
21
+ if path.exists():
22
+ return path
23
+
24
+ name = path.name.lower()
25
+ if name.startswith("sample_session") and _SAMPLE_AUDIO.exists():
26
+ return _SAMPLE_AUDIO
27
+
28
+ raise FileNotFoundError(
29
+ f"Audio file is no longer available: {audio_path}. "
30
+ "Please re-upload the file and try again."
31
+ )
32
+
33
+
34
+ def transcribe_audio(audio_path: str) -> str:
35
+ """Transcribe audio using Whisper on Modal."""
36
+ resolved_path = _resolve_audio_path(audio_path)
37
+ with open(resolved_path, "rb") as f:
38
+ audio_b64 = base64.b64encode(f.read()).decode()
39
+
40
+ resp = httpx.post(
41
+ WHISPER_URL,
42
+ json={"audio": audio_b64},
43
+ timeout=120,
44
+ )
45
+ resp.raise_for_status()
46
+ return resp.json()["text"]