HONEST_SUBMISSION.md

# Honest Submission Notes — TurboSkillSlug

This document is the unvarnished account of what TurboSkillSlug actually does,
what is genuinely strong, what is partial, and what is aspirational. If a claim
is not in here with its caveats, treat the marketing copy as marketing.

Demo video: https://youtu.be/qSP9olWRv7o
Space: https://huggingface.co/spaces/build-small-hackathon/TurboSkillSlug
Code: https://github.com/AnubhavBharadwaaj/turbo-skill-slug

## What it does, honestly

You give the slug a build session two ways: narrate it aloud (audio), or drop a
Claude Code / Codex CLI session trace (`.jsonl`). It transcribes/parses, extracts
a structured record, and returns four artifacts: a transferable SKILL.md, a
grounded second-person recap in a fine-tuned "slug voice," a procedural SVG shell,
and a thermal receipt. The shell is born on screen as a scroll that unrolls along
its spiral arm, with a byōbu (folding-screen) battle drawn across it.

## What is genuinely strong

- **Two custom 1.5B LoRA adapters, both published and both used.** Voice
  (500 pairs, loss 0.89) and extraction (167 pairs, loss 0.81). The extraction
  LoRA replaced a prompted Qwen-7B, bringing the primary pipeline to ~2.6B active
  params (Whisper 809M + one 1.5B base serving both adapters on a single T4).
- **The groundedness eval is real and published with its raw data.** On 25
  held-out transcripts the fine-tuned 1.5B reaches semantic groundedness 0.76 vs
  the 7B's 0.72, at a third the active size, by paraphrasing rather than copying.
- **The shell is fully procedural and traceable.** Every visual element derives
  from a real session feature: dead ends are knots/fallen warriors, gotchas are
  jewels/archers, the breakthrough is the aperture/dragon, sentiment is the color
  arc. No image generation.
- **The SKILL.md is built for real uplift**, gotchas-first (symptom→cause→fix),
  "what does NOT work and why," transferable principles. Built and de-leaked
  deterministically from the structured extraction, not trusted to model prose.
- **Trace input works.** A real Claude Code or Codex session log runs through the
  same pipeline as audio. Judges can feed their own logs.
- **Graceful degradation is real and now observable.** If the extract adapter
  emits invalid JSON, the 7B fallback catches it. If the voice adapter is down, a
  deterministic extraction-based voice net prevents placeholder recaps. Every
  voice-path outcome is logged with a `[VOICE]` tag (EXTRACT_DOWN / VOICE_DOWN /
  VOICE_EMPTY / VOICE_OK / FALLBACK_VOICE_OK / NET_FROM_EXTRACTION / …) so failures
  are diagnosable, not mysterious.

## What is partial or has real caveats

- **Extraction parse reliability: 21/25 vs the 7B's 24/25.** The smaller model
  produces invalid JSON more often. In the app a brace-walking parser and field
  validators recover most of this, and the 7B fallback catches the rest, but the
  raw rate is a real cost of going small, and we report it unsoftened. (You will
  see `[VOICE] EXTRACT_DOWN` in the logs when this fires; the fallback then runs.)
- **The groundedness metric is calibrated to 5/6, not 6/6.** A hand-labeled
  calibration block (run before scoring, printed in the logs) missed one case:
  a grounded paraphrase scored just under threshold. The miss understates the
  LoRA rather than inflating it. 25 transcripts is a small sample; treat the
  LoRA-vs-7B gap as "matches or slightly exceeds," not "beats."
- **SKILL.md quality depends on extraction quality.** A good session yields
  transferable gotchas; a thin session yields thinner ones. An optional one-shot
  enrichment pass expands terse gotchas, gated by env and best-effort. We caught
  and fixed a real bug where few-shot examples in the enrichment prompt leaked
  into output (a game-theory skill got tree-coloring gotchas); the prompts are
  de-leaked and a guard now rejects any leaked phrasing.
- **The scroll-unroll animation ships ~1.7MB per shell** (14 stacked growth
  stages, so the parchment lays down ALONG the spiral arm rather than as a radial
  wipe). Browser-rendered via SMIL in a sandboxed iframe, with a "watch it unroll
  again" replay. On a slow connection the first paint takes a beat. If SMIL fails,
  the full shell still shows (nothing is hard-hidden); animation is enhancement,
  not a dependency.
- **The 3D paper curl rides the spiral tip via animateMotion.** Verified
  frame-by-frame in offline renders (cairosvg + headless Chromium). Exact
  tip-tracking in every browser is the part most likely to need a small tweak.

## What was freshly added near the deadline (use with that in mind)

- **Shared terrarium gallery.** Kept shells save to a Modal Volume; a grid shows
  all of them newest-first; each has a complete clickable permalink that re-loads
  and re-animates that shell. Freshly built and lightly tested. Honest caveats:
  each grid card is its own iframe, so past ~30-40 shells the grid gets heavy
  (lazy-loading is the fix, not yet done). Gallery saves depend on the Modal
  endpoint URLs matching the client; if they drift, a save fails and the UI says
  so rather than failing silently.
- **Battle Trace (experimental).** A temporal replay of the same session as a war
  between you (the Agent) and the Environment, fed by the SAME extraction (no
  second parser, no OTel). Framed as complementary to the shell: the shell is the
  slug's *memory* of how the battle ended (the frozen folding screen); the trace
  is the *replay* of it in time. It is Canvas-2D with simple figurative
  combatants (a samurai general, a horned adversary, fallen warriors, a dragon at
  the breakthrough). Honest about its level: these are clean vector figures, not
  game-quality sprite art. Labeled "experimental" in the UI.

## What is aspirational / not built

- **Session diff view** (compare two sessions' shells) is not built.
- **Closing the parse-rate gap** needs more training pairs and constrained
  decoding; not done.
- **Higher-fidelity Battle Trace art** (detailed sprite-grade samurai) is not
  built; the current figures are deliberately simple vector shapes (see the
  Battle Trace note above).

## Infrastructure honesty

- The Qwen-7B still exists in the codebase but ONLY as a labeled fallback when the
  Modal extract path returns nothing usable (including the invalid-JSON case
  above). The primary path is the 1.5B dual adapter. The README architecture table
  reflects this.
- The dual-adapter server can hold one warm container for demo reliability
  (~$0.60/hr). It should be stopped after judging (`modal app stop
  slug-dual-serve`).

## How to verify our claims

- Groundedness: `modal run semantic_eval.py` reproduces the table; raw
  generations are published in the eval dataset so anyone can re-score.
- Trace input: drop the sample `.jsonl` (one click) or your own session log.
- Models: both LoRA adapters are public on the Hub.
- Shell traceability: change a session's sentiment or dead-end count and watch
  the shell's colors and knots change accordingly.
- Failure handling: the Space logs tag every voice-path outcome with `[VOICE]`.

## The one-sentence honest summary

A small, slow, genuinely ~2.6B pipeline that turns a coding session into a
transferable skill, a grounded recap, and a procedural shell, measured honestly
(including where the small model costs us), with graceful, observable degradation,
and every model and the eval data published for anyone to check.