Upload folder using huggingface_hub

.gitignore

@@ -1,5 +1,6 @@
 # Local index is generated; large transcripts live on HF dataset repo
 data/*
+!data/public_traces.jsonl
 !data/transcripts/
 !data/transcripts/**
 data/transcripts/office_generated.txt
@@ -45,6 +46,7 @@ coverage.xml
 # IDE and editors
 .idea/
 .vscode/
+docs/private-demo-pitch.md
 *.swp
 *.swo
 *~

README.md

@@ -11,288 +11,63 @@ pinned: false
 license: apache-2.0
 short_description: Paste awkward. Get one sharp wit line + coach drills.
 tags:
+  - build-small-hackathon
+  - track:wood
+  - thousand-token-wood
   - comedy
   - rag
   - case-based-reasoning
   - qwen
-  - thousand-token-wood
   - achievement:offbrand
-  - achievement:best-agent
-  - achievement:best-demo
-  - achievement:bonus-quest-champion
+  - achievement:sharing
+  - achievement:fieldnotes
   - sponsor:openai
 ---
 
 # 🎭 WitGym
 
-> *A comedy coaching engine grounded in human precedent — not vibes.*
+**One sharp line, grounded in human precedent. Then drills to sharpen it.**
 
-Paste an awkward real-life situation (or tap a starter). WitGym dissects the **structural comedy of the moment**, retrieves analogous scenes from The Office, drafts 2–3 persona candidates (cynic, conviction, absurdist — twist-gated), ranks them, and returns one compressed sharp line. After a reply, use coach drills to sharpen it, try a different angle, or explain why it lands.
+WitGym is a comedy coaching engine for awkward real-life moments. It extracts the **comedy structure**, retrieves analogous precedent from *The Office*, drafts constrained persona candidates, runs a **tournament ranker**, and returns one crisp line with optional coaching.
 
-**[→ Try it live on Hugging Face Spaces](https://huggingface.co/spaces/build-small-hackathon/WitGym)**
-
----
+**Live Space**: [build-small-hackathon/WitGym](https://huggingface.co/spaces/build-small-hackathon/WitGym)
 
-## How to use
-
-1. **Start** — click *Start Training* on the landing screen.
-2. **Describe the moment** — paste your own awkward situation, or tap a starter chip in the sidebar (*Status*, *Social*, *Delusion*, etc.).
-3. **Get a line** — WitGym returns one sharp wit line grounded in Office precedent (not an in-character Michael/Dwight impersonation).
-4. **Coach drills** — after a reply, use *sharpen it*, *different angle*, or *explain the joke* to iterate on the same situation.
-5. **Character panel** — the Office cast cards on the landing page are reference bios only; you do **not** pick a character to role-play.
-
----
-
-## Why comedy is hard for AI
-
-Every comedy coaching app just asks a model to "be funny." That's like asking someone to "be good at chess" without ever showing them a game.
-
-Wit has **structure**. It emerges from the gap between what's socially expected and what gets said. The Office didn't write jokes — it wrote *situations*, then let characters navigate them in character-consistent ways. That's the thing to learn from.
-
-WitGym treats comedy the way researchers do: as a system of **social violations, status games, and tension types** — then grounds responses in scenes that worked for the same reason.
-
----
+### Why I built this
+Comedy has always been a personal interest — not just watching it, but understanding how a line lands. I wanted a **humor coach** for real awkward moments: paste what happened, get one sharp line, then drill on it. *The Office* felt like the right precedent library (I'm a longtime fan) — not to impersonate characters, but to learn from situations that already work. WitGym is my attempt to make that coach real under the ≤32B constraint.
 
-## Architecture
+### 30‑second demo
+- Paste: “My boss says he trusts me, but he rewrites every message I send.”
+- Watch the phases: extract → retrieve → draft → rank → polish (streaming)
+- Tap drills: **sharpen it**, **different angle**, **explain the joke**
 
-```mermaid
-flowchart TD
-    A[User: paste awkward situation] --> B{Small talk?}
-    B -- yes --> C[Identity reply — no pipeline]
-    B -- no --> D
-
-    D[Pass 1 — Neurology of Comedy\nQwen3.5-27B extracts 10 structural fields]
-    D --> E[Archetype + Tension + Distance\nComedyMetadata schema]
-    E --> F[BGE-small embedder\n384-dim semantic vector]
-    F --> G[Cosine retrieval + rerank\n4021 Office scene index]
-    G --> H[Top-2 precedent scenes\nwith why_it_works annotations]
-    H --> I[Pass 2 — Persona Generation\n2–3 candidates: Cynic · Conviction · Absurdist]
-    I --> J[Pass 3 — Tournament Ranking\nranked by structural fit to the metadata]
-    J --> K[Pass 4 — Compression\nreduce to one sharp line]
-    K --> L[Coaching\noptional: what made this work?]
-    L --> M[Response + full debug trace]
-
-    style D fill:#1a1a2e,color:#ffd700,stroke:#ffd700
-    style G fill:#16213e,color:#4fc3f7,stroke:#4fc3f7
-    style I fill:#0f3460,color:#f8f8f8,stroke:#e94560
-    style M fill:#1a1a2e,color:#b0f4b0,stroke:#4caf50
-```
-
----
-
-## The Comedy Science
-
-WitGym models comedy as three interacting structural properties:
-
-| Property | Enum | What it captures |
-|---|---|---|
-| **Archetype** | `ComedyArchetype` | *Why* this moment is funny in principle — e.g. `STATUS_ASSERTION`, `SELF_DELUSION_EXPOSED`, `POWER_INVERSION` |
-| **Tension type** | `TensionType` | *What* is at stake — e.g. `SOCIAL_EMBARRASSMENT`, `STATUS_THREAT`, `IDENTITY_EXPOSURE` |
-| **Violation distance** | `ViolationDistance` | *How far* to push — `mild`, `moderate`, `sharp` |
-
-These aren't vibes. They're used to:
-1. Select the right **precedent scenes** from the index (cosine similarity on archetype + semantic embedding)
-2. Constrain **persona generation** (each persona must violate in a structurally consistent way)
-3. **Rank** candidates (the winner resolves the tension most cleanly)
-
-The pipeline produces a `ComedyMetadata` object with 10 fields — including `connector` (the double-meaning word that makes a line land), `subtext` (what's actually being communicated), and `twist_potential` (comedy richness score 1–10 used to gate the full pipeline vs. quick response).
-
----
+### What makes it different
+- **CBR‑RAG on comedy mechanics**: retrieve by archetype, tension, and violation distance — not topic keywords
+- **Behavioral observation as the generative seed**: name the human move, then write from that
+- **Tournament ranking for landing**: truth precision, final-clause quality, domain anchoring
+- **Inspectable traces**: progressive disclosure in the UI; public JSONL export for Sharing is Caring
 
-## The Retrieval System
+### How it works
+1. **Extract** — `Qwen/Qwen3.5-27B` → `ComedyMetadata` (12 fields)
+2. **Retrieve** — `BAAI/bge-small-en-v1.5` cosine pool + optional `cross-encoder/ettin-reranker-32m-v1`
+3. **Generate** — 2–3 twist-gated persona candidates
+4. **Rank** — fixed rubric selects the winner
+5. **Compress** — optional polish to one sharp line
 
-4,021 indexed scenes from The Office, each annotated with:
-- `archetype`, `tension_type`, `violation_distance` — structural labels
-- `why_it_works` — a one-sentence explanation of the comedy mechanism
-- `setup` + `response` — the actual scene
-
-At query time, the user's situation is embedded with **BGE-small** (33M params) and retrieved against the index via cosine similarity. The retrieval finds scenes with the **same comedy structure** — not the same topic.
-
-```mermaid
-flowchart LR
-    A[User situation] --> B[BGE-small encoder\n33M params]
-    B --> C[384-dim vector]
-    C --> D[NumPy cosine index\n4021 scenes × 384 dims]
-    D --> E[Top-2 scenes\nthen cross-encoder rerank]
-    E --> F[Injected as few-shot\nexamples into Pass 2 prompt]
-    style B fill:#1a3a5c,color:#4fc3f7
-    style D fill:#1a3a5c,color:#4fc3f7
-```
-
----
-
-## Pipeline flow
-
-```mermaid
-sequenceDiagram
-    participant U as User
-    participant A as app.py
-    participant E as WitGymEngine
-    participant L as LLM (Qwen3.5-27B)
-    participant I as BGE Index
-
-    U->>A: Paste situation
-    A->>E: respond(user_input)
-
-    Note over E,L: Pass 1 — Metadata extraction
-    E->>L: Extract ComedyMetadata (10 fields)
-    L-->>E: archetype + tension + distance + subtext...
-
-    Note over E,I: RAG retrieval
-    E->>I: embed(surface) → cosine search
-    I-->>E: 2 precedent scenes
-
-    Note over E,L: Pass 2 — Candidate generation
-    E->>L: Generate 2–3 personas (cynic, conviction, absurdist — twist-gated)
-    L-->>E: 3 candidate responses (streaming)
-
-    Note over E,L: Pass 3 — Tournament ranking
-    E->>L: Rank candidates by structural fit
-    L-->>E: winner + explanation
-
-    Note over E,L: Pass 4 — Compression
-    E->>L: Compress winner to one sharp line
-    L-->>E: final response
-
-    E-->>A: PipelineEvent stream
-    A-->>U: Streaming response + debug trace
-```
-
----
-
-## UI: Progressive Disclosure
-
-The practice screen is situation-first: you paste or pick a starter, then receive a streaming coach reply. The landing **coaching panel** shows Office character bios (tap for a popup) — reference flavor, not a character picker.
-
-The trace uses **progressive disclosure** — the reply stays front-and-center, while the expandable rail reveals the underlying structured pipeline as JSON:
-
-- **Trace** — expandable JSON payload with metadata, retrieved scenes, candidates, and selected output
-- **Chips / capsules** — used only where the product needs human-facing explanation, not as a substitute for the execution trace
-
-New elements animate in with a shimmer sweep + border glow system that stops on first interaction. This mirrors how Notion and Apple iOS handle progressive discovery — purposeful discoverability signaling, not decoration.
-
----
+### Evidence / badges
+- **Sharing is Caring** (`achievement:sharing`): [public pipeline traces](data/public_traces.jsonl) — sanitized JSONL (metadata, scene IDs, candidate stats, execution log; no Office dialogue text). Regenerate: `uv run python scripts/export_public_traces.py`
+- **Field Notes** (`achievement:fieldnotes`): [docs/field-notes.md](docs/field-notes.md)
+- **Off‑Brand UI** (`achievement:offbrand`): custom Gradio UI + streaming trace disclosure
 
-## Tech Stack
+> **Validator**: add demo video + social post links here before submission. [Validate README](https://build-small-hackathon-field-guide.hf.space/submit)
 
-| Layer | Choice | Why |
-|---|---|---|
-| **LLM** | Qwen3.5-27B via HF Inference Providers | ≤32B constraint; best instruction-following at this size |
-| **Embedder** | BGE-small (33M params) | Fast, accurate, runs on CPU in < 50ms |
-| **Index** | NumPy cosine retrieval + optional 32M cross-encoder rerank, 4021 scenes × 384 dims | No server needed; loaded at startup from Hub dataset |
-| **UI** | Gradio 6.x on HF Spaces | Streaming SSE, custom CSS theming |
-| **Validation** | Pydantic v2 | Schema-enforced extraction; fallback on parse failure |
-| **Retry** | Exponential backoff on all LLM calls | Handles upstream provider flakiness gracefully |
-
----
-
-## Build Small compliance
-
-- **Model size**: Qwen3.5-27B ≤ 32B ✓
-- **Embedder**: BGE-small 33M — runs on CPU, no GPU needed for retrieval ✓
-- **Deployed on HF Spaces**: Gradio app, streams via SSE ✓
-- **Open source**: Apache 2.0 licensed ✓
-- **Use case**: whimsical entertainment / comedy coaching grounded in structured precedent ✓
-- **Interaction design**: custom Gradio interface with progressive disclosure and streaming feedback ✓
-- **Agentic flow**: route -> extract -> retrieve -> generate -> rank -> compress ✓
-- No fine-tuning required — all comedy structure is in the retrieval index and prompts
-
----
-
-## How to cite WitGym
-
-If you use or build on this project, please cite:
-
-```bibtex
-@software{babbar2026witgym,
-  author = {Babbar, Akshay},
-  title = {WitGym: CBR-RAG Comedy Coaching Engine},
-  year = {2026},
-  url = {https://github.com/akshay-babbar/witgym},
-  note = {Hugging Face Build Small Hackathon 2026 submission}
-}
-```
-
-See also [`CITATION.cff`](CITATION.cff) for machine-readable metadata.
-
----
-
-## What makes this different
-
-| Approach | Problem |
-|---|---|
-| "Respond wittily to: [situation]" | No structural understanding; generic, contextless outputs |
-| RAG on jokes | Jokes don't transfer — the *situation structure* transfers |
-| **WitGym** | Extracts comedy structure → retrieves same-structure precedents → generates constrained by that structure |
-
-The insight: The Office didn't write great jokes. It wrote great **situations**, then populated them with characters who respond in structurally consistent ways. WitGym learns from the situations, not the punchlines.
-
----
-
-## Run locally
+### Run locally
 
 ```bash
-# Install
-uv sync  # or: pip install -e .
-
-# Build index (The Office transcripts)
+uv sync
 witgym-index
-
-# Run with HF Inference API (recommended — no local weights needed)
-export HF_TOKEN=hf_...
 export LLM_BACKEND=hf_api
-python app.py
-
-# Run with local weights (MPS/CUDA)
-python app.py
-```
-
----
-
-## HF Spaces configuration
-
-Set these in Space settings — **not** in GitHub. CI only syncs code; runtime auth is separate.
-
-| Secret | Value |
-|---|---|
-| `HF_TOKEN` | Write token with read access to the data dataset |
-| `LLM_BACKEND` | `hf_api` |
-| `WITGYM_DATA_REPO` | `build-small-hackathon/witgym-data` (default) |
-
-Optional: `HF_INFERENCE_PROVIDER` (defaults to `together`), `WITGYM_INDEX_PATH`.
-
-Do **not** set `WITGYM_SKIP_HUB` on the Space.
-
----
-
-## Large data on HF Hub
-
-Files over 1 MB (`office_generated.txt`, `index.npz`) live in a private org dataset, not in git. At startup the app fetches `index.npz` from the Hub; if auth fails it falls back to rebuilding from bundled transcripts.
-
-```bash
-hf upload build-small-hackathon/witgym-data \
-  data/index.npz index.npz --repo-type dataset
-
-hf upload build-small-hackathon/witgym-data \
-  data/transcripts/office_generated.txt office_generated.txt \
-  --repo-type dataset --private
+export HF_TOKEN=hf_...
+uv run python app.py
 ```
 
-When transcripts change: re-run `witgym-index`, then re-upload `index.npz`. Offline dev: set `WITGYM_SKIP_HUB=1`.
-
----
-
-## CI/CD
-
-Pushes to `main` sync code to the Space via [`.github/workflows/sync-to-hub.yml`](.github/workflows/sync-to-hub.yml).
-
-| Where | Secret | Purpose |
-|---|---|---|
-| GitHub repo secrets | `HF_TOKEN` | CI push to Space only |
-| Space secrets | `HF_TOKEN`, `WITGYM_DATA_REPO`, `LLM_BACKEND` | Runtime Hub API + dataset + inference |
-
-These are **two different** `HF_TOKEN` placements. Configuring GitHub does not configure the Space runtime.
-
----
-
-Built for the [Hugging Face Build Small Hackathon 2026](https://huggingface.co/build-small-hackathon).
+Built for the [Build Small Hackathon 2026](https://huggingface.co/build-small-hackathon) — Thousand Token Wood.

data/public_traces.jsonl

@@ -0,0 +1,3 @@
+{"kind": "single_turn", "input": "I just got promoted to manager and I have no idea what I'm doing.", "route": "quick_wit", "model_id": "Qwen/Qwen3.5-27B", "llm_backend": "hf_api", "latency_s": 24.0, "metadata": {"surface": "I just got promoted to manager and I have no idea what I'm doing.", "subtext": "The speaker is overwhelmed by impostor syndrome and fears their new authority will expose their incompetence.", "behavioral_observation": null, "archetype": "status_assertion", "archetype_confidence": 9, "tension_type": "identity_expose", "power_dynamic": "The speaker holds formal power as a manager but lacks the actual competence to wield it effectively.", "speaker_strategy": "vulnerable honesty", "obvious_response": "Don't worry, everyone feels that way at first.", "violation_distance": "mild", "twist_potential": 7, "connector": "manager"}, "retrieved_scenes": [{"scene_id": "08ca192f647e", "show": "The Office", "character": "Michael Scott", "archetype": "status_assertion", "tension_type": "identity_expose", "violation_distance": "sharp"}, {"scene_id": "17182d6cf3eb", "show": "The Office", "character": "Robert", "archetype": "status_assertion", "tension_type": "status_threat", "violation_distance": "sharp"}], "candidates": [{"persona": "cynic", "word_count": 30}, {"persona": "conviction", "word_count": 18}, {"persona": "absurdist", "word_count": 23}], "winning_persona": "cynic", "selected": "Congratulations on the promotion; you now have authority to tell everyone what you're doing while they pretend not to notice you're making it up as you go.", "logs": [{"step": "metadata", "status": "ok", "detail": "twist=7 archetype=status_assertion"}, {"step": "retrieval", "status": "ok", "detail": "Michael Scott:status_assertion, Robert:status_assertion"}, {"step": "candidate_generation", "status": "ok", "detail": "cynic:30w, conviction:18w, absurdist:23w"}, {"step": "ranking", "status": "ok", "detail": "cynic"}, {"step": "compression", "status": "ok", "detail": "selected line finalized"}]}
+{"kind": "single_turn", "input": "My coworker keeps stealing my lunch from the fridge.", "route": "quick_wit", "model_id": "Qwen/Qwen3.5-27B", "llm_backend": "hf_api", "latency_s": 11.47, "metadata": {"surface": "My coworker keeps stealing my lunch from the fridge.", "subtext": "I feel violated and powerless against a recurring theft that threatens my basic needs and workplace safety.", "behavioral_observation": null, "archetype": "anxiety_escalation", "archetype_confidence": 8, "tension_type": "status_threat", "power_dynamic": "The coworker holds power through repeated successful transgression while the speaker remains a passive victim unable to stop it.", "speaker_strategy": "seeking validation for victimhood", "obvious_response": "That is really annoying, you should probably report it to HR or put your food in a locked container.", "violation_distance": "moderate", "twist_potential": 6, "connector": null}, "retrieved_scenes": [{"scene_id": "0039dc5d8663", "show": "The Office", "character": "Kevin", "archetype": "anxiety_escalation", "tension_type": "status_threat", "violation_distance": "moderate"}, {"scene_id": "29f42f59ca32", "show": "The Office", "character": "Michael", "archetype": "anxiety_escalation", "tension_type": "status_threat", "violation_distance": "moderate"}], "candidates": [{"persona": "cynic", "word_count": 18}, {"persona": "absurdist", "word_count": 25}], "winning_persona": "absurdist", "selected": "My lunchbox is a vending machine where I'm the currency, and he just inserted a quarter made of my dignity.", "logs": [{"step": "metadata", "status": "ok", "detail": "twist=6 archetype=anxiety_escalation"}, {"step": "retrieval", "status": "ok", "detail": "Kevin:anxiety_escalation, Michael:anxiety_escalation"}, {"step": "candidate_generation", "status": "ok", "detail": "cynic:18w, absurdist:25w"}, {"step": "ranking", "status": "ok", "detail": "absurdist"}, {"step": "compression", "status": "ok", "detail": "selected line finalized"}]}
+{"kind": "single_turn", "input": "My boss says he trusts me, but he rewrites every message I send.", "route": "quick_wit", "model_id": "Qwen/Qwen3.5-27B", "llm_backend": "hf_api", "latency_s": 16.01, "metadata": {"surface": "My boss says he trusts me, but he rewrites every message I send.", "subtext": "The speaker feels their competence is undermined and that the boss's claim of trust is a hollow lie contradicted by controlling behavior.", "behavioral_observation": null, "archetype": "power_inversion", "archetype_confidence": 9, "tension_type": "status_threat", "power_dynamic": "The boss holds formal authority but reveals insecurity through micromanagement, while the employee holds the moral high ground of being the actual competent worker.", "speaker_strategy": "exposing hypocrisy", "obvious_response": "That sounds frustrating.", "violation_distance": "moderate", "twist_potential": 8, "connector": null}, "retrieved_scenes": [{"scene_id": "44eea9bd8663", "show": "The Office", "character": "Toby Flenderson", "archetype": "power_inversion", "tension_type": "status_threat", "violation_distance": "sharp"}, {"scene_id": "d7bfe3ce399e", "show": "The Office", "character": "Michael", "archetype": "power_inversion", "tension_type": "status_threat", "violation_distance": "sharp"}], "candidates": [{"persona": "cynic", "word_count": 20}, {"persona": "conviction", "word_count": 19}, {"persona": "absurdist", "word_count": 28}], "winning_persona": "absurdist", "selected": "If he trusts me so much, let's just stop hitting \"send\" entirely and hand him the keyboard while I watch him type my own thoughts back to himself.", "logs": [{"step": "metadata", "status": "ok", "detail": "twist=8 archetype=power_inversion"}, {"step": "retrieval", "status": "ok", "detail": "Toby Flenderson:power_inversion, Michael:power_inversion"}, {"step": "candidate_generation", "status": "ok", "detail": "cynic:20w, conviction:19w, absurdist:28w"}, {"step": "ranking", "status": "ok", "detail": "absurdist"}, {"step": "compression", "status": "ok", "detail": "selected line finalized"}]}

docs/field-notes.md

@@ -0,0 +1,31 @@
+# WitGym — Field Notes (Build Small 2026)
+
+## What I built
+WitGym is a comedy coaching engine for real-life awkward moments. You paste a situation, it produces **one sharp line**, then lets you iterate with drills (sharpen / different angle / explain).
+
+The core bet: **comedy transfers by structure, not by topic**. Instead of “RAG on jokes”, WitGym does **CBR-RAG on comedy mechanics** and uses precedent from *The Office* to ground the response.
+
+## The small-model constraint (≤32B) changed the design
+Under the Build Small constraint, the goal wasn’t “generate funnier text by scaling”, it was “get reliable *wit* by adding structure”:
+
+- **Pass 1 (extraction)**: extract a compact schema (`ComedyMetadata`) describing the moment: archetype, tension, violation distance, subtext, behavioral observation, etc.\n
+- **Retrieval (CBR-RAG)**: retrieve *structurally similar* precedent scenes from a prebuilt index.\n
+- **Pass 2 (generation)**: draft 2–3 persona candidates with strict constraints.\n
+- **Pass 3 (ranking)**: pick a winner with an explicit judging rubric (truth precision + strong ending + domain anchoring).\n
+- **Pass 4 (compression)**: optionally tighten the winner to one crisp line.\n
+
+## What was unexpectedly important
+- **Behavioral observation > feelings**: naming the *move* (“renamed procrastination as ‘keeping options open’”) is a better generative seed than therapy-language subtext.\n
+- **Ranking beats clever prompting**: the biggest quality jumps came from forcing a tournament-style selection rubric, especially “truth precision” and “final clause” quality.\n
+- **Progressive disclosure UX matters**: streaming phase updates and an expandable trace makes judges trust the system (it’s not “vibes”; you can see what it did).\n
+
+## Models used
+- **LLM**: `Qwen/Qwen3.5-27B` (≤32B) via Hugging Face Inference Providers (recommended runtime path).\n
+- **Embedder**: `BAAI/bge-small-en-v1.5` (33M) for retrieval.\n
+- **Reranker (optional)**: `cross-encoder/ettin-reranker-32m-v1` (CPU) for pool reranking.\n
+
+## What I’d improve next (post-hackathon)
+- Make “coach mode” differentiate *response style*, not just “add an explanation panel”.\n
+- Add a public, privacy-safe trace export that covers **both** evaluation runs and real usage patterns (with de-identification).\n
+- Tighten the “small talk” and “low twist” path so it’s still delightful without running the full pipeline.\n
+

docs/linkedin-post.md

@@ -0,0 +1,12 @@
+# LinkedIn post (copy-paste)
+
+Happy to share a project I've been building for the Hugging Face Build Small Hackathon.
+
+Comedy has always been a personal interest for me — not just watching it, but understanding how a line actually lands. I wanted something closer to a **humor coach** for awkward real-life moments: you paste what happened, get one sharp line back, then iterate on it.
+
+That's **WitGym**. It extracts the comedy structure of the moment, finds analogous precedent from *The Office* (a show I've loved for years — it felt like the right sandbox), drafts a few constrained candidates, ranks them, and returns one line you can use. Then you can sharpen it, try a different angle, or unpack why it works.
+
+More detail in the README and live demo on Hugging Face Spaces:
+https://huggingface.co/spaces/build-small-hackathon/WitGym
+
+Built for Build Small 2026 — Thousand Token Wood track.

scripts/eval_behavioral_ab.py

@@ -0,0 +1,243 @@
+"""A/B eval: baseline (main/deployed) vs behavioral_observation branch changes.
+
+Uses LLM_BACKEND=hf_api only. Does not touch corpus/index.
+"""
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+from dotenv import load_dotenv
+
+load_dotenv()
+os.environ.setdefault("LLM_BACKEND", "hf_api")
+
+ROOT = Path(__file__).resolve().parents[1]
+CHANGED = [
+    "witgym/schemas.py",
+    "witgym/extractor.py",
+    "witgym/prompts.py",
+    "witgym/generator.py",
+]
+BACKUP_DIR = ROOT / "data" / ".eval_behavioral_backup"
+
+SINGLE_TURN = [
+    "I just got promoted to manager and I have no idea what I'm doing.",
+    "My coworker keeps stealing my lunch from the fridge.",
+    "I've been cc'd on an email chain I definitely should not be reading.",
+    "I'm pretending to understand cryptocurrency at dinner parties.",
+    "My therapist fell asleep during our session.",
+    "I still haven't replied to that email from three weeks ago.",
+    "My boss says he trusts me, but he rewrites every message I send.",
+    "I told everyone the meeting would be quick and it is now ruining lives.",
+    "I keep calling it networking when really I'm just begging professionally.",
+    'I said "circle back" because I ran out of courage.',
+    "I keep acting like I'm choosing not to date when the market has actually made that decision for me.",
+    "I described panic as being detail-oriented and now people believe me.",
+]
+
+COACHING_FLOWS = [
+    (
+        "Help me come up with something funny to say to my micromanaging boss.",
+        "He keeps interrupting me in meetings and then repeating my point like he invented it.",
+    ),
+    (
+        "Coach me on a funny response for a social situation.",
+        'A friend asked if I was free this weekend and I panicked and said I was "booked" when I meant emotionally.',
+    ),
+    (
+        "Help me respond better in awkward situations.",
+        "I told my date I love routines and then described anxiety like it was a hobby.",
+    ),
+]
+
+
+def _backup_modified() -> None:
+    if BACKUP_DIR.exists():
+        shutil.rmtree(BACKUP_DIR)
+    BACKUP_DIR.mkdir(parents=True)
+    for rel in CHANGED:
+        src = ROOT / rel
+        if src.exists():
+            shutil.copy2(src, BACKUP_DIR / Path(rel).name)
+
+
+def _restore_modified() -> None:
+    for rel in CHANGED:
+        name = Path(rel).name
+        src = BACKUP_DIR / name
+        if src.exists():
+            shutil.copy2(src, ROOT / rel)
+
+
+def _reset_to_baseline() -> None:
+    subprocess.run(
+        ["git", "checkout", "--", *CHANGED],
+        cwd=ROOT,
+        check=True,
+    )
+
+
+def _run_one(engine, user_input: str, *, coaching_turn2: str | None = None) -> dict:
+    from witgym.engine import WitGymEngine
+
+    eng = WitGymEngine(index_path="data/index.npz", resources=engine)
+    t0 = time.time()
+    if coaching_turn2 is None:
+        r = eng.respond(user_input)
+        meta = r.metadata
+        return {
+            "input": user_input,
+            "kind": "single",
+            "selected": r.selected,
+            "word_count": len(r.selected.split()),
+            "behavioral_observation": getattr(meta, "behavioral_observation", None),
+            "subtext": meta.subtext,
+            "archetype": meta.archetype.value,
+            "latency_s": round(time.time() - t0, 1),
+            "candidates": [c.text for c in r.candidates],
+        }
+
+    turn1 = eng.respond(user_input)
+    eng2 = WitGymEngine(
+        index_path="data/index.npz",
+        resources=engine,
+        conversation=eng.conversation,
+    )
+    t1 = time.time()
+    r2 = eng2.respond(coaching_turn2)
+    meta = r2.metadata
+    return {
+        "input": f"{user_input} → {coaching_turn2}",
+        "kind": "coaching_turn2",
+        "coaching_question": turn1.coaching_question or turn1.selected,
+        "selected": r2.selected,
+        "word_count": len(r2.selected.split()),
+        "behavioral_observation": getattr(meta, "behavioral_observation", None),
+        "subtext": meta.subtext,
+        "archetype": meta.archetype.value,
+        "latency_s": round(time.time() - t1, 1),
+        "candidates": [c.text for c in r2.candidates],
+    }
+
+
+def _run_variant_subprocess(tag: str) -> None:
+    """Run one variant in a fresh Python process so code changes take effect."""
+    env = os.environ.copy()
+    env.setdefault("LLM_BACKEND", "hf_api")
+    cmd = [
+        sys.executable,
+        str(ROOT / "scripts" / "eval_behavioral_ab.py"),
+        "--worker",
+        tag,
+    ]
+    subprocess.run(cmd, cwd=ROOT, env=env, check=True)
+
+
+def run_variant(tag: str) -> list[dict]:
+    from witgym.engine import SharedResources
+
+    if not Path("data/index.npz").exists():
+        print("[ERROR] data/index.npz missing")
+        sys.exit(1)
+
+    shared = SharedResources(index_path="data/index.npz")
+    results: list[dict] = []
+    total = len(SINGLE_TURN) + len(COACHING_FLOWS)
+    n = 0
+
+    for prompt in SINGLE_TURN:
+        n += 1
+        print(f"\n[{tag}] [{n}/{total}] {prompt[:70]}...")
+        row = _run_one(shared, prompt)
+        print(f"  → {row['selected']}")
+        if row.get("behavioral_observation"):
+            print(f"  obs: {row['behavioral_observation'][:100]}")
+        results.append(row)
+
+    for turn1, turn2 in COACHING_FLOWS:
+        n += 1
+        print(f"\n[{tag}] [{n}/{total}] coaching: {turn1[:50]}...")
+        row = _run_one(shared, turn1, coaching_turn2=turn2)
+        print(f"  → {row['selected']}")
+        if row.get("behavioral_observation"):
+            print(f"  obs: {row['behavioral_observation'][:100]}")
+        results.append(row)
+
+    out = ROOT / "data" / f"eval_behavioral_{tag}.json"
+    payload = {"tag": tag, "results": results}
+    out.write_text(json.dumps(payload, indent=2), encoding="utf-8")
+    print(f"\n✓ Saved {out}")
+    return results
+
+
+def compare() -> None:
+    b_path = ROOT / "data" / "eval_behavioral_baseline.json"
+    m_path = ROOT / "data" / "eval_behavioral_modified.json"
+    if not b_path.exists() or not m_path.exists():
+        print("[ERROR] Run full eval first")
+        sys.exit(1)
+
+    baseline = {r["input"]: r for r in json.loads(b_path.read_text())["results"]}
+    modified = {r["input"]: r for r in json.loads(m_path.read_text())["results"]}
+
+    print("\n" + "=" * 100)
+    print("BEHAVIORAL_OBSERVATION A/B — baseline (main/HF) vs modified")
+    print("=" * 100)
+
+    rows = []
+    for key in baseline:
+        b, m = baseline[key], modified.get(key)
+        if not m:
+            continue
+        print(f"\n📝 {key[:90]}")
+        print(f"   BASE [{b['word_count']}w]: {b['selected']}")
+        print(f"   MOD  [{m['word_count']}w]: {m['selected']}")
+        if m.get("behavioral_observation") and not b.get("behavioral_observation"):
+            print(f"   OBS  (modified only): {m['behavioral_observation']}")
+        elif b.get("behavioral_observation") or m.get("behavioral_observation"):
+            print(f"   OBS baseline: {b.get('behavioral_observation')}")
+            print(f"   OBS modified: {m.get('behavioral_observation')}")
+        rows.append({"input": key, "baseline": b["selected"], "modified": m["selected"]})
+
+    sheet = ROOT / "data" / "eval_behavioral_sheet.json"
+    sheet.write_text(json.dumps(rows, indent=2), encoding="utf-8")
+    print(f"\n✓ Comparison sheet → {sheet}")
+
+
+def main() -> None:
+    import argparse
+
+    p = argparse.ArgumentParser()
+    p.add_argument("--compare-only", action="store_true")
+    p.add_argument("--worker", choices=["baseline", "modified"])
+    args = p.parse_args()
+
+    if args.worker:
+        run_variant(args.worker)
+        return
+
+    if args.compare_only:
+        compare()
+        return
+
+    _backup_modified()
+    try:
+        print("=== BASELINE (main / deployed HF) ===")
+        _reset_to_baseline()
+        _run_variant_subprocess("baseline")
+        print("\n=== MODIFIED (behavioral_observation) ===")
+        _restore_modified()
+        _run_variant_subprocess("modified")
+        compare()
+    finally:
+        _restore_modified()
+
+
+if __name__ == "__main__":
+    main()

scripts/export_public_traces.py

@@ -0,0 +1,135 @@
+"""Export a small, public-safe set of WitGym traces (Sharing is Caring).
+
+This writes deterministic-ish JSONL traces for a fixed set of canonical prompts.
+It intentionally omits any raw Office transcript text from retrieved scenes.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import time
+import hashlib
+import sys
+from pathlib import Path
+
+
+CANONICAL_SINGLE_TURN = [
+    "I just got promoted to manager and I have no idea what I'm doing.",
+    "My coworker keeps stealing my lunch from the fridge.",
+    "My boss says he trusts me, but he rewrites every message I send.",
+]
+
+
+def _scene_id(setup: str, response: str) -> str:
+    h = hashlib.sha256((setup + "\n" + response).encode("utf-8")).hexdigest()
+    return h[:12]
+
+
+def _safe_scene(scene) -> dict:
+    # Do not export setup/response verbatim; this keeps traces inspectable without
+    # publishing transcript text.
+    return {
+        "scene_id": _scene_id(scene.setup, scene.response),
+        "show": scene.show,
+        "character": scene.character,
+        "archetype": scene.archetype.value,
+        "tension_type": scene.tension_type.value,
+        "violation_distance": scene.violation_distance.value,
+    }
+
+
+def _pipeline_logs(result) -> list[dict]:
+    meta = result.metadata
+    scenes = result.retrieved_scenes
+    candidates = result.candidates
+    return [
+        {
+            "step": "metadata",
+            "status": "ok",
+            "detail": f"twist={meta.twist_potential} archetype={meta.archetype.value}",
+        },
+        {
+            "step": "retrieval",
+            "status": "ok",
+            "detail": ", ".join(f"{s.character}:{s.archetype.value}" for s in scenes) or "no precedent scenes",
+        },
+        {
+            "step": "candidate_generation",
+            "status": "ok",
+            "detail": ", ".join(f"{c.persona}:{len(c.text.split())}w" for c in candidates) or "no candidates",
+        },
+        {"step": "ranking", "status": "ok", "detail": result.winning_persona or "none"},
+        {"step": "compression", "status": "ok", "detail": "selected line finalized"},
+    ]
+
+
+def _run_single(engine, user_input: str) -> dict:
+    t0 = time.time()
+    result = engine.respond(user_input)
+    dt = time.time() - t0
+
+    meta = result.metadata
+    return {
+        "kind": "single_turn",
+        "input": user_input,
+        "route": result.route,
+        "model_id": os.getenv("LLM_MODEL_ID", "Qwen/Qwen3.5-27B"),
+        "llm_backend": os.getenv("LLM_BACKEND", "hf_api"),
+        "latency_s": round(dt, 2),
+        "metadata": {
+            "surface": meta.surface,
+            "subtext": meta.subtext,
+            "behavioral_observation": getattr(meta, "behavioral_observation", None),
+            "archetype": meta.archetype.value,
+            "archetype_confidence": meta.archetype_confidence,
+            "tension_type": meta.tension_type.value,
+            "power_dynamic": meta.power_dynamic,
+            "speaker_strategy": meta.speaker_strategy,
+            "obvious_response": meta.obvious_response,
+            "violation_distance": meta.violation_distance.value,
+            "twist_potential": meta.twist_potential,
+            "connector": meta.connector,
+        },
+        "retrieved_scenes": [_safe_scene(s) for s in result.retrieved_scenes],
+        "candidates": [
+            {"persona": c.persona, "word_count": len(c.text.split())}
+            for c in result.candidates
+        ],
+        "winning_persona": result.winning_persona,
+        "selected": result.selected,
+        "logs": _pipeline_logs(result),
+    }
+
+
+def main() -> int:
+    # Prefer HF API for reproducible “no local weights” runs.
+    os.environ.setdefault("LLM_BACKEND", "hf_api")
+
+    # Allow running without installing the package (pip/uv editable install).
+    repo_root = Path(__file__).resolve().parents[1]
+    if str(repo_root) not in sys.path:
+        sys.path.insert(0, str(repo_root))
+
+    from witgym.engine import WitGymEngine
+
+    out_path = Path("data/public_traces.jsonl")
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+
+    engine = WitGymEngine(index_path="data/index.npz")
+
+    rows: list[dict] = []
+    for prompt in CANONICAL_SINGLE_TURN:
+        rows.append(_run_single(engine, prompt))
+
+    with out_path.open("w", encoding="utf-8") as f:
+        for row in rows:
+            f.write(json.dumps(row, ensure_ascii=False) + "\n")
+
+    print(f"✓ Wrote {len(rows)} trace rows → {out_path}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
+