docs: add smart gap analysis & question cards design spec

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docs/superpowers/specs/2026-04-12-smart-gap-analysis-design.md

@@ -0,0 +1,333 @@
+# Smart Gap Analysis & Question Cards
+
+**Date:** 2026-04-12
+**Status:** Draft
+
+## Problem
+
+When CAD, CNC, or CAM agents detect missing information, they respond with unstructured text ("NOT READY: need dimensions and material"). The orchestrator passes this message through to the user without analysis. The user must figure out what to provide and who to address. CNC and CAM agents lack any structured problem-reporting protocol at all.
+
+## Solution
+
+1. **Standardized NOT READY protocol** for CAD, CNC, and CAM agents.
+2. **Gap analyzer** that scans agent responses for NOT READY prefixes and categorizes missing items via regex/keyword matching.
+3. **Question cards** — actionable UI cards with the responsible agent, a question, and clickable suggestions — appended to the chat response.
+4. **Answers go through normal `sendMessage()` flow** — full crew routing, no special handling.
+
+## Agent Problem Reporting Protocol
+
+All three output-producing agents (CAD, CNC, CAM) use the same prefix: `NOT READY:` followed by natural language listing what's missing.
+
+### Agent Task Description Changes
+
+**CNC agent** — currently only says "ask a SPECIFIC clarifying question". New instruction appended to task description:
+
+```
+If the design lacks critical manufacturability info (material, dimensions,
+feature access, tolerance), start with 'NOT READY:' and list missing items.
+If enough info exists, provide your manufacturability assessment.
+```
+
+**CAM agent** — currently only says "create optimal machining strategy". New instruction appended to task description:
+
+```
+If there is no CAD model generated yet or critical machining info is missing
+(material, dimensions, surface finish requirements), start with 'NOT READY:'
+and list missing items. If enough info exists, create the machining plan.
+```
+
+**CAD agent** — unchanged. Already uses `NOT READY:` protocol.
+
+**Design and Engineering agents** — unchanged. They stay conversational and exploratory.
+
+## Gap Analysis
+
+### Data Models
+
+```python
+class MissingItem(BaseModel):
+    category: str       # "dimension", "material", "feature", "constraint", etc.
+    description: str    # "width", "wall thickness", "material selection"
+    agent_id: str       # which agent flagged it
+
+class GapAnalysis(BaseModel):
+    has_gaps: bool
+    missing_items: list[MissingItem]
+```
+
+### How It Works
+
+1. After all agent responses are collected, `analyze_gaps(responses)` scans each response.
+2. If an agent's message starts with `NOT READY:`, extract the text after the prefix.
+3. Run keyword/regex matching on that text to categorize missing items:
+
+| Keywords in NOT READY text | Category | Responsible Agent |
+|---|---|---|
+| width, height, depth, length, diameter, dimension, size | `dimension` | engineering |
+| material, alloy, grade, aluminum, steel, metal | `material` | engineering |
+| shape, form, type, profile, geometry, what kind | `shape` | design |
+| hole, fillet, chamfer, pocket, slot, feature | `feature` | design |
+| tolerance, wall thickness, constraint, min wall, max size | `constraint` | engineering |
+| axis, machine, setup, fixture, tool access | `machining` | cnc |
+| surface finish, roughness, Ra | `finish` | cnc |
+| model, CAD, 3D, generate | `model` | cad |
+
+4. Deduplicate — if both CAD and CNC flag "material", only one `MissingItem` is kept (first wins).
+5. Return `GapAnalysis` with the deduplicated list.
+
+### Agent Assignment
+
+Each category maps to a "responsible agent" — the agent best suited to help the user provide that info. This mapping is configurable in `config.yaml` under `gap_analysis.category_agents`.
+
+## Question Cards
+
+### Data Model
+
+```python
+class QuestionCard(BaseModel):
+    category: str           # from MissingItem
+    question: str           # human-readable question
+    responsible_agent: str  # agent_id
+    agent_name: str         # display name
+    agent_color: str        # for frontend rendering
+    suggestions: list[str]  # clickable options
+    allow_custom: bool      # show free-text input
+```
+
+### Question Templates
+
+| Category | Question | Suggestions |
+|---|---|---|
+| `dimension` | "What are the part dimensions?" | (none — free input fields for width/height/depth) |
+| `material` | "What material should this be made from?" | ["Aluminum 6061", "Aluminum 7075", "Steel 304", "Steel 316", "Brass", "Titanium"] |
+| `shape` | "What type of part are you designing?" | ["Bracket", "Enclosure", "Plate", "Shaft", "Gear", "Flange"] |
+| `feature` | "What features does the part need?" | ["Mounting holes", "Fillets", "Chamfers", "Pockets", "Slots"] |
+| `constraint` | "Are there any manufacturing constraints?" | ["Min wall 2mm", "Min wall 3mm", "Min wall 5mm"] |
+| `machining` | "What machining approach?" | ["3-axis", "3+2-axis", "5-axis", "Auto"] |
+| `finish` | "What surface finish is required?" | ["Standard", "Fine (Ra 1.6)", "Mirror (Ra 0.4)"] |
+| `model` | "A 3D model is needed first. Provide more design details or approve the plan to generate." | (no suggestions — informational only) |
+
+These templates are defined in code (not config) — they are tightly coupled to the category set.
+
+### Filtering Against Existing State
+
+Before generating cards, `generate_question_cards(gap_analysis, design_state)` checks each `MissingItem` against the current `DesignState`:
+- If `material` is already set, skip the material card even if an agent flagged it.
+- If `dimensions` has entries, skip the dimension card.
+- This prevents asking the user for info they already provided.
+
+## Orchestrator Integration
+
+### Position in the Flow
+
+In `CrewOrchestrator._run_crew`, after agent responses are collected, before final return:
+
+```
+Agent responses collected
+    |
+    v
+Extract decisions (existing) -> updated_state
+    |
+    v
+analyze_gaps(responses) -> GapAnalysis
+    |
+    v
+if has_gaps:
+    generate_question_cards(gap_analysis, updated_state) -> list[QuestionCard]
+    attach to response
+    |
+    v
+Check score / auto-trigger plan (existing)
+    |
+    v
+Return result
+```
+
+### Modified Response Shape
+
+```json
+{
+    "responses": [...],
+    "preview": null,
+    "design_state": {...},
+    "question_cards": [
+        {
+            "category": "material",
+            "question": "What material should this be made from?",
+            "responsible_agent": "engineering",
+            "agent_name": "Engineering Agent",
+            "agent_color": "#00b4d8",
+            "suggestions": ["Aluminum 6061", "Aluminum 7075", "Steel 304", "Brass"],
+            "allow_custom": true
+        },
+        {
+            "category": "dimension",
+            "question": "What are the part dimensions?",
+            "responsible_agent": "engineering",
+            "agent_name": "Engineering Agent",
+            "agent_color": "#00b4d8",
+            "suggestions": [],
+            "allow_custom": true
+        }
+    ]
+}
+```
+
+`question_cards` is an empty list when no gaps exist.
+
+### Interaction with Planning Phase
+
+- **`exploring` phase:** cards are generated normally.
+- **`approved` phase:** if CAD/CNC/CAM say NOT READY, phase resets to `exploring` AND cards are generated from the gaps — the user sees both the reset and what to fix.
+- **`planning` phase:** no cards (user is reviewing the plan, not chatting).
+
+## Frontend Question Cards
+
+### Layout
+
+Rendered inline in the chat, after agent messages:
+
+```
++------------------------------+
+|  MISSING INFO                |
+|                              |
+|  EA  What material?          |
+|  [Aluminum 6061] [Alu 7075] |
+|  [Steel 304] [Brass]        |
+|  [Custom: ___________]      |
+|                              |
+|  EA  Part dimensions?        |
+|  Width: [____] mm            |
+|  Height: [____] mm           |
+|  Depth: [____] mm            |
+|  [Submit]                    |
+|                              |
++------------------------------+
+```
+
+### Card Behavior
+
+- **Chip-based cards** (material, shape, features, machining, finish, constraint): clicking a chip immediately sends the value through `sendMessage()` as a natural language message. Cards are removed after submission.
+- **Dimension cards:** render input fields (width/height/depth). A "Submit" button sends all dimension values at once as a composed message (e.g., "60mm wide, 40mm high, 20mm deep"). Cards are removed after submission.
+- **Informational cards** (model category): no interaction — just a message explaining what's needed.
+- Only one set of question cards is active at a time. New cards from a subsequent turn replace previous ones.
+- Cards are removed on any `sendMessage()` call — whether the user interacts with a card or types freely in the chat input.
+
+### Styling
+
+- Reuses existing `.wizard-chip`, `.wizard-dim-input` classes for consistency with the guided wizard.
+- New `.gap-card` container class with a left border colored by the responsible agent's color.
+- Agent avatar dot shown next to the agent name in each card.
+
+### Client-Side State Update
+
+When the user clicks a suggestion or submits dimensions:
+1. Update `designState` immediately on the client side (same as the wizard does).
+2. Send the composed message through `sendMessage()`.
+3. Normal crew routing handles it — no special routing.
+
+## Configuration
+
+### New `gap_analysis` section in `config.yaml`
+
+```yaml
+gap_analysis:
+  category_keywords:
+    dimension:
+      - width
+      - height
+      - depth
+      - length
+      - diameter
+      - dimension
+      - size
+    material:
+      - material
+      - alloy
+      - grade
+      - aluminum
+      - steel
+      - metal
+    shape:
+      - shape
+      - form
+      - type
+      - profile
+      - geometry
+      - what kind
+    feature:
+      - hole
+      - fillet
+      - chamfer
+      - pocket
+      - slot
+      - feature
+    constraint:
+      - tolerance
+      - wall thickness
+      - constraint
+      - min wall
+      - max size
+    machining:
+      - axis
+      - machine
+      - setup
+      - fixture
+      - tool access
+    finish:
+      - surface finish
+      - roughness
+      - Ra
+    model:
+      - model
+      - CAD
+      - 3D
+      - generate
+  category_agents:
+    dimension: engineering
+    material: engineering
+    shape: design
+    feature: design
+    constraint: engineering
+    machining: cnc
+    finish: cnc
+    model: cad
+```
+
+### Pydantic Config Model
+
+```python
+class GapAnalysisConfig(BaseModel):
+    category_keywords: dict[str, list[str]] = Field(default_factory=dict)
+    category_agents: dict[str, str] = Field(default_factory=dict)
+```
+
+Added to `Settings` alongside existing `planning` field.
+
+## Files Changed
+
+| File | Change |
+|------|--------|
+| `agents/gap_analyzer.py` | **New file.** `MissingItem`, `GapAnalysis`, `QuestionCard` models. `analyze_gaps()` and `generate_question_cards()` functions. Question templates. |
+| `agents/crew_orchestrator.py` | Import gap analyzer. Call `analyze_gaps()` after response collection. Call `generate_question_cards()` if gaps found. Add `question_cards` to return dict. Update CNC/CAM task descriptions with NOT READY protocol. |
+| `config/settings.py` | Add `GapAnalysisConfig` model, add `gap_analysis` field to `Settings`. |
+| `config.yaml` | Add `gap_analysis` section. Update CNC and CAM agent backstories with NOT READY instructions. |
+| `web/index.html` | Add `.gap-card` CSS. Add `renderQuestionCards()` JS function. Update `sendMessage()` to render cards when present. Add card interaction handlers (chip click, dimension submit). |
+| `tests/test_gap_analyzer.py` | **New file.** Tests for `analyze_gaps()`, `generate_question_cards()`, deduplication, state filtering. |
+
+## Files NOT Changed
+
+| File | Reason |
+|------|--------|
+| `agents/design_state.py` | Gap analysis is separate from state accumulation |
+| `server/routes.py` | No new endpoints — cards flow through existing `/api/chat` response |
+| `server/web.py` | No new serving needs |
+| `core/executor.py` | Unchanged |
+| `core/cam.py` | Unchanged |
+
+## Testing Strategy
+
+- **Unit tests for `analyze_gaps()`** — verify NOT READY detection, keyword categorization, deduplication across agents, handling of responses without NOT READY prefix.
+- **Unit tests for `generate_question_cards()`** — verify correct question/suggestion templates per category, filtering against populated DesignState fields, agent metadata lookup.
+- **Integration test** — verify that a `chat_turn` with a CAD agent returning NOT READY produces `question_cards` in the response alongside agent messages.
+- **Frontend manual testing** — verify cards render after NOT READY response, chip clicks send messages, dimension submit works, cards are removed after interaction.