docs/08-build-plan.md

# 08 — Build Plan

## Timeline overview

```
NOW (3 weeks before)        Pre-hack prep
HACK DAY 1 (Sat Aug 22)     Build core features
HACK DAY 2 (Sun Aug 23)     Polish, integrate, demo
POST-HACK                   Publish, ship, follow up
```

## Pre-hackathon (you have ~3 weeks)

Goal: arrive at the event with a working skeleton, training data ready, and most patterns written. This compresses day 1 to "polish core, add stretch features" instead of "build from zero."

### Week 1: foundations

**By end of week 1:**

- [ ] Get Stable Audio 3 API access / model weights downloaded
  - [ ] HuggingFace account created
  - [ ] Request gated access to `stabilityai/stable-audio-3-small`
  - [ ] Get inference running locally on Vega (or document why cloud-only)
  - [ ] Time a generation, establish baseline performance
- [ ] Set up RunPod account (or chosen cloud GPU provider)
  - [ ] Test deploying a simple inference container
  - [ ] Verify you can ssh/exec into it
- [ ] Stand up the repo
  - [ ] Next.js + TypeScript + Tailwind initialized
  - [ ] Folder structure per [`02-architecture.md`](02-architecture.md#folder-structure)
  - [ ] CI with axe-core for accessibility (even on empty pages, it's a foundation)
  - [ ] Deploy "coming soon" placeholder to Vercel
- [ ] Set up audio service skeleton
  - [ ] FastAPI + SA3 inference wrapper
  - [ ] Single `/generate` endpoint, no LoRA yet
- [ ] Install NVDA on Windows, run through Reaper once to baseline familiarity
- [ ] Set up Reaper with a drum sampler VST for testing MIDI export

### Week 2: data and templates

**By end of week 2:**

- [ ] Curate training data for brutal-drum LoRA
  - [ ] 20–40 one-shots (recordings or sourced, all licensed)
  - [ ] 10–20 short loops
  - [ ] Manifest in `data/training/manifest.yaml`
  - [ ] All preprocessed to 44100 Hz mono
- [ ] Write the 6 demo-critical pattern templates
  - [ ] `d-beat.json`
  - [ ] `blast-traditional.json`
  - [ ] `skank.json`
  - [ ] `half-time-metal.json`
  - [ ] `djent-polyrhythm.json`
  - [ ] `punk-rock.json`
- [ ] Write the onomatopoeia mapping table with at least 10 entries
- [ ] Implement prompt parser MVP (handles the 6 patterns + 10 onomatopoeias)
- [ ] Implement pattern engine MVP (loads templates, expands to bars, applies tempo)
- [ ] Implement MIDI generation (downloads work, drags into Reaper)

### Week 3: integration and rehearsal

**By end of week 3:**

- [ ] Train brutal-drum LoRA on cloud GPU
  - [ ] First attempt: 500 steps, see how it shapes up
  - [ ] Iterate: adjust learning rate, dataset, prompt format
  - [ ] Final: ship-ready LoRA + publish to HuggingFace (draft, don't publish yet)
- [ ] Implement Reaper Web Control client
  - [ ] Connect, parse state, extract tempo
  - [ ] Tempo resolution priority implemented
- [ ] Implement Web Speech API integration
  - [ ] STT works in Chrome
  - [ ] TTS works
  - [ ] Voice state machine (idle → listening → parsing → generating → ready)
- [ ] Build basic UI shell
  - [ ] Voice button, transcript display, status panel
  - [ ] Action buttons (play, download, regenerate)
  - [ ] Visual grid component (basic version)
- [ ] Accessibility pass 1
  - [ ] Tab through every element
  - [ ] NVDA run-through of the voice flow
  - [ ] ARIA labels on every interactive element
  - [ ] Skip links
  - [ ] High contrast check
- [ ] End-to-end rehearsal
  - [ ] Generate a pattern in PatternTalk
  - [ ] Drag MIDI into Reaper
  - [ ] Play it
  - [ ] Generate sample
  - [ ] Play sample
  - [ ] Switch on NVDA, redo the flow
- [ ] Write the team-pitch and post to Music Hackspace Discord

## Hackathon Day 1 (Saturday Aug 22)

### Morning (9:00 AM – 12:30 PM): setup + team formation

- [ ] Arrive at PHI Centre, get settled
- [ ] Finalize team composition (if recruiting on-site)
- [ ] Confirm Reaper + PatternTalk working environment
- [ ] Connect to venue WiFi, verify cloud GPU access

### Early afternoon (12:30 PM – 3:00 PM): core demo path

**Goal:** End-to-end voice → pattern → MIDI → Reaper working.

- [ ] Finalize pattern engine: tempo, bars, time signature, cymbal overrides
- [ ] Finalize MIDI export, verify it sounds right in Reaper
- [ ] Wire up Web Speech API end-to-end
- [ ] Wire up sample generation from cloud GPU
- [ ] First smoke test: voice prompt → MIDI + sample → download → Reaper

### Late afternoon (3:00 PM – 6:00 PM): variations and audio context

- [ ] Implement 4-variation engine (humanize velocities, micro-timing, fill variants)
- [ ] Implement uploaded audio BPM detection (Meyda)
- [ ] Wire up Reaper tempo sync (Web Control client)
- [ ] Pre-generate demo variations for the showcase patterns

### Evening (6:00 PM – 9:00 PM): polish + accessibility

- [ ] Visual grid component (sighted users)
- [ ] Pattern library page (skeleton)
- [ ] Accessibility pass with NVDA
  - [ ] Tab through everything
  - [ ] Voice announcements work
  - [ ] Error states announced
- [ ] Demo rehearsal (round 1)
- [ ] Evening check-in / informal demos at the venue

**Day 1 deliverable:** A working PatternTalk that takes voice prompts, generates MIDI, downloads it, and plays in Reaper. Plus 4-variation picker. Plus screenreader-tested.

## Hackathon Day 2 (Sunday Aug 23)

### Morning (9:00 AM – 12:00 PM): stretch features

Priority order (cut from the bottom if time runs short):

1. [ ] **Sample preview player** with progress indicator
2. [ ] **Pattern library** with search/filter and shareable URLs
3. [ ] **User-defined onomatopoeias** (localStorage)
4. [ ] **CLAP plugin wrap** of the web UI (if a teammate can take it)
5. [ ] **Real-time audio captioning** (stretch, probably cut)

### Midday (12:00 PM – 2:00 PM): polish

- [ ] Final visual grid polish (animations, color tuning)
- [ ] Voice response tuning (speak rate, prompts)
- [ ] Pre-generate all demo cache samples
- [ ] Pre-stage the LoRA weights download for live demo
- [ ] Test with cloud GPU off (Vega-only path) as fallback

### Afternoon (2:00 PM – 4:00 PM): demo rehearsal

- [ ] Full demo run-through, timed (target: 3–4 minutes)
- [ ] NVDA demo run-through, timed
- [ ] VoiceOver demo run-through, timed
- [ ] Rehearse the pitch (3 sentences max)
- [ ] Backup plan: video recording of working demo, in case of network issues

### Late afternoon (4:00 PM – 6:00 PM): present

- [ ] Public demos
- [ ] Jury presentation
- [ ] Feedback session

### Evening (6:00 PM onwards): wind down

- [ ] Final cleanup
- [ ] Publish LoRA to HuggingFace (if not already)
- [ ] Push code to GitHub, write a proper README
- [ ] Write blog post / social announcement

## Checkpoints

Use these to catch problems early. If a checkpoint fails, stop and address it before moving on.

| Checkpoint | When | What "passing" means |
|---|---|---|
| **C1: Inference works** | Pre-hack week 1 | SA3 small generates a sample on Vega in < 10 min OR cloud GPU works |
| **C2: Training data ready** | Pre-hack week 2 | 30+ samples in `data/training/` with manifest, all licensed |
| **C3: 6 patterns authored** | Pre-hack week 2 | All 6 demo patterns load, expand, generate MIDI |
| **C4: End-to-end MIDI** | Pre-hack week 3 | Voice prompt → MIDI → Reaper plays correctly |
| **C5: NVDA reads the UI** | Pre-hack week 3 | Screenreader announces every state change |
| **C6: Demo path works** | Hack day 1 | Full voice → MIDI → sample → Reaper in < 60 seconds |
| **C7: LoRA published** | Hack day 1 evening | Weights on HuggingFace with model card |
| **C8: Variations work** | Hack day 2 morning | 4 variations show in UI, picker works |
| **C9: Demo polished** | Hack day 2 afternoon | Rehearsal runs in 3–4 minutes without mistakes |

## What to cut if time runs out

In priority order (drop from the bottom):

1. ❌ Real-time audio captioning
2. ❌ Haptic metronome PWA
3. ❌ Reduced-physical-load mode
4. ❌ CLAP plugin wrap
5. ❌ User-defined onomatopoeias
6. ❌ Pattern library page (if a minimal in-app picker works)
7. ❌ Audio context BPM detection (fall back to prompt/Reaper-only)
8. ❌ Visual grid polish (fall back to functional ugly grid)
9. ❌ Dark mode (fall back to light only)

The must-haves are: voice prompt, pattern generation, MIDI export, sample generation, Reaper tempo sync. If we have those five, we have a demo that lands.

## What to add if time is plentiful

In priority order:

1. ✅ CLAP plugin wrap
2. ✅ Audio context BPM detection
3. ✅ Pattern library page with shareable URLs
4. ✅ User-defined onomatopoeias
5. ✅ Visual grid polish with animations
6. ✅ Dark mode + theme support
7. ✅ Multiple LoRA adapters (one per genre: rock, jazz, funk)
8. ✅ Real-time audio captioning

## Communication cadence

- **Standup** (15 min) at start of each day
- **Mid-day check-in** (5 min, async) — what's blocked, what's next
- **End-of-day retro** (15 min) — what shipped, what's deferred
- **Demo** (3–4 min) at end of each day for informal feedback

## Post-hackathon

**Within 1 week:**
- [ ] Publish LoRA weights to HuggingFace
- [ ] Publish repo on GitHub with full README
- [ ] Blog post / dev.to article about the project
- [ ] Submit to Music Hackspace showcase if invited

**Within 1 month:**
- [ ] Add more pattern templates (target: 30+)
- [ ] Train LoRAs for additional genres
- [ ] Add CLAP plugin wrap (if not done at hackathon)
- [ ] Integrate with Narwall for continuous accessibility testing

**Within 3 months:**
- [ ] Public launch of the web app
- [ ] Community pattern library with moderation
- [ ] "Train your own LoRA" UI
- [ ] Mobile companion app for haptic metronome