| # 08 — Build Plan |
|
|
| ## Timeline overview |
|
|
| ``` |
| NOW (3 weeks before) Pre-hack prep |
| HACK DAY 1 (Sat Aug 22) Build core features |
| HACK DAY 2 (Sun Aug 23) Polish, integrate, demo |
| POST-HACK Publish, ship, follow up |
| ``` |
|
|
| ## Pre-hackathon (you have ~3 weeks) |
|
|
| Goal: arrive at the event with a working skeleton, training data ready, and most patterns written. This compresses day 1 to "polish core, add stretch features" instead of "build from zero." |
|
|
| ### Week 1: foundations |
|
|
| **By end of week 1:** |
|
|
| - [ ] Get Stable Audio 3 API access / model weights downloaded |
| - [ ] HuggingFace account created |
| - [ ] Request gated access to `stabilityai/stable-audio-3-small` |
| - [ ] Get inference running locally on Vega (or document why cloud-only) |
| - [ ] Time a generation, establish baseline performance |
| - [ ] Set up RunPod account (or chosen cloud GPU provider) |
| - [ ] Test deploying a simple inference container |
| - [ ] Verify you can ssh/exec into it |
| - [ ] Stand up the repo |
| - [ ] Next.js + TypeScript + Tailwind initialized |
| - [ ] Folder structure per [`02-architecture.md`](02-architecture.md#folder-structure) |
| - [ ] CI with axe-core for accessibility (even on empty pages, it's a foundation) |
| - [ ] Deploy "coming soon" placeholder to Vercel |
| - [ ] Set up audio service skeleton |
| - [ ] FastAPI + SA3 inference wrapper |
| - [ ] Single `/generate` endpoint, no LoRA yet |
| - [ ] Install NVDA on Windows, run through Reaper once to baseline familiarity |
| - [ ] Set up Reaper with a drum sampler VST for testing MIDI export |
|
|
| ### Week 2: data and templates |
|
|
| **By end of week 2:** |
|
|
| - [ ] Curate training data for brutal-drum LoRA |
| - [ ] 20–40 one-shots (recordings or sourced, all licensed) |
| - [ ] 10–20 short loops |
| - [ ] Manifest in `data/training/manifest.yaml` |
| - [ ] All preprocessed to 44100 Hz mono |
| - [ ] Write the 6 demo-critical pattern templates |
| - [ ] `d-beat.json` |
| - [ ] `blast-traditional.json` |
| - [ ] `skank.json` |
| - [ ] `half-time-metal.json` |
| - [ ] `djent-polyrhythm.json` |
| - [ ] `punk-rock.json` |
| - [ ] Write the onomatopoeia mapping table with at least 10 entries |
| - [ ] Implement prompt parser MVP (handles the 6 patterns + 10 onomatopoeias) |
| - [ ] Implement pattern engine MVP (loads templates, expands to bars, applies tempo) |
| - [ ] Implement MIDI generation (downloads work, drags into Reaper) |
|
|
| ### Week 3: integration and rehearsal |
|
|
| **By end of week 3:** |
|
|
| - [ ] Train brutal-drum LoRA on cloud GPU |
| - [ ] First attempt: 500 steps, see how it shapes up |
| - [ ] Iterate: adjust learning rate, dataset, prompt format |
| - [ ] Final: ship-ready LoRA + publish to HuggingFace (draft, don't publish yet) |
| - [ ] Implement Reaper Web Control client |
| - [ ] Connect, parse state, extract tempo |
| - [ ] Tempo resolution priority implemented |
| - [ ] Implement Web Speech API integration |
| - [ ] STT works in Chrome |
| - [ ] TTS works |
| - [ ] Voice state machine (idle → listening → parsing → generating → ready) |
| - [ ] Build basic UI shell |
| - [ ] Voice button, transcript display, status panel |
| - [ ] Action buttons (play, download, regenerate) |
| - [ ] Visual grid component (basic version) |
| - [ ] Accessibility pass 1 |
| - [ ] Tab through every element |
| - [ ] NVDA run-through of the voice flow |
| - [ ] ARIA labels on every interactive element |
| - [ ] Skip links |
| - [ ] High contrast check |
| - [ ] End-to-end rehearsal |
| - [ ] Generate a pattern in PatternTalk |
| - [ ] Drag MIDI into Reaper |
| - [ ] Play it |
| - [ ] Generate sample |
| - [ ] Play sample |
| - [ ] Switch on NVDA, redo the flow |
| - [ ] Write the team-pitch and post to Music Hackspace Discord |
|
|
| ## Hackathon Day 1 (Saturday Aug 22) |
|
|
| ### Morning (9:00 AM – 12:30 PM): setup + team formation |
|
|
| - [ ] Arrive at PHI Centre, get settled |
| - [ ] Finalize team composition (if recruiting on-site) |
| - [ ] Confirm Reaper + PatternTalk working environment |
| - [ ] Connect to venue WiFi, verify cloud GPU access |
|
|
| ### Early afternoon (12:30 PM – 3:00 PM): core demo path |
|
|
| **Goal:** End-to-end voice → pattern → MIDI → Reaper working. |
|
|
| - [ ] Finalize pattern engine: tempo, bars, time signature, cymbal overrides |
| - [ ] Finalize MIDI export, verify it sounds right in Reaper |
| - [ ] Wire up Web Speech API end-to-end |
| - [ ] Wire up sample generation from cloud GPU |
| - [ ] First smoke test: voice prompt → MIDI + sample → download → Reaper |
|
|
| ### Late afternoon (3:00 PM – 6:00 PM): variations and audio context |
|
|
| - [ ] Implement 4-variation engine (humanize velocities, micro-timing, fill variants) |
| - [ ] Implement uploaded audio BPM detection (Meyda) |
| - [ ] Wire up Reaper tempo sync (Web Control client) |
| - [ ] Pre-generate demo variations for the showcase patterns |
|
|
| ### Evening (6:00 PM – 9:00 PM): polish + accessibility |
|
|
| - [ ] Visual grid component (sighted users) |
| - [ ] Pattern library page (skeleton) |
| - [ ] Accessibility pass with NVDA |
| - [ ] Tab through everything |
| - [ ] Voice announcements work |
| - [ ] Error states announced |
| - [ ] Demo rehearsal (round 1) |
| - [ ] Evening check-in / informal demos at the venue |
|
|
| **Day 1 deliverable:** A working PatternTalk that takes voice prompts, generates MIDI, downloads it, and plays in Reaper. Plus 4-variation picker. Plus screenreader-tested. |
|
|
| ## Hackathon Day 2 (Sunday Aug 23) |
|
|
| ### Morning (9:00 AM – 12:00 PM): stretch features |
|
|
| Priority order (cut from the bottom if time runs short): |
|
|
| 1. [ ] **Sample preview player** with progress indicator |
| 2. [ ] **Pattern library** with search/filter and shareable URLs |
| 3. [ ] **User-defined onomatopoeias** (localStorage) |
| 4. [ ] **CLAP plugin wrap** of the web UI (if a teammate can take it) |
| 5. [ ] **Real-time audio captioning** (stretch, probably cut) |
|
|
| ### Midday (12:00 PM – 2:00 PM): polish |
|
|
| - [ ] Final visual grid polish (animations, color tuning) |
| - [ ] Voice response tuning (speak rate, prompts) |
| - [ ] Pre-generate all demo cache samples |
| - [ ] Pre-stage the LoRA weights download for live demo |
| - [ ] Test with cloud GPU off (Vega-only path) as fallback |
|
|
| ### Afternoon (2:00 PM – 4:00 PM): demo rehearsal |
|
|
| - [ ] Full demo run-through, timed (target: 3–4 minutes) |
| - [ ] NVDA demo run-through, timed |
| - [ ] VoiceOver demo run-through, timed |
| - [ ] Rehearse the pitch (3 sentences max) |
| - [ ] Backup plan: video recording of working demo, in case of network issues |
|
|
| ### Late afternoon (4:00 PM – 6:00 PM): present |
|
|
| - [ ] Public demos |
| - [ ] Jury presentation |
| - [ ] Feedback session |
|
|
| ### Evening (6:00 PM onwards): wind down |
|
|
| - [ ] Final cleanup |
| - [ ] Publish LoRA to HuggingFace (if not already) |
| - [ ] Push code to GitHub, write a proper README |
| - [ ] Write blog post / social announcement |
|
|
| ## Checkpoints |
|
|
| Use these to catch problems early. If a checkpoint fails, stop and address it before moving on. |
|
|
| | Checkpoint | When | What "passing" means | |
| |---|---|---| |
| | **C1: Inference works** | Pre-hack week 1 | SA3 small generates a sample on Vega in < 10 min OR cloud GPU works | |
| | **C2: Training data ready** | Pre-hack week 2 | 30+ samples in `data/training/` with manifest, all licensed | |
| | **C3: 6 patterns authored** | Pre-hack week 2 | All 6 demo patterns load, expand, generate MIDI | |
| | **C4: End-to-end MIDI** | Pre-hack week 3 | Voice prompt → MIDI → Reaper plays correctly | |
| | **C5: NVDA reads the UI** | Pre-hack week 3 | Screenreader announces every state change | |
| | **C6: Demo path works** | Hack day 1 | Full voice → MIDI → sample → Reaper in < 60 seconds | |
| | **C7: LoRA published** | Hack day 1 evening | Weights on HuggingFace with model card | |
| | **C8: Variations work** | Hack day 2 morning | 4 variations show in UI, picker works | |
| | **C9: Demo polished** | Hack day 2 afternoon | Rehearsal runs in 3–4 minutes without mistakes | |
|
|
| ## What to cut if time runs out |
|
|
| In priority order (drop from the bottom): |
|
|
| 1. ❌ Real-time audio captioning |
| 2. ❌ Haptic metronome PWA |
| 3. ❌ Reduced-physical-load mode |
| 4. ❌ CLAP plugin wrap |
| 5. ❌ User-defined onomatopoeias |
| 6. ❌ Pattern library page (if a minimal in-app picker works) |
| 7. ❌ Audio context BPM detection (fall back to prompt/Reaper-only) |
| 8. ❌ Visual grid polish (fall back to functional ugly grid) |
| 9. ❌ Dark mode (fall back to light only) |
|
|
| The must-haves are: voice prompt, pattern generation, MIDI export, sample generation, Reaper tempo sync. If we have those five, we have a demo that lands. |
|
|
| ## What to add if time is plentiful |
|
|
| In priority order: |
|
|
| 1. ✅ CLAP plugin wrap |
| 2. ✅ Audio context BPM detection |
| 3. ✅ Pattern library page with shareable URLs |
| 4. ✅ User-defined onomatopoeias |
| 5. ✅ Visual grid polish with animations |
| 6. ✅ Dark mode + theme support |
| 7. ✅ Multiple LoRA adapters (one per genre: rock, jazz, funk) |
| 8. ✅ Real-time audio captioning |
|
|
| ## Communication cadence |
|
|
| - **Standup** (15 min) at start of each day |
| - **Mid-day check-in** (5 min, async) — what's blocked, what's next |
| - **End-of-day retro** (15 min) — what shipped, what's deferred |
| - **Demo** (3–4 min) at end of each day for informal feedback |
|
|
| ## Post-hackathon |
|
|
| **Within 1 week:** |
| - [ ] Publish LoRA weights to HuggingFace |
| - [ ] Publish repo on GitHub with full README |
| - [ ] Blog post / dev.to article about the project |
| - [ ] Submit to Music Hackspace showcase if invited |
|
|
| **Within 1 month:** |
| - [ ] Add more pattern templates (target: 30+) |
| - [ ] Train LoRAs for additional genres |
| - [ ] Add CLAP plugin wrap (if not done at hackathon) |
| - [ ] Integrate with Narwall for continuous accessibility testing |
|
|
| **Within 3 months:** |
| - [ ] Public launch of the web app |
| - [ ] Community pattern library with moderation |
| - [ ] "Train your own LoRA" UI |
| - [ ] Mobile companion app for haptic metronome |
|
|
