docs/REMAINING_WORK.md

Remaining work

Last updated: 2026-05-12

Current gap assessment

The project is now a usable extraction workstation, not a complete interactive sample editor. The largest remaining gaps are UX/editor capabilities rather than core batch extraction.

Failed API requests are now visible in the UI, and browser-style parameter values are coerced before backend validation.

Highest-priority remaining gaps

Completed since the previous snapshot: reproduced audio now incorporates non-target context stems, and common/advanced parameters are separated in the right sidebar.

1. Cluster editing: allow merge, split, relabel, and manual reassignment of groups from the Review & edit workbench.
2. Waveform editing depth: add onset drag/shift, hit trim boundaries, and rerun-from-edited-onsets without redoing Demucs.
3. Run comparison: compare two manifests side-by-side for parameter tuning.
4. Lower-level progress: expose internal Demucs/clustering progress where libraries make that possible.
5. Frontend engineering hardening: migrate the frontend to TypeScript after the UX stabilizes and add browser-level tests.
6. Benchmark panel: add an in-app benchmark view that can run synthetic fixtures and compare parameter profiles.
7. Reproduction diagnostics: add source-vs-reproduced A/B error visualization and region ranking.

Known constraints

• Demucs is not a realtime stage and should stay explicitly offline/cached.
• Batch agglomerative clustering is not realtime; online_preview is the progressive clustering path.
• First run on a fresh environment can be slower due to imports, model download, and library initialization.
• The current job queue is process-local and single-worker. That is fine for local use, but not enough for a shared public deployment.
• Run history is filesystem-backed via .runs/; deleting .runs/ deletes history.

Suggested implementation order

1. Add edit state to manifests: deleted hits, shifted onsets, labels, cluster overrides.
2. Add rerender/repack endpoint that starts from edited hit/cluster state.
3. Add cluster merge/split/relabel actions in the UI.
4. Add run comparison view.
5. Add lower-level progress hooks inside expensive stages where practical.
6. Convert frontend to TypeScript and add UI tests.
7. Add an in-app benchmark/parameter profile panel.

Remaining after interactive UX foundation

Completed since the previous remaining-work snapshot:

• Supplied docs/interactive-ux document set embedded and aligned.
• Persistent supervised state added via supervised_state.py and supervision_state.json.
• Constraint store and event log added.
• Hit/cluster confidence and outlier-first review queue added.
• Move hit, pull-out, lock/unlock, suppress, review/favorite, suggestions, explanations, and undo endpoints added.
• Interactive supervision UI panel added.

Highest-priority remaining work now:

1. Supervised artifact export: regenerate edited sample WAVs, MIDI, reconstruction, manifest, and ZIP from supervision_state.json without rerunning Demucs/onset detection.
2. Force-onset correction: add an onset by clicking/shift-clicking the waveform, slice from cached stem.wav, classify, assign, and store a force-onset constraint.
3. Suppression restore: restore suppressed hits individually and in batches.
4. Real constrained local reclustering: cache hit feature vectors and recompute affected neighborhoods after edits.
5. Suggestion diff preview: show exact before/after membership changes before accepting a suggestion.
6. Constraint violation detection: explicitly report conflicting user constraints.
7. Frontend tests and TypeScript migration: harden the increasingly stateful UI.

Closed in Pass 5

• Supervised edited-state export now writes supervised/manifest.json, edited samples, edited MIDI, edited reconstruction WAV, and edited ZIP.
• Suppressed hits are excluded from edited exports.
• Favorite/pinned representatives are honored by edited exports.
• Add-onset mode writes forced hit slices from stem.wav.
• Suppressed hits can be restored without undoing unrelated edits.
• Suggestions expose exact before/after diffs and the UI can preview them.

Current top remaining gaps

1. Cluster merge/relabel/split workflows.
2. Cached feature-vector local reclustering around edited hits.
3. Edited-vs-original comparison view.
4. Batch restore / bulk operations for suppressed hits.
5. Browser-level UI tests and TypeScript/Vite hardening.

UI status note

The default UI is now a cleaner fixed, non-scrolling workstation layout with collapsed side/bottom utility panels, explicit upload, and whole-app drag/drop. Remaining UI work is interaction depth and engineering hardening: browser screenshot regression tests, TypeScript/Vite migration, edited-vs-original comparison, waveform zoom/pan, a shortcut/help overlay for discoverability, and richer cluster merge/split/relabel workflows.

Progress and flow follow-up

• Add internal progress callbacks to onset detection instead of boundary-only updates.
• Add internal progress callbacks to clustering, especially mel fingerprint generation and transient NCC pair processing.
• Add a small source-vs-stem-vs-reproduced comparison view without making the default UI cluttered.
• Add Playwright/browser tests for file drop, immediate waveform rendering, and progress tint updates.

Remaining after automatic card-flow pass

• Promote drawn replacement cards into persisted backend representative choices instead of preview-only additions.
• Make card trim/extend rewrite preview audio immediately, not only create a forced hit for edited export.
• Add true cached feature-vector local reclustering after dismiss/draw/move decisions.
• Add browser-level tests for drag/drop auto-run, progressive cards, waveform zoom/pan, dismiss, draw, and clip edit flows.

Remaining UI fidelity work

• Add true per-card checkbox state instead of treating all visible cards as selected.
• Add selected-only backend export so Export Selected creates an artifact containing only selected representatives.
• Replace Unicode icons with a small icon system if exact visual parity is required.
• Validate with a real browser screenshot comparison against the supplied reference image.

Closed after selected-card/Spleeter pass

• Export Selected now renders a selected-only backend artifact instead of downloading the full generated pack.
• Sample card checkboxes are real per-card state.
• Draw-next is persisted as a representative override in supervision_state.json.
• Trim/extend rewrites preview audio immediately and persists the edited representative hit.
• Spleeter is now the configured default backend; when unavailable, the UI/backend fall back to full-mix mode and Demucs remains an explicit quality backend.

Remaining after selected-card/Spleeter pass

1. Add cluster column merge/split/relabel directly in the card UI.
2. Add localized high-quality separation refinement: run Demucs or another backend on short candidate regions instead of the entire file.
3. Investigate AudioSep-like query-guided separation for overlapping drum events as an optional refinement path.
4. Investigate inpainting for cleaning hit tails/bleed after localization, not for first-pass discovery.
5. Add browser-level regression tests for drop-to-process, waveform zoom/pan, card selection, selected export, draw-next, and trim/extend.
6. Add source-vs-stem-vs-reproduced diagnostics for cards where overlaps remain audible.