Spaces:
Sleeping
Sleeping
File size: 7,552 Bytes
eb1a122 b8fa9bf eb1a122 b8fa9bf ce84147 b8fa9bf 0b5f0f0 3e9170f 0b5f0f0 eb1a122 b8fa9bf eb1a122 b8fa9bf eb1a122 3703c4e 03d531b e07820e f08496f 8c10624 7f1f066 ab6f318 f026127 fa35534 5a90820 fa35534 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | # Remaining work
Last updated: 2026-05-12
## Current gap assessment
The project is now a usable extraction workstation, not a complete interactive sample editor. The largest remaining gaps are UX/editor capabilities rather than core batch extraction.
Failed API requests are now visible in the UI, and browser-style parameter values are coerced before backend validation.
## Highest-priority remaining gaps
Completed since the previous snapshot: reproduced audio now incorporates non-target context stems, and common/advanced parameters are separated in the right sidebar.
1. **Cluster editing**: allow merge, split, relabel, and manual reassignment of groups from the `Review & edit` workbench.
2. **Waveform editing depth**: add onset drag/shift, hit trim boundaries, and rerun-from-edited-onsets without redoing Demucs.
3. **Run comparison**: compare two manifests side-by-side for parameter tuning.
4. **Lower-level progress**: expose internal Demucs/clustering progress where libraries make that possible.
5. **Frontend engineering hardening**: migrate the frontend to TypeScript after the UX stabilizes and add browser-level tests.
6. **Benchmark panel**: add an in-app benchmark view that can run synthetic fixtures and compare parameter profiles.
7. **Reproduction diagnostics**: add source-vs-reproduced A/B error visualization and region ranking.
## Known constraints
- Demucs is not a realtime stage and should stay explicitly offline/cached.
- Batch agglomerative clustering is not realtime; `online_preview` is the progressive clustering path.
- First run on a fresh environment can be slower due to imports, model download, and library initialization.
- The current job queue is process-local and single-worker. That is fine for local use, but not enough for a shared public deployment.
- Run history is filesystem-backed via `.runs/`; deleting `.runs/` deletes history.
## Suggested implementation order
1. Add edit state to manifests: deleted hits, shifted onsets, labels, cluster overrides.
2. Add rerender/repack endpoint that starts from edited hit/cluster state.
3. Add cluster merge/split/relabel actions in the UI.
4. Add run comparison view.
5. Add lower-level progress hooks inside expensive stages where practical.
6. Convert frontend to TypeScript and add UI tests.
7. Add an in-app benchmark/parameter profile panel.
## Remaining after interactive UX foundation
Completed since the previous remaining-work snapshot:
- Supplied `docs/interactive-ux` document set embedded and aligned.
- Persistent supervised state added via `supervised_state.py` and `supervision_state.json`.
- Constraint store and event log added.
- Hit/cluster confidence and outlier-first review queue added.
- Move hit, pull-out, lock/unlock, suppress, review/favorite, suggestions, explanations, and undo endpoints added.
- Interactive supervision UI panel added.
Highest-priority remaining work now:
1. **Supervised artifact export**: regenerate edited sample WAVs, MIDI, reconstruction, manifest, and ZIP from `supervision_state.json` without rerunning Demucs/onset detection.
2. **Force-onset correction**: add an onset by clicking/shift-clicking the waveform, slice from cached `stem.wav`, classify, assign, and store a `force-onset` constraint.
3. **Suppression restore**: restore suppressed hits individually and in batches.
4. **Real constrained local reclustering**: cache hit feature vectors and recompute affected neighborhoods after edits.
5. **Suggestion diff preview**: show exact before/after membership changes before accepting a suggestion.
6. **Constraint violation detection**: explicitly report conflicting user constraints.
7. **Frontend tests and TypeScript migration**: harden the increasingly stateful UI.
## Closed in Pass 5
- Supervised edited-state export now writes `supervised/manifest.json`, edited samples, edited MIDI, edited reconstruction WAV, and edited ZIP.
- Suppressed hits are excluded from edited exports.
- Favorite/pinned representatives are honored by edited exports.
- Add-onset mode writes forced hit slices from `stem.wav`.
- Suppressed hits can be restored without undoing unrelated edits.
- Suggestions expose exact before/after diffs and the UI can preview them.
## Current top remaining gaps
1. Cluster merge/relabel/split workflows.
2. Cached feature-vector local reclustering around edited hits.
3. Edited-vs-original comparison view.
4. Batch restore / bulk operations for suppressed hits.
5. Browser-level UI tests and TypeScript/Vite hardening.
## UI status note
The default UI is now a cleaner fixed, non-scrolling workstation layout with collapsed side/bottom utility panels, explicit upload, and whole-app drag/drop. Remaining UI work is interaction depth and engineering hardening: browser screenshot regression tests, TypeScript/Vite migration, edited-vs-original comparison, waveform zoom/pan, a shortcut/help overlay for discoverability, and richer cluster merge/split/relabel workflows.
## Progress and flow follow-up
- Add internal progress callbacks to onset detection instead of boundary-only updates.
- Add internal progress callbacks to clustering, especially mel fingerprint generation and transient NCC pair processing.
- Add a small source-vs-stem-vs-reproduced comparison view without making the default UI cluttered.
- Add Playwright/browser tests for file drop, immediate waveform rendering, and progress tint updates.
## Remaining after automatic card-flow pass
- Promote drawn replacement cards into persisted backend representative choices instead of preview-only additions.
- Make card trim/extend rewrite preview audio immediately, not only create a forced hit for edited export.
- Add true cached feature-vector local reclustering after dismiss/draw/move decisions.
- Add browser-level tests for drag/drop auto-run, progressive cards, waveform zoom/pan, dismiss, draw, and clip edit flows.
## Remaining UI fidelity work
- Add true per-card checkbox state instead of treating all visible cards as selected.
- Add selected-only backend export so `Export Selected` creates an artifact containing only selected representatives.
- Replace Unicode icons with a small icon system if exact visual parity is required.
- Validate with a real browser screenshot comparison against the supplied reference image.
## Closed after selected-card/Spleeter pass
- `Export Selected` now renders a selected-only backend artifact instead of downloading the full generated pack.
- Sample card checkboxes are real per-card state.
- Draw-next is persisted as a representative override in `supervision_state.json`.
- Trim/extend rewrites preview audio immediately and persists the edited representative hit.
- Spleeter is now the configured default backend; when unavailable, the UI/backend fall back to full-mix mode and Demucs remains an explicit quality backend.
## Remaining after selected-card/Spleeter pass
1. Add cluster column merge/split/relabel directly in the card UI.
2. Add localized high-quality separation refinement: run Demucs or another backend on short candidate regions instead of the entire file.
3. Investigate AudioSep-like query-guided separation for overlapping drum events as an optional refinement path.
4. Investigate inpainting for cleaning hit tails/bleed after localization, not for first-pass discovery.
5. Add browser-level regression tests for drop-to-process, waveform zoom/pan, card selection, selected export, draw-next, and trim/extend.
6. Add source-vs-stem-vs-reproduced diagnostics for cards where overlaps remain audible.
|