Spaces:
Sleeping
Sleeping
| # Interactive UX feature requirements | |
| ## Goal | |
| Add interactions that make extraction faster and more accurate by converting user actions into reusable supervision signals. | |
| The application should progressively converge toward the user's intended drum vocabulary with minimal manual cleanup. | |
| ## Success criteria | |
| - User corrections affect more than the single edited item when safe. | |
| - Explicit user intent is preserved across reclustering and reloads. | |
| - The system surfaces uncertain/high-leverage items before stable ones. | |
| - The user can understand why items are grouped or separated. | |
| - Cached stem/source analysis can be reused while downstream parameters update quickly. | |
| - The UI supports fast audition, correction, locking, and export loops. | |
| ## Current implementation summary | |
| Implemented now: | |
| - Persistent semantic state per run: `supervision_state.json`. | |
| - Event log and constraint store. | |
| - Confidence-weighted hit/cluster state. | |
| - Outlier-first review queue. | |
| - Move hit to cluster. | |
| - Pull hit into new cluster. | |
| - Lock/unlock cluster. | |
| - Suppress hit as bleed/noise. | |
| - Accept/favorite selected hit. | |
| - Suggestion inbox with accept/reject. | |
| - Cluster explanation endpoint and UI drawer. | |
| - Undo for the last semantic edit. | |
| Partially implemented: | |
| - Local recomputation is currently semantic-state recomputation, not full feature-neighborhood reclustering. | |
| - Suggestions are heuristic and preview-count based, not full diff previews. | |
| - Favorite/pin changes semantic representative and supervised export honors it when generating the edited sample pack. | |
| - Confidence scoring is heuristic, not feature-margin/stability based. | |
| Implemented in Pass 5 where noted; remaining items listed below: | |
| - Click-to-add missed onset. | |
| - Restore suppressed hit. | |
| - Supervised re-export from edited state. | |
| - Counterfactual parameter previews. | |
| - Reconstruction-error correction. | |
| - Teach mode across songs. | |
| ## Functional requirements and status | |
| ### FR-001: Cluster move as positive supervision | |
| Status: **implemented as semantic-state edit**. | |
| When a user moves a hit/sample into a cluster, the backend creates `force-cluster` and, when a representative exists, `must-link`. State confidence is recomputed and similar hit suggestions may be generated. | |
| Remaining gap: true local feature-neighborhood reclustering and exact before/after diff preview. | |
| ### FR-002: Pull-out as negative supervision | |
| Status: **implemented as semantic-state edit**. | |
| Pulling a hit out creates a new user cluster, records a `cannot-link` to the source representative when possible, and stores a `force-cluster` assignment for the new cluster. | |
| Remaining gap: automatic split suggestions are heuristic and do not yet run constrained reclustering. | |
| ### FR-003: Lock confirmed cluster identity | |
| Status: **implemented**. | |
| Clusters can be locked/unlocked through the API/UI. Lock state is persisted and influences confidence. | |
| Remaining gap: future full reruns do not yet replay locks into batch clustering. | |
| ### FR-004: Outlier-first review queue | |
| Status: **implemented**. | |
| The backend returns a `review_queue` sorted by low confidence, singleton status, review status, and suppression state. The UI renders the queue and lets the user select/audition items. | |
| Remaining gap: expected-impact ranking should eventually use feature margin and reconstruction contribution. | |
| ### FR-005: Confidence-weighted visual emphasis | |
| Status: **implemented**. | |
| Hit rows display confidence and flags. Low-confidence rows get emphasis; suppressed rows visually recede. | |
| ### FR-006: Click-to-add missed onset | |
| Status: **not implemented**. | |
| Current waveform click auditions the nearest existing hit only. | |
| Required next behavior: | |
| - Add `force-onset` constraint at selected time. | |
| - Slice candidate hit from cached `stem.wav`. | |
| - Classify and assign locally. | |
| - Store it as a forced hit in `supervision_state.json`. | |
| ### FR-007: Bleed brush / false-positive suppression | |
| Status: **implemented for selected hits; brush region not implemented**. | |
| Selected hits can be suppressed as bleed/noise. The system stores `suppress-pattern` and proposes similar suppressions. | |
| Remaining gap: region brush, restore, and supervised export exclusion. | |
| ### FR-008: Favorite/pin sample optimization | |
| Status: **partial**. | |
| Favorite action records `pin-representative` and updates the semantic representative. Exported WAV/ZIP does not yet change. | |
| ### FR-009: Explain cluster | |
| Status: **implemented**. | |
| Cluster explanation includes representative, hit counts, confidence reasons, label distribution, outliers, and relevant constraints. | |
| ### FR-010: Predictive batch questions | |
| Status: **partial**. | |
| Suggestions exist for move/split/suppress patterns and can be accepted/rejected. They do not yet show exact diff previews. | |
| ### FR-011: Live counterfactual parameter preview | |
| Status: **not implemented**. | |
| ### FR-012: Reconstruction-error correction | |
| Status: **not implemented**. | |
| ## Non-functional requirements and status | |
| ### NFR-001: Reversibility | |
| Status: **implemented for semantic edits** via undo stack. | |
| ### NFR-002: Explainability | |
| Status: **partial**. Events, constraints, confidence reasons, and cluster explanations are visible. Suggestion diff previews are not yet implemented. | |
| ### NFR-003: Local recomputation first | |
| Status: **partial**. Current recomputation is cheap semantic-state recomputation. True local feature reclustering remains open. | |
| ### NFR-004: Cached preprocessing | |
| Status: **partial**. Stems/source loads are cached and hit audio is exported. Feature-vector caching is still open. | |
| ### NFR-005: Deterministic replay | |
| Status: **partial**. Constraints/events persist and can be reloaded. A dedicated replay command/export pipeline is still open. | |
| ### NFR-006: No silent override of explicit user intent | |
| Status: **implemented in current semantic layer**. Explicit moves, locks, suppressions, and favorites persist unless undone or explicitly changed. | |
| ## Pass 5 implementation status | |
| Implemented after initial alignment: | |
| - supervised edited-state export under `supervised/`, | |
| - add-onset waveform mode backed by `POST /api/jobs/{job_id}/hits/force-onset`, | |
| - suppressed-hit restore backed by `POST /api/jobs/{job_id}/hits/{hit_id}/restore`, | |
| - exact suggestion diff previews in API state and UI, | |
| - validation via `scripts/test_supervised_export_and_force_onset.py`. | |
| Still open: | |
| - cluster merge/relabel/split workflows, | |
| - cached feature-vector local reclustering, | |
| - edited-vs-original comparison, | |
| - browser-level UI tests. | |