docs/interactive-ux/PROGRESS.md

# Interactive UX progress

## Last updated

2026-05-12

## Current phase

Implementation is now in **Phase 1–3 foundation**: persistent state, constraints, events, confidence/review queue, and supervised cluster interactions are implemented at the semantic-state layer.

## Completed in this pass

| Item | Status | Notes |
|---|---|---|
| Add supplied docs to `docs/interactive-ux/` | done | All supplied Markdown docs were copied into this directory and aligned with the implemented project. |
| Persistent job state | done | `supervision_state.json` is created beside `manifest.json` for each completed run. |
| Hit/cluster state schema | done | Implemented in `supervised_state.py` using JSON-serializable dictionaries. |
| Event log | done | State mutations append `job.state.created`, `constraint.created`, `hit.moved`, `hit.pulled_out`, `cluster.locked`, `hit.suppressed`, `suggestion.created`, etc. |
| Constraint store | done | Supports `force-cluster`, `must-link`, `cannot-link`, `lock-cluster`, `suppress-pattern`, and `pin-representative`. |
| Confidence scoring | partial | Heuristic scores based on cluster size, label agreement, energy rank, representative/favorite/explicit state, suppression, and lock state. Feature-vector margin scoring is not implemented yet. |
| Outlier-first review queue | done | Backend computes `review_queue`; UI renders it and lets the user jump to the selected hit. |
| Move hit to cluster | done | Endpoint creates constraints and updates state; it also proposes similar move suggestions. |
| Pull hit into new cluster | done | Endpoint creates `cannot-link` and `force-cluster` constraints and a user cluster. |
| Lock cluster | done | Endpoint toggles lock state and records a lock constraint. |
| Suppress hit as bleed/noise | done | Endpoint creates `suppress-pattern`, marks the hit suppressed, and proposes similar suppressions. |
| Favorite sample / pin representative | partial | Endpoint supports `review` status `favorite`, records `pin-representative`, and updates the representative hit in semantic state. Audio artifact selection is not re-exported yet. |
| Suggestion inbox | partial | Open suggestions render in the UI and can be accepted/rejected. Suggestion generation is heuristic and limited to move/split/suppress patterns. |
| Cluster explanation drawer | done | Endpoint and UI show representative, confidence reasons, outliers, relevant constraints, and label distribution. |
| Undo | done | Last semantic edit can be restored using an undo snapshot stack. |
| Validation script | done | Added `scripts/test_interactive_supervision.py`. |

## Not yet implemented

- Real cached feature-vector store for local reclustering.
- Artifact re-export after semantic edits.
- Waveform click-to-add missed onset.
- Restore suppressed hit/batch restore.
- Real local neighborhood reclustering that changes assignments beyond explicit move/suggestion acceptance.
- Constraint violation detection and reporting.
- Predictive diff preview before accepting suggestions.
- Reconstruction-error-driven correction.
- Multi-resolution/hierarchical clusters.
- User correction profiles / teach mode across songs.
- Frontend TypeScript migration and browser automation tests.

## Current risks

| Risk | Impact | Mitigation |
|---|---|---|
| Semantic edits do not rewrite exports yet | User may expect moved/suppressed hits to affect ZIP/MIDI immediately | Next task should be edited-state export. |
| Confidence scores are heuristic | Review queue may sometimes prioritize the wrong hits | Add cached mel/transient features and margin-to-next-cluster scoring. |
| Suggestions are simple | May over-suggest or under-suggest | Keep them previewable, explicit, and undoable; never silently apply. |
| Locks are semantic only | Batch reruns do not yet replay constraints | Add deterministic replay/local recluster using constraints. |
| No browser tests | UI regressions are easy | Add Playwright or lightweight DOM tests. |

## Next implementation milestone

Milestone: **edited-state export and force-onset correction**.

Minimum deliverables:

1. Add `POST /api/jobs/{job_id}/export/supervised` that creates a ZIP/MIDI/manifest from `supervision_state.json`.
2. Exclude suppressed hits/clusters from the supervised export.
3. Honor favorite/pinned representatives in exported samples.
4. Add force-onset endpoint that slices a new hit from cached `stem.wav`.
5. Add waveform shift-click or add-onset mode in the UI.
6. Add tests proving semantic edits change the supervised export without rerunning stem extraction.

## Definition of done for the current foundation

This loop now works:

```text
analyze audio
→ inspect clusters
→ load semantic state
→ move one wrong hit
→ store constraints/events
→ see confidence/review queue update
→ lock corrected cluster
→ suppress bleed
→ inspect explanations and suggestions
→ undo semantic edits
→ reload the job and preserve explicit decisions
```

The remaining missing piece is that edited semantic state is not yet reflected in a regenerated sample pack.


## Pass 5 alignment

The interactive UX docs are now aligned with the implemented semantic edit/export loop. The project supports move, pull-out, suppress, restore, favorite, lock, force-onset, suggestion diff preview, undo, and edited artifact export. The current boundary is no longer “semantic only”; edits can now produce separate supervised WAV/MIDI/reconstruction/ZIP artifacts while original batch outputs remain immutable.

Remaining UX work is concentrated around cluster-level editing and comparison: merge/relabel/split, feature-vector local reclustering, edited-vs-original diff views, and browser tests.

## 2026-05-12 automatic card-flow update

The default UX now follows the interactive-doc direction more closely by hiding most expert controls and making the user action model concrete:

- drop/upload starts processing automatically;
- waveform appears before backend processing finishes;
- real backend progress tints the waveform;
- sample candidates appear as cards grouped by type;
- dismissing a card becomes a supervised suppression when state is available;
- drawing another candidate models the "draw a card" interaction for missing samples;
- trim/extend controls can save adjusted timing as a forced hit;
- waveform zoom/pan supports close inspection without leaving the main flow.

The remaining mismatch is that drawn candidate cards are still frontend candidate previews, not persisted representative-selection constraints. That should be promoted into the semantic state model next.


## Pass 14: selected cards and Spleeter backend

Completed in this pass:

1. Added `spleeter` as the default separation backend, with selectable `spleeter:2stems`, `spleeter:4stems`, and `spleeter:5stems` profiles.
2. Kept `demucs` as a quality/fallback backend and `none` as the full-mix preview backend.
3. Added optional `requirements-spleeter.txt` instead of forcing TensorFlow/Spleeter into the base install.
4. Added per-card checkbox state with manual select-all/clear behavior.
5. Added selected-only backend export via `POST /api/jobs/{job_id}/export-selected`.
6. Made `draw another` persist the chosen representative in semantic state.
7. Made trim/extend rewrite playable preview audio immediately under `overrides/hits/`.
8. Added `scripts/test_selected_export_card_actions.py`.

Outcome:

The default app now behaves more like a card review tool: drop audio, let Spleeter/fallback separation run, review grouped cards, select/dismiss/draw/trim, and export only the selected pack.