docs/UI_REPLACEMENT.md

# Custom UI replacement

Last updated: 2026-05-12

## What changed

The active interface is a custom browser UI served from `web/` by the FastAPI app in `app.py`. The old Gradio files live in `legacy/` and are no longer used by the active application.

## UX goals

1. Match the supplied minimal reference image as the default view: file identity, one purple extract action, large waveform, compact right controls, and sample cards.
2. Keep the first screen focused on extraction and audition, not pipeline/debug/editor internals.
3. Preserve advanced workstation features behind a single secondary `Review & edit` workbench.
4. Keep stage timing, logs, run history, and semantic supervision available without dominating the default layout.
5. Make `stem=all` and `online_preview` available as advanced presets instead of primary controls.
6. Keep the frontend deployable without a JavaScript build step until the interaction model stabilizes.


## Pass 6 visual simplification

The UI was first restyled to the supplied minimal reference direction:

- light native-feeling canvas instead of the previous dark dashboard;
- compact top bar with file selector/status and one primary purple `Extract Samples` action;
- large waveform-first workspace with colored lollipop onset markers;
- right-side control card for the primary extraction decisions;
- advanced DSP/model parameters moved into a collapsible panel;
- representative samples rendered as auditionable cards with waveform thumbnails;
- pipeline, run history, supervision, and raw tables kept available as collapsible utility panels.

## Pass 7 reference-alignment hardening

This pass closed the visual fidelity gaps from the previous approximation:

- removed the visible waveform header so the canvas is quiet like the reference image;
- replaced separate native stem/reconstruction audio controls with one minimal transport row: play button, time, progress line, and Source/Stem/Reproduced preview modes;
- renamed the right card to `Common controls` and limited it to stem, hit sensitivity, sample groups, plus fast-preview/best-quality presets;
- collapsed pipeline/history/supervision/tables into one `Review & edit` workbench below the sample cards;
- hid selected-hit/sample audio elements from the default layout while preserving click-to-audition behavior;
- tightened card spacing, border radii, font scale, waveform height, and sample-card proportions to better match the supplied image.

The UI is still not a pixel-for-pixel clone because it must remain functional across arbitrary audio files and preserve the project’s editing tools, but the default screen is now intentionally aligned with the reference composition.


## Pass 8 fixed workstation layout

This pass responds to the no-scroll workstation requirement and the missing-upload affordance:

- the document body is fixed to the viewport with `overflow: hidden`;
- the app uses a top bar, left sidebar, center workspace, right sidebar, and bottom dock;
- pipeline/history/selection tools moved into the left sidebar;
- extraction/export/advanced controls moved into the right sidebar;
- semantic review/edit tools and raw tables moved into the bottom bar;
- all long content now scrolls only inside its own panel;
- the upload affordance is now an explicit `Upload audio` button in the top bar;
- dropping a file anywhere on the app loads it and shows a full-screen drop overlay while dragging.

## UI structure

| Area | Purpose |
|---|---|
| Top bar | App identity, explicit upload button, selected-file metadata, backend status, and one primary purple `Extract Samples` action. |
| Left sidebar | Source/drop guidance, selected-hit/sample context, pipeline logs, and run history. |
| Center workspace | Quiet waveform canvas, Source/Stem/Reproduced transport row, and representative sample cards. |
| Right sidebar | Common controls, exports, and collapsed advanced parameters grouped by stem separation, hit detection, grouping, export, and cache. |
| Bottom dock | Review/edit semantic supervision tools and raw tables in expandable panels. |

## Frontend implementation

Files:

- `web/index.html`
- `web/styles.css`
- `web/app.js`

The frontend uses modern browser APIs directly:

- `fetch` for API calls.
- `FormData` for upload.
- `<audio>` for previews.
- `<canvas>` for waveform/onset visualization.
- CSS grid, responsive layout, custom properties, and backdrop filters for layout/polish.

No Gradio runtime, iframe, or generated UI framework is involved.

## Backend integration

The frontend creates a job with `POST /api/jobs`, then polls `GET /api/jobs/{id}` until completion. Completed jobs expose direct download URLs for:

- sample pack ZIP
- MIDI reconstruction
- source mix WAV
- target stem WAV
- non-target context bed WAV
- target reconstruction WAV
- full-context reproduced mix WAV
- individual sample WAVs

The run history panel calls `GET /api/jobs` and can reload any completed manifest still present under `.runs/`.

## Clustering UX

Two modes are exposed:

| Mode | UX intent |
|---|---|
| `batch_quality` | Slower, final-quality clustering using all-pairs similarity plus agglomerative clustering. |
| `online_preview` | Faster near-realtime-style clustering using prototype assignment. Best for quick iteration after bypassing Demucs. |

## Why SSE progress with polling fallback instead of websockets

The active UI uses Server-Sent Events through `GET /api/jobs/{job_id}/events` for stage/log updates, with the older polling loop retained as a fallback. WebSockets are unnecessary here because the pipeline is stage-oriented and the frontend does not need bidirectional streaming while extraction runs.

## Remaining UI improvements

- Add waveform zoom/pan while keeping the fixed workstation layout uncluttered.
- Add inline cluster merge/split/relabel workflows inside `Review & edit`.
- Add A/B comparison between parameter runs.
- Add downloadable timing report per job.
- Add filters/search to the run history browser.
- Convert the frontend to TypeScript when the UX stops moving quickly.

## Latest review UI additions

The current UI now includes:

- Hidden selected-hit and selected-sample audio elements for click-to-audition without visible player clutter.
- Clickable waveform onset markers that select and audition the nearest detected hit.
- A detected-hit review table backed by `review/hits/*.wav` artifacts.
- Audition buttons for representative sample rows.
- Server-sent-events job progress via `GET /api/jobs/{job_id}/events`, with polling fallback.

This still stops short of destructive editing. The next UI layer should store edits as manifest overlays, then call a re-export endpoint that reuses cached hit audio instead of rerunning Demucs/onset detection.


## Pass 9 reproduced audio and parameter hierarchy

This pass made the audio preview and control model more explicit:

- `reconstruction.wav` is now a full-context reproduced mix, not just the sample-triggered target layer.
- `target_reconstruction.wav` preserves the sample-only target reconstruction for focused inspection.
- `source.wav`, `stem.wav`, and `context_bed.wav` are exported as explicit layers.
- The transport has Source, Stem, and Reproduced preview buttons and switches to Reproduced after extraction.
- The right sidebar now separates Common controls from Advanced parameters.
- Advanced parameters are grouped by pipeline stage: stem separation, hit detection, grouping, export/cache.

See `docs/REPRODUCED_AUDIO_AND_PARAMETERS.md`.

## Pass 10 clean-default refinement

This pass responds to the UI being too cluttered after the workstation and reproduced-audio additions.

Changes:

- all non-essential side/bottom panels are collapsed by default;
- the bottom dock is compact until a review/table tool is opened;
- top-bar controls are shorter and visually lighter;
- common controls hide explanatory copy and show only the core knobs;
- pipeline logs, run history, exports, supervision, and raw tables remain present but opt-in;
- the no-scroll workstation constraint is preserved.

See `docs/CLEAN_DEFAULT_UI.md`.


## Immediate waveform and extraction progress update

The default workflow now starts with visible feedback: selecting or dropping a file causes the browser to decode the uploaded audio and render a peak-envelope waveform immediately. This happens before `/api/jobs` is called. The user can play the uploaded source through the transport while choosing common controls.

During extraction, the waveform is reused as the progress surface. The backend sends a real `progress` object through the normal job payload and SSE stream. The frontend clips an accent-colored copy of the waveform from the left to exactly that progress fraction and leaves the remaining waveform neutral. No estimated time progress is animated.

A compact `Start here` panel was added to the left sidebar to make the sequence obvious: Load audio → Set controls → Extract → Review/export.