docs/UI_REPLACEMENT.md

Custom UI replacement

Last updated: 2026-05-12

What changed

The active interface is a custom browser UI served from web/ by the FastAPI app in app.py. The old Gradio files live in legacy/ and are no longer used by the active application.

UX goals

1. Match the supplied minimal reference image as the default view: file identity, one purple extract action, large waveform, compact right controls, and sample cards.
2. Keep the first screen focused on extraction and audition, not pipeline/debug/editor internals.
3. Preserve advanced workstation features behind a single secondary Review & edit workbench.
4. Keep stage timing, logs, run history, and semantic supervision available without dominating the default layout.
5. Make stem=all and online_preview available as advanced presets instead of primary controls.
6. Keep the frontend deployable without a JavaScript build step until the interaction model stabilizes.

Pass 6 visual simplification

The UI was first restyled to the supplied minimal reference direction:

• light native-feeling canvas instead of the previous dark dashboard;
• compact top bar with file selector/status and one primary purple Extract Samples action;
• large waveform-first workspace with colored lollipop onset markers;
• right-side control card for the primary extraction decisions;
• advanced DSP/model parameters moved into a collapsible panel;
• representative samples rendered as auditionable cards with waveform thumbnails;
• pipeline, run history, supervision, and raw tables kept available as collapsible utility panels.

Pass 7 reference-alignment hardening

This pass closed the visual fidelity gaps from the previous approximation:

• removed the visible waveform header so the canvas is quiet like the reference image;
• replaced separate native stem/reconstruction audio controls with one minimal transport row: play button, time, progress line, and Source/Stem/Reproduced preview modes;
• renamed the right card to Common controls and limited it to stem, hit sensitivity, sample groups, plus fast-preview/best-quality presets;
• collapsed pipeline/history/supervision/tables into one Review & edit workbench below the sample cards;
• hid selected-hit/sample audio elements from the default layout while preserving click-to-audition behavior;
• tightened card spacing, border radii, font scale, waveform height, and sample-card proportions to better match the supplied image.

The UI is still not a pixel-for-pixel clone because it must remain functional across arbitrary audio files and preserve the project’s editing tools, but the default screen is now intentionally aligned with the reference composition.

Pass 8 fixed workstation layout

This pass responds to the no-scroll workstation requirement and the missing-upload affordance:

• the document body is fixed to the viewport with overflow: hidden;
• the app uses a top bar, left sidebar, center workspace, right sidebar, and bottom dock;
• pipeline/history/selection tools moved into the left sidebar;
• extraction/export/advanced controls moved into the right sidebar;
• semantic review/edit tools and raw tables moved into the bottom bar;
• all long content now scrolls only inside its own panel;
• the upload affordance is now an explicit Upload audio button in the top bar;
• dropping a file anywhere on the app loads it and shows a full-screen drop overlay while dragging.

UI structure

Area	Purpose
Top bar	App identity, explicit upload button, selected-file metadata, backend status, and one primary purple Extract Samples action.
Left sidebar	Source/drop guidance, selected-hit/sample context, pipeline logs, and run history.
Center workspace	Quiet waveform canvas, Source/Stem/Reproduced transport row, and representative sample cards.
Right sidebar	Common controls, exports, and collapsed advanced parameters grouped by stem separation, hit detection, grouping, export, and cache.
Bottom dock	Review/edit semantic supervision tools and raw tables in expandable panels.

Frontend implementation

Files:

• web/index.html
• web/styles.css
• web/app.js

The frontend uses modern browser APIs directly:

• fetch for API calls.
• FormData for upload.
• <audio> for previews.
• <canvas> for waveform/onset visualization.
• CSS grid, responsive layout, custom properties, and backdrop filters for layout/polish.

No Gradio runtime, iframe, or generated UI framework is involved.

Backend integration

The frontend creates a job with POST /api/jobs, then polls GET /api/jobs/{id} until completion. Completed jobs expose direct download URLs for:

• sample pack ZIP
• MIDI reconstruction
• source mix WAV
• target stem WAV
• non-target context bed WAV
• target reconstruction WAV
• full-context reproduced mix WAV
• individual sample WAVs

The run history panel calls GET /api/jobs and can reload any completed manifest still present under .runs/.

Clustering UX

Two modes are exposed:

Mode	UX intent
batch_quality	Slower, final-quality clustering using all-pairs similarity plus agglomerative clustering.
online_preview	Faster near-realtime-style clustering using prototype assignment. Best for quick iteration after bypassing Demucs.

Why SSE progress with polling fallback instead of websockets

The active UI uses Server-Sent Events through GET /api/jobs/{job_id}/events for stage/log updates, with the older polling loop retained as a fallback. WebSockets are unnecessary here because the pipeline is stage-oriented and the frontend does not need bidirectional streaming while extraction runs.

Remaining UI improvements

• Add waveform zoom/pan while keeping the fixed workstation layout uncluttered.
• Add inline cluster merge/split/relabel workflows inside Review & edit.
• Add A/B comparison between parameter runs.
• Add downloadable timing report per job.
• Add filters/search to the run history browser.
• Convert the frontend to TypeScript when the UX stops moving quickly.

Latest review UI additions

The current UI now includes:

• Hidden selected-hit and selected-sample audio elements for click-to-audition without visible player clutter.
• Clickable waveform onset markers that select and audition the nearest detected hit.
• A detected-hit review table backed by review/hits/*.wav artifacts.
• Audition buttons for representative sample rows.
• Server-sent-events job progress via GET /api/jobs/{job_id}/events, with polling fallback.

This still stops short of destructive editing. The next UI layer should store edits as manifest overlays, then call a re-export endpoint that reuses cached hit audio instead of rerunning Demucs/onset detection.

Pass 9 reproduced audio and parameter hierarchy

This pass made the audio preview and control model more explicit:

• reconstruction.wav is now a full-context reproduced mix, not just the sample-triggered target layer.
• target_reconstruction.wav preserves the sample-only target reconstruction for focused inspection.
• source.wav, stem.wav, and context_bed.wav are exported as explicit layers.
• The transport has Source, Stem, and Reproduced preview buttons and switches to Reproduced after extraction.
• The right sidebar now separates Common controls from Advanced parameters.
• Advanced parameters are grouped by pipeline stage: stem separation, hit detection, grouping, export/cache.

See docs/REPRODUCED_AUDIO_AND_PARAMETERS.md.

Pass 10 clean-default refinement

This pass responds to the UI being too cluttered after the workstation and reproduced-audio additions.

Changes:

• all non-essential side/bottom panels are collapsed by default;
• the bottom dock is compact until a review/table tool is opened;
• top-bar controls are shorter and visually lighter;
• common controls hide explanatory copy and show only the core knobs;
• pipeline logs, run history, exports, supervision, and raw tables remain present but opt-in;
• the no-scroll workstation constraint is preserved.

See docs/CLEAN_DEFAULT_UI.md.

Immediate waveform and extraction progress update

The default workflow now starts with visible feedback: selecting or dropping a file causes the browser to decode the uploaded audio and render a peak-envelope waveform immediately. This happens before /api/jobs is called. The user can play the uploaded source through the transport while choosing common controls.

During extraction, the waveform is reused as the progress surface. The backend sends a real progress object through the normal job payload and SSE stream. The frontend clips an accent-colored copy of the waveform from the left to exactly that progress fraction and leaves the remaining waveform neutral. No estimated time progress is animated.

A compact Start here panel was added to the left sidebar to make the sequence obvious: Load audio → Set controls → Extract → Review/export.