docs/PROJECT_REVIEW.md

Project review

Last updated: 2026-05-12

Summary

The project has evolved from a Gradio-driven prototype into a usable FastAPI + custom frontend extraction workstation. The core DSP pipeline is still compact and script-oriented, but it now has a clearer boundary between API/UI orchestration (app.py), timed pipeline execution (pipeline_runner.py), and lower-level sample extraction (sample_extractor.py).

What is strong

1. Useful core pipeline: stem extraction, onset detection, classification, clustering, selection, synthesis, MIDI rendering, and packaging are all present.
2. Small deployable surface: active runtime is FastAPI plus static files; no frontend build is required.
3. Good local iteration path: stem=all bypasses Demucs for fast tuning.
4. Per-stage timing: every job manifest records stage durations and details.
5. Artifacts are explicit: stem WAV, reconstruction WAV, MIDI, sample WAVs, ZIP, and manifest are written per run.
6. Legacy preservation: old Gradio apps remain available under legacy/ without being active.
7. New near-realtime path: online_preview clustering gives a practical alternative to all-pairs batch clustering.

Main risks

1. Interactive editing is missing: users can inspect outputs but cannot correct onsets or cluster decisions in the UI yet.
2. Job state is process-local: active jobs disappear from memory on restart; completed history is recovered from manifests only.
3. Progress is stage-level: Demucs and clustering do not expose fine-grained progress.
4. Frontend is plain JavaScript: good for speed, weaker for long-term maintainability than TypeScript modules/tests.
5. Demucs cost remains dominant: source separation is necessarily offline; disk cache mitigates repeated runs but not first-run latency.
6. DSP code is dense: sample_extractor.py is effective but would benefit from smaller modules and stronger tests.

Development decisions made

Decision	Rationale
Replace Gradio with FastAPI/static UI	More control over workflow, layout, artifacts, and progress display.
Keep no-build frontend for now	Fastest robust replacement; avoids adding Node/Vite just to ship the first custom UI.
Preserve Gradio in legacy/	Avoids data loss and gives reference behavior.
Add pipeline_runner.py	Keeps API orchestration separate from DSP primitives.
Add disk cache in pipeline layer	Avoids invasive Demucs changes and caches both full mix and stems.
Add online_preview rather than replacing batch clustering	Preserves final-quality path while adding a near-realtime option.

Current implementation quality

Area	Rating	Notes
Extraction functionality	Good	Core path works on synthetic tests.
UI/UX foundation	Good	Custom flow is much better than generic Gradio controls.
Realtime architecture	Partial	Online clustering exists; streaming onset/audio pipeline does not.
Documentation	Good	Feature/task/progress/API/timing docs are now embedded.
Test coverage	Basic	Smoke tests exist; no formal unit/browser tests yet.
Maintainability	Medium	Better boundaries now, but DSP module remains dense.

Recommendation

Next development should not add more global parameters. It should add an editing loop: audition detected hits, manually fix bad onsets, merge/split clusters, relabel samples, then repack from edited state without rerunning expensive stages.