Spaces:
Sleeping
Sleeping
ChatGPT commited on
Commit ·
03d531b
1
Parent(s): 3703c4e
feat: add supervised interactive editing state
Browse files- README.md +59 -13
- app.py +164 -2
- docs/API.md +167 -0
- docs/FEATURES.md +22 -0
- docs/PROGRESS.md +64 -0
- docs/REMAINING_WORK.md +21 -0
- docs/TASKS.md +29 -0
- docs/interactive-ux/ARCHITECTURE_NOTES.md +204 -0
- docs/interactive-ux/FEASIBILITY_MATRIX.md +100 -0
- docs/interactive-ux/FEATURE_REQUIREMENTS.md +162 -0
- docs/interactive-ux/PROGRESS.md +87 -0
- docs/interactive-ux/README.md +59 -0
- docs/interactive-ux/SCOPE.md +173 -0
- docs/interactive-ux/TASKS.md +105 -0
- scripts/test_interactive_supervision.py +112 -0
- supervised_state.py +675 -0
- web/app.js +232 -16
- web/index.html +46 -1
- web/styles.css +21 -0
README.md
CHANGED
|
@@ -10,18 +10,18 @@ pinned: false
|
|
| 10 |
|
| 11 |
# Drum Sample Extractor
|
| 12 |
|
| 13 |
-
A custom FastAPI + browser workstation for extracting reusable drum samples from an audio file.
|
| 14 |
|
| 15 |
-
The pipeline can isolate a stem with Demucs, detect onsets, classify hits, cluster similar transients, choose representative samples, optionally synthesize alternate samples, and export WAVs, MIDI, reconstruction audio, manifests, and a complete ZIP sample pack.
|
| 16 |
|
| 17 |
## Current status
|
| 18 |
|
| 19 |
The project is usable as a local/Hugging Face Space application. Gradio is no longer the active UI; the active app is a custom FastAPI backend plus a no-build browser frontend.
|
| 20 |
|
| 21 |
-
Implemented
|
| 22 |
|
| 23 |
- Custom web frontend in `web/`, served by `app.py`.
|
| 24 |
-
- FastAPI job API with upload, polling, safe artifact downloads, config, health, cache clearing,
|
| 25 |
- Timed pipeline runner in `pipeline_runner.py`.
|
| 26 |
- Per-stage timing in every `manifest.json`.
|
| 27 |
- Two clustering modes:
|
|
@@ -31,15 +31,34 @@ Implemented in the current development pass:
|
|
| 31 |
- Run history panel indexing `.runs/*/output/manifest.json`.
|
| 32 |
- Individual review WAVs for every detected hit under `review/hits/`.
|
| 33 |
- Click-to-audition workflow for waveform onsets, detected hit rows, and representative sample rows.
|
| 34 |
-
-
|
| 35 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
- Legacy Gradio apps preserved in `legacy/` for reference only.
|
| 37 |
|
| 38 |
Not fully complete yet:
|
| 39 |
|
| 40 |
-
-
|
| 41 |
-
- No
|
| 42 |
-
- No
|
|
|
|
|
|
|
| 43 |
- Demucs remains offline/batch by design.
|
| 44 |
|
| 45 |
See:
|
|
@@ -47,7 +66,8 @@ See:
|
|
| 47 |
- `docs/FEATURES.md`
|
| 48 |
- `docs/TASKS.md`
|
| 49 |
- `docs/PROGRESS.md`
|
| 50 |
-
- `docs/
|
|
|
|
| 51 |
- `docs/REMAINING_WORK.md`
|
| 52 |
|
| 53 |
## Run locally
|
|
@@ -68,11 +88,19 @@ For fast iteration, set:
|
|
| 68 |
|
| 69 |
That bypasses Demucs and uses the near-realtime clustering path.
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
## Run benchmarks
|
| 72 |
|
| 73 |
```bash
|
| 74 |
python3 scripts/benchmark_subprocesses.py --runs 2 --bars 4 --output docs/benchmark-subprocesses.json
|
| 75 |
-
python3 scripts/test_sse_and_review_hits.py
|
| 76 |
```
|
| 77 |
|
| 78 |
The benchmark uses synthetic drum fixtures and `stem=all` so the DSP stages are measured without Demucs model download/runtime noise.
|
|
@@ -93,6 +121,20 @@ Then poll the returned job id:
|
|
| 93 |
curl http://127.0.0.1:7860/api/jobs/<job-id>
|
| 94 |
```
|
| 95 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
List active/completed runs:
|
| 97 |
|
| 98 |
```bash
|
|
@@ -103,11 +145,14 @@ curl http://127.0.0.1:7860/api/jobs
|
|
| 103 |
|
| 104 |
| Path | Purpose |
|
| 105 |
|---|---|
|
| 106 |
-
| `app.py` | FastAPI app, static UI serving, job API, run history, artifact downloads |
|
| 107 |
| `pipeline_runner.py` | Timed extraction pipeline, disk stem/source cache, batch/online clustering routing |
|
| 108 |
| `sample_extractor.py` | Core DSP/sample extraction implementation |
|
| 109 |
-
| `
|
|
|
|
| 110 |
| `scripts/benchmark_subprocesses.py` | Synthetic benchmark runner for stage timings |
|
|
|
|
|
|
|
| 111 |
| `docs/` | Review, timing, API, UI, feature, task, progress, and remaining-work documentation |
|
| 112 |
| `legacy/` | Previous Gradio apps retained for reference |
|
| 113 |
|
|
@@ -122,6 +167,7 @@ Each run is stored under `.runs/<job-id>/output/`:
|
|
| 122 |
- `samples/*.wav`
|
| 123 |
- `review/hits/*.wav`
|
| 124 |
- `manifest.json`
|
|
|
|
| 125 |
|
| 126 |
Generated runtime directories are ignored by git:
|
| 127 |
|
|
|
|
| 10 |
|
| 11 |
# Drum Sample Extractor
|
| 12 |
|
| 13 |
+
A custom FastAPI + browser workstation for extracting, reviewing, and now semantically supervising reusable drum samples from an audio file.
|
| 14 |
|
| 15 |
+
The pipeline can isolate a stem with Demucs, detect onsets, classify hits, cluster similar transients, choose representative samples, optionally synthesize alternate samples, and export WAVs, MIDI, reconstruction audio, manifests, and a complete ZIP sample pack. The interactive layer stores user corrections as replayable semantic state beside each run manifest.
|
| 16 |
|
| 17 |
## Current status
|
| 18 |
|
| 19 |
The project is usable as a local/Hugging Face Space application. Gradio is no longer the active UI; the active app is a custom FastAPI backend plus a no-build browser frontend.
|
| 20 |
|
| 21 |
+
Implemented:
|
| 22 |
|
| 23 |
- Custom web frontend in `web/`, served by `app.py`.
|
| 24 |
+
- FastAPI job API with upload, polling, safe artifact downloads, config, health, cache clearing, run history, and SSE progress.
|
| 25 |
- Timed pipeline runner in `pipeline_runner.py`.
|
| 26 |
- Per-stage timing in every `manifest.json`.
|
| 27 |
- Two clustering modes:
|
|
|
|
| 31 |
- Run history panel indexing `.runs/*/output/manifest.json`.
|
| 32 |
- Individual review WAVs for every detected hit under `review/hits/`.
|
| 33 |
- Click-to-audition workflow for waveform onsets, detected hit rows, and representative sample rows.
|
| 34 |
+
- Interactive supervised state in `supervised_state.py`:
|
| 35 |
+
- persisted `supervision_state.json`,
|
| 36 |
+
- hit/cluster confidence,
|
| 37 |
+
- outlier-first review queue,
|
| 38 |
+
- constraints,
|
| 39 |
+
- event log,
|
| 40 |
+
- suggestions,
|
| 41 |
+
- undo stack.
|
| 42 |
+
- Supervision UI:
|
| 43 |
+
- selected-hit actions,
|
| 44 |
+
- move hit to cluster,
|
| 45 |
+
- pull hit into a new cluster,
|
| 46 |
+
- accept/favorite hit,
|
| 47 |
+
- suppress hit as bleed,
|
| 48 |
+
- lock/unlock cluster,
|
| 49 |
+
- suggestion inbox,
|
| 50 |
+
- cluster explanation drawer,
|
| 51 |
+
- constraint/event log.
|
| 52 |
+
- Documentation for features, progress, tasks, API, timing, hit review, realtime suitability, UI, remaining work, and interactive UX.
|
| 53 |
- Legacy Gradio apps preserved in `legacy/` for reference only.
|
| 54 |
|
| 55 |
Not fully complete yet:
|
| 56 |
|
| 57 |
+
- Semantic edits do not yet regenerate WAV/MIDI/ZIP exports.
|
| 58 |
+
- No force-onset/click-to-add missed onset yet.
|
| 59 |
+
- No restore for suppressed hits yet.
|
| 60 |
+
- No true cached feature-vector local reclustering yet.
|
| 61 |
+
- No frontend TypeScript build/test harness yet.
|
| 62 |
- Demucs remains offline/batch by design.
|
| 63 |
|
| 64 |
See:
|
|
|
|
| 66 |
- `docs/FEATURES.md`
|
| 67 |
- `docs/TASKS.md`
|
| 68 |
- `docs/PROGRESS.md`
|
| 69 |
+
- `docs/API.md`
|
| 70 |
+
- `docs/interactive-ux/README.md`
|
| 71 |
- `docs/REMAINING_WORK.md`
|
| 72 |
|
| 73 |
## Run locally
|
|
|
|
| 88 |
|
| 89 |
That bypasses Demucs and uses the near-realtime clustering path.
|
| 90 |
|
| 91 |
+
## Run checks
|
| 92 |
+
|
| 93 |
+
```bash
|
| 94 |
+
python3 -m py_compile app.py pipeline_runner.py sample_extractor.py supervised_state.py scripts/*.py
|
| 95 |
+
node --check web/app.js
|
| 96 |
+
python3 scripts/test_sse_and_review_hits.py
|
| 97 |
+
python3 scripts/test_interactive_supervision.py
|
| 98 |
+
```
|
| 99 |
+
|
| 100 |
## Run benchmarks
|
| 101 |
|
| 102 |
```bash
|
| 103 |
python3 scripts/benchmark_subprocesses.py --runs 2 --bars 4 --output docs/benchmark-subprocesses.json
|
|
|
|
| 104 |
```
|
| 105 |
|
| 106 |
The benchmark uses synthetic drum fixtures and `stem=all` so the DSP stages are measured without Demucs model download/runtime noise.
|
|
|
|
| 121 |
curl http://127.0.0.1:7860/api/jobs/<job-id>
|
| 122 |
```
|
| 123 |
|
| 124 |
+
Read supervised state:
|
| 125 |
+
|
| 126 |
+
```bash
|
| 127 |
+
curl http://127.0.0.1:7860/api/jobs/<job-id>/state
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
Move a hit into a target cluster:
|
| 131 |
+
|
| 132 |
+
```bash
|
| 133 |
+
curl -X POST http://127.0.0.1:7860/api/jobs/<job-id>/hits/hit%3A00003/move \
|
| 134 |
+
-H 'Content-Type: application/json' \
|
| 135 |
+
-d '{"target_cluster_id":"cluster:0"}'
|
| 136 |
+
```
|
| 137 |
+
|
| 138 |
List active/completed runs:
|
| 139 |
|
| 140 |
```bash
|
|
|
|
| 145 |
|
| 146 |
| Path | Purpose |
|
| 147 |
|---|---|
|
| 148 |
+
| `app.py` | FastAPI app, static UI serving, job API, run history, artifact downloads, supervised editing endpoints |
|
| 149 |
| `pipeline_runner.py` | Timed extraction pipeline, disk stem/source cache, batch/online clustering routing |
|
| 150 |
| `sample_extractor.py` | Core DSP/sample extraction implementation |
|
| 151 |
+
| `supervised_state.py` | Persistent semantic state, confidence, constraints, events, suggestions, undo |
|
| 152 |
+
| `web/` | Custom no-build browser frontend with waveform, hit review, sample audition, and supervision panel |
|
| 153 |
| `scripts/benchmark_subprocesses.py` | Synthetic benchmark runner for stage timings |
|
| 154 |
+
| `scripts/test_interactive_supervision.py` | Smoke test for supervised state endpoints |
|
| 155 |
+
| `docs/interactive-ux/` | Supplied interactive UX docs aligned to current implementation |
|
| 156 |
| `docs/` | Review, timing, API, UI, feature, task, progress, and remaining-work documentation |
|
| 157 |
| `legacy/` | Previous Gradio apps retained for reference |
|
| 158 |
|
|
|
|
| 167 |
- `samples/*.wav`
|
| 168 |
- `review/hits/*.wav`
|
| 169 |
- `manifest.json`
|
| 170 |
+
- `supervision_state.json`
|
| 171 |
|
| 172 |
Generated runtime directories are ignored by git:
|
| 173 |
|
app.py
CHANGED
|
@@ -19,20 +19,33 @@ from pathlib import Path
|
|
| 19 |
from threading import Lock
|
| 20 |
from typing import Any
|
| 21 |
|
| 22 |
-
from fastapi import FastAPI, File, Form, HTTPException, UploadFile
|
| 23 |
from fastapi.middleware.cors import CORSMiddleware
|
| 24 |
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
|
| 25 |
from fastapi.staticfiles import StaticFiles
|
| 26 |
|
| 27 |
from pipeline_runner import PipelineParams, clear_disk_cache, initial_stages, run_extraction_pipeline
|
| 28 |
from sample_extractor import DEMUCS_MODELS, DEMUCS_STEMS, cache_clear
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
ROOT = Path(__file__).resolve().parent
|
| 31 |
WEB_DIR = ROOT / "web"
|
| 32 |
RUNS_DIR = ROOT / ".runs"
|
| 33 |
RUNS_DIR.mkdir(exist_ok=True)
|
| 34 |
|
| 35 |
-
app = FastAPI(title="Drum Sample Extractor", version="
|
| 36 |
app.add_middleware(
|
| 37 |
CORSMiddleware,
|
| 38 |
allow_origins=["*"],
|
|
@@ -155,6 +168,7 @@ def _run_job(job_id: str) -> None:
|
|
| 155 |
|
| 156 |
try:
|
| 157 |
result = run_extraction_pipeline(input_path, output_dir, PipelineParams.from_mapping(params), progress_cb=progress)
|
|
|
|
| 158 |
_update_job(job_id, status="complete", result=asdict(result), error=None)
|
| 159 |
except Exception as exc: # deliberately explicit for UI diagnostics
|
| 160 |
_update_job(job_id, status="error", error=str(exc), traceback=traceback.format_exc())
|
|
@@ -248,6 +262,33 @@ def get_job(job_id: str) -> dict[str, Any]:
|
|
| 248 |
raise HTTPException(status_code=404, detail="Job not found")
|
| 249 |
|
| 250 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 251 |
@app.get("/api/jobs/{job_id}/events")
|
| 252 |
def get_job_events(job_id: str) -> StreamingResponse:
|
| 253 |
with jobs_lock:
|
|
@@ -286,6 +327,127 @@ def get_job_events(job_id: str) -> StreamingResponse:
|
|
| 286 |
)
|
| 287 |
|
| 288 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 289 |
@app.get("/api/jobs/{job_id}/files/{relative_path:path}")
|
| 290 |
def get_job_file(job_id: str, relative_path: str) -> FileResponse:
|
| 291 |
root = (RUNS_DIR / job_id / "output").resolve()
|
|
|
|
| 19 |
from threading import Lock
|
| 20 |
from typing import Any
|
| 21 |
|
| 22 |
+
from fastapi import Body, FastAPI, File, Form, HTTPException, UploadFile
|
| 23 |
from fastapi.middleware.cors import CORSMiddleware
|
| 24 |
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
|
| 25 |
from fastapi.staticfiles import StaticFiles
|
| 26 |
|
| 27 |
from pipeline_runner import PipelineParams, clear_disk_cache, initial_stages, run_extraction_pipeline
|
| 28 |
from sample_extractor import DEMUCS_MODELS, DEMUCS_STEMS, cache_clear
|
| 29 |
+
from supervised_state import (
|
| 30 |
+
accept_suggestion,
|
| 31 |
+
explain_cluster as build_cluster_explanation,
|
| 32 |
+
load_or_create_state,
|
| 33 |
+
lock_cluster as apply_cluster_lock,
|
| 34 |
+
move_hit as apply_hit_move,
|
| 35 |
+
public_state,
|
| 36 |
+
pull_hit_to_new_cluster,
|
| 37 |
+
reject_suggestion,
|
| 38 |
+
set_hit_review_status,
|
| 39 |
+
suppress_hit as apply_hit_suppression,
|
| 40 |
+
undo_last as apply_undo,
|
| 41 |
+
)
|
| 42 |
|
| 43 |
ROOT = Path(__file__).resolve().parent
|
| 44 |
WEB_DIR = ROOT / "web"
|
| 45 |
RUNS_DIR = ROOT / ".runs"
|
| 46 |
RUNS_DIR.mkdir(exist_ok=True)
|
| 47 |
|
| 48 |
+
app = FastAPI(title="Drum Sample Extractor", version="12.0.0")
|
| 49 |
app.add_middleware(
|
| 50 |
CORSMiddleware,
|
| 51 |
allow_origins=["*"],
|
|
|
|
| 168 |
|
| 169 |
try:
|
| 170 |
result = run_extraction_pipeline(input_path, output_dir, PipelineParams.from_mapping(params), progress_cb=progress)
|
| 171 |
+
load_or_create_state(job_id, output_dir)
|
| 172 |
_update_job(job_id, status="complete", result=asdict(result), error=None)
|
| 173 |
except Exception as exc: # deliberately explicit for UI diagnostics
|
| 174 |
_update_job(job_id, status="error", error=str(exc), traceback=traceback.format_exc())
|
|
|
|
| 262 |
raise HTTPException(status_code=404, detail="Job not found")
|
| 263 |
|
| 264 |
|
| 265 |
+
|
| 266 |
+
|
| 267 |
+
def _job_output_dir(job_id: str) -> Path:
|
| 268 |
+
with jobs_lock:
|
| 269 |
+
job = jobs.get(job_id)
|
| 270 |
+
if job and job.get("output_dir"):
|
| 271 |
+
return Path(job["output_dir"])
|
| 272 |
+
manifest = _manifest_path(job_id)
|
| 273 |
+
if manifest.exists():
|
| 274 |
+
return manifest.parent
|
| 275 |
+
raise HTTPException(status_code=404, detail="Job not found")
|
| 276 |
+
|
| 277 |
+
|
| 278 |
+
def _state_payload(job_id: str) -> dict[str, Any]:
|
| 279 |
+
out = _job_output_dir(job_id)
|
| 280 |
+
try:
|
| 281 |
+
state = load_or_create_state(job_id, out)
|
| 282 |
+
except FileNotFoundError as exc:
|
| 283 |
+
raise HTTPException(status_code=409, detail="Job has no manifest yet; wait until extraction completes") from exc
|
| 284 |
+
except Exception as exc:
|
| 285 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 286 |
+
return public_state(state, url_for=lambda rel: _job_url(job_id, rel))
|
| 287 |
+
|
| 288 |
+
|
| 289 |
+
def _json_patch(payload: dict[str, Any] | None) -> dict[str, Any]:
|
| 290 |
+
return dict(payload or {})
|
| 291 |
+
|
| 292 |
@app.get("/api/jobs/{job_id}/events")
|
| 293 |
def get_job_events(job_id: str) -> StreamingResponse:
|
| 294 |
with jobs_lock:
|
|
|
|
| 327 |
)
|
| 328 |
|
| 329 |
|
| 330 |
+
|
| 331 |
+
|
| 332 |
+
@app.get("/api/jobs/{job_id}/state")
|
| 333 |
+
def get_job_state(job_id: str) -> dict[str, Any]:
|
| 334 |
+
return _state_payload(job_id)
|
| 335 |
+
|
| 336 |
+
|
| 337 |
+
@app.post("/api/jobs/{job_id}/hits/{hit_id}/move")
|
| 338 |
+
def post_move_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
|
| 339 |
+
target_cluster_id = _json_patch(payload).get("target_cluster_id")
|
| 340 |
+
if not target_cluster_id:
|
| 341 |
+
raise HTTPException(status_code=400, detail="target_cluster_id is required")
|
| 342 |
+
try:
|
| 343 |
+
apply_hit_move(_job_output_dir(job_id), job_id, hit_id, str(target_cluster_id))
|
| 344 |
+
except KeyError as exc:
|
| 345 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 346 |
+
except Exception as exc:
|
| 347 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 348 |
+
return _state_payload(job_id)
|
| 349 |
+
|
| 350 |
+
|
| 351 |
+
@app.post("/api/jobs/{job_id}/hits/{hit_id}/pull-out")
|
| 352 |
+
def post_pull_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
|
| 353 |
+
label = _json_patch(payload).get("label")
|
| 354 |
+
try:
|
| 355 |
+
pull_hit_to_new_cluster(_job_output_dir(job_id), job_id, hit_id, label=label)
|
| 356 |
+
except KeyError as exc:
|
| 357 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 358 |
+
except Exception as exc:
|
| 359 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 360 |
+
return _state_payload(job_id)
|
| 361 |
+
|
| 362 |
+
|
| 363 |
+
@app.post("/api/jobs/{job_id}/hits/{hit_id}/suppress")
|
| 364 |
+
def post_suppress_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
|
| 365 |
+
reason = str(_json_patch(payload).get("reason") or "bleed")
|
| 366 |
+
try:
|
| 367 |
+
apply_hit_suppression(_job_output_dir(job_id), job_id, hit_id, reason=reason)
|
| 368 |
+
except KeyError as exc:
|
| 369 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 370 |
+
except Exception as exc:
|
| 371 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 372 |
+
return _state_payload(job_id)
|
| 373 |
+
|
| 374 |
+
|
| 375 |
+
@app.post("/api/jobs/{job_id}/hits/{hit_id}/review")
|
| 376 |
+
def post_review_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
|
| 377 |
+
status = str(_json_patch(payload).get("status") or "accepted")
|
| 378 |
+
try:
|
| 379 |
+
set_hit_review_status(_job_output_dir(job_id), job_id, hit_id, status=status)
|
| 380 |
+
except KeyError as exc:
|
| 381 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 382 |
+
except ValueError as exc:
|
| 383 |
+
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
| 384 |
+
except Exception as exc:
|
| 385 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 386 |
+
return _state_payload(job_id)
|
| 387 |
+
|
| 388 |
+
|
| 389 |
+
@app.post("/api/jobs/{job_id}/clusters/{cluster_id:path}/lock")
|
| 390 |
+
def post_lock_cluster(job_id: str, cluster_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
|
| 391 |
+
locked = bool(_json_patch(payload).get("locked", True))
|
| 392 |
+
try:
|
| 393 |
+
apply_cluster_lock(_job_output_dir(job_id), job_id, cluster_id, locked=locked)
|
| 394 |
+
except KeyError as exc:
|
| 395 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 396 |
+
except Exception as exc:
|
| 397 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 398 |
+
return _state_payload(job_id)
|
| 399 |
+
|
| 400 |
+
|
| 401 |
+
@app.get("/api/jobs/{job_id}/suggestions")
|
| 402 |
+
def get_suggestions(job_id: str) -> dict[str, Any]:
|
| 403 |
+
state = _state_payload(job_id)
|
| 404 |
+
return {"suggestions": state.get("suggestions", []), "summary": state.get("summary", {})}
|
| 405 |
+
|
| 406 |
+
|
| 407 |
+
@app.post("/api/jobs/{job_id}/suggestions/{suggestion_id}/accept")
|
| 408 |
+
def post_accept_suggestion(job_id: str, suggestion_id: str) -> dict[str, Any]:
|
| 409 |
+
try:
|
| 410 |
+
accept_suggestion(_job_output_dir(job_id), job_id, suggestion_id)
|
| 411 |
+
except KeyError as exc:
|
| 412 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 413 |
+
except ValueError as exc:
|
| 414 |
+
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
| 415 |
+
except Exception as exc:
|
| 416 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 417 |
+
return _state_payload(job_id)
|
| 418 |
+
|
| 419 |
+
|
| 420 |
+
@app.post("/api/jobs/{job_id}/suggestions/{suggestion_id}/reject")
|
| 421 |
+
def post_reject_suggestion(job_id: str, suggestion_id: str) -> dict[str, Any]:
|
| 422 |
+
try:
|
| 423 |
+
reject_suggestion(_job_output_dir(job_id), job_id, suggestion_id)
|
| 424 |
+
except KeyError as exc:
|
| 425 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 426 |
+
except Exception as exc:
|
| 427 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 428 |
+
return _state_payload(job_id)
|
| 429 |
+
|
| 430 |
+
|
| 431 |
+
@app.get("/api/jobs/{job_id}/explain/cluster/{cluster_id:path}")
|
| 432 |
+
def get_cluster_explanation(job_id: str, cluster_id: str) -> dict[str, Any]:
|
| 433 |
+
out = _job_output_dir(job_id)
|
| 434 |
+
state = load_or_create_state(job_id, out)
|
| 435 |
+
try:
|
| 436 |
+
explanation = build_cluster_explanation(state, cluster_id)
|
| 437 |
+
except KeyError as exc:
|
| 438 |
+
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
| 439 |
+
return explanation
|
| 440 |
+
|
| 441 |
+
|
| 442 |
+
@app.post("/api/jobs/{job_id}/undo")
|
| 443 |
+
def post_undo(job_id: str) -> dict[str, Any]:
|
| 444 |
+
try:
|
| 445 |
+
apply_undo(_job_output_dir(job_id), job_id)
|
| 446 |
+
except Exception as exc:
|
| 447 |
+
raise HTTPException(status_code=500, detail=str(exc)) from exc
|
| 448 |
+
return _state_payload(job_id)
|
| 449 |
+
|
| 450 |
+
|
| 451 |
@app.get("/api/jobs/{job_id}/files/{relative_path:path}")
|
| 452 |
def get_job_file(job_id: str, relative_path: str) -> FileResponse:
|
| 453 |
root = (RUNS_DIR / job_id / "output").resolve()
|
docs/API.md
CHANGED
|
@@ -212,3 +212,170 @@ Defined in `pipeline_runner.PipelineParams`.
|
|
| 212 |
| `subdivision` | `16` | MIDI grid subdivision. |
|
| 213 |
| `device` | `cpu` | Torch device for Demucs. |
|
| 214 |
| `use_disk_cache` | `true` | Cache decoded full mix/stems by source digest and extraction settings. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 212 |
| `subdivision` | `16` | MIDI grid subdivision. |
|
| 213 |
| `device` | `cpu` | Torch device for Demucs. |
|
| 214 |
| `use_disk_cache` | `true` | Cache decoded full mix/stems by source digest and extraction settings. |
|
| 215 |
+
|
| 216 |
+
## Interactive supervision API
|
| 217 |
+
|
| 218 |
+
The interactive supervision API is backed by `supervised_state.py` and persists state as:
|
| 219 |
+
|
| 220 |
+
```text
|
| 221 |
+
.runs/<job_id>/output/supervision_state.json
|
| 222 |
+
```
|
| 223 |
+
|
| 224 |
+
The batch `manifest.json` remains immutable. Supervised edits currently update semantic state only; they do not yet regenerate WAV/MIDI/ZIP artifacts.
|
| 225 |
+
|
| 226 |
+
### `GET /api/jobs/{job_id}/state`
|
| 227 |
+
|
| 228 |
+
Returns the supervised state for a completed job. If the state file does not exist yet, it is created from the batch manifest.
|
| 229 |
+
|
| 230 |
+
Response keys:
|
| 231 |
+
|
| 232 |
+
| Key | Meaning |
|
| 233 |
+
|---|---|
|
| 234 |
+
| `summary` | Counts for hits, clusters, constraints, events, suggestions, suppressed hits, locked clusters, undo availability. |
|
| 235 |
+
| `hits` | Semantic hit rows with confidence, suppression/favorite/review flags, file URLs, and current cluster assignment. |
|
| 236 |
+
| `clusters` | Semantic clusters with hit IDs, representative hit, confidence, locked state, and suppressed count. |
|
| 237 |
+
| `review_queue` | Low-confidence/high-priority hits sorted for review. |
|
| 238 |
+
| `constraints` | Recent replayable constraints. |
|
| 239 |
+
| `events` | Recent state mutation events. |
|
| 240 |
+
| `suggestions` | Open move/split/suppress suggestions. |
|
| 241 |
+
|
| 242 |
+
```bash
|
| 243 |
+
curl http://127.0.0.1:7860/api/jobs/<job-id>/state
|
| 244 |
+
```
|
| 245 |
+
|
| 246 |
+
### `POST /api/jobs/{job_id}/hits/{hit_id}/move`
|
| 247 |
+
|
| 248 |
+
Moves a hit into an existing target cluster.
|
| 249 |
+
|
| 250 |
+
Body:
|
| 251 |
+
|
| 252 |
+
```json
|
| 253 |
+
{"target_cluster_id":"cluster:0"}
|
| 254 |
+
```
|
| 255 |
+
|
| 256 |
+
Effects:
|
| 257 |
+
|
| 258 |
+
- updates hit membership in `supervision_state.json`,
|
| 259 |
+
- creates `force-cluster`,
|
| 260 |
+
- creates `must-link` to the target representative when possible,
|
| 261 |
+
- appends events,
|
| 262 |
+
- recomputes confidence/review queue,
|
| 263 |
+
- may create similar-hit move suggestions,
|
| 264 |
+
- pushes an undo snapshot.
|
| 265 |
+
|
| 266 |
+
Example:
|
| 267 |
+
|
| 268 |
+
```bash
|
| 269 |
+
curl -X POST http://127.0.0.1:7860/api/jobs/<job-id>/hits/hit%3A00003/move \
|
| 270 |
+
-H 'Content-Type: application/json' \
|
| 271 |
+
-d '{"target_cluster_id":"cluster:0"}'
|
| 272 |
+
```
|
| 273 |
+
|
| 274 |
+
### `POST /api/jobs/{job_id}/hits/{hit_id}/pull-out`
|
| 275 |
+
|
| 276 |
+
Pulls a hit into a new user cluster.
|
| 277 |
+
|
| 278 |
+
Optional body:
|
| 279 |
+
|
| 280 |
+
```json
|
| 281 |
+
{"label":"snare_user_1"}
|
| 282 |
+
```
|
| 283 |
+
|
| 284 |
+
Effects:
|
| 285 |
+
|
| 286 |
+
- creates a new `cluster:user:*` cluster,
|
| 287 |
+
- creates `cannot-link` from the source representative when possible,
|
| 288 |
+
- creates `force-cluster`,
|
| 289 |
+
- may create split suggestions,
|
| 290 |
+
- pushes an undo snapshot.
|
| 291 |
+
|
| 292 |
+
### `POST /api/jobs/{job_id}/hits/{hit_id}/suppress`
|
| 293 |
+
|
| 294 |
+
Marks a hit as bleed/noise/non-sample material.
|
| 295 |
+
|
| 296 |
+
Body:
|
| 297 |
+
|
| 298 |
+
```json
|
| 299 |
+
{"reason":"bleed"}
|
| 300 |
+
```
|
| 301 |
+
|
| 302 |
+
Effects:
|
| 303 |
+
|
| 304 |
+
- marks the hit `suppressed`,
|
| 305 |
+
- creates `suppress-pattern`,
|
| 306 |
+
- may create similar suppression suggestions,
|
| 307 |
+
- recomputes confidence and review priority.
|
| 308 |
+
|
| 309 |
+
### `POST /api/jobs/{job_id}/hits/{hit_id}/review`
|
| 310 |
+
|
| 311 |
+
Stores a review decision for a hit.
|
| 312 |
+
|
| 313 |
+
Body:
|
| 314 |
+
|
| 315 |
+
```json
|
| 316 |
+
{"status":"accepted"}
|
| 317 |
+
```
|
| 318 |
+
|
| 319 |
+
Supported statuses:
|
| 320 |
+
|
| 321 |
+
| Status | Meaning |
|
| 322 |
+
|---|---|
|
| 323 |
+
| `unreviewed` | Clear explicit review status. |
|
| 324 |
+
| `accepted` | Mark the hit as reviewed/accepted. |
|
| 325 |
+
| `favorite` | Mark as favorite and pin as semantic representative for its cluster. |
|
| 326 |
+
|
| 327 |
+
### `POST /api/jobs/{job_id}/clusters/{cluster_id}/lock`
|
| 328 |
+
|
| 329 |
+
Locks or unlocks a cluster.
|
| 330 |
+
|
| 331 |
+
Body:
|
| 332 |
+
|
| 333 |
+
```json
|
| 334 |
+
{"locked":true}
|
| 335 |
+
```
|
| 336 |
+
|
| 337 |
+
Lock state is persisted and shown in the cluster board. It does not yet alter future full pipeline reruns.
|
| 338 |
+
|
| 339 |
+
### `GET /api/jobs/{job_id}/suggestions`
|
| 340 |
+
|
| 341 |
+
Returns open suggestions and the state summary.
|
| 342 |
+
|
| 343 |
+
```bash
|
| 344 |
+
curl http://127.0.0.1:7860/api/jobs/<job-id>/suggestions
|
| 345 |
+
```
|
| 346 |
+
|
| 347 |
+
### `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/accept`
|
| 348 |
+
|
| 349 |
+
Applies a suggestion and records accepted constraints/examples.
|
| 350 |
+
|
| 351 |
+
Supported suggestion types:
|
| 352 |
+
|
| 353 |
+
- `move-hits`,
|
| 354 |
+
- `split-hits`,
|
| 355 |
+
- `suppress-hits`.
|
| 356 |
+
|
| 357 |
+
### `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/reject`
|
| 358 |
+
|
| 359 |
+
Marks a suggestion rejected and records an event.
|
| 360 |
+
|
| 361 |
+
### `GET /api/jobs/{job_id}/explain/cluster/{cluster_id}`
|
| 362 |
+
|
| 363 |
+
Returns explanation data for one cluster:
|
| 364 |
+
|
| 365 |
+
- label,
|
| 366 |
+
- locked state,
|
| 367 |
+
- confidence and reasons,
|
| 368 |
+
- representative hit,
|
| 369 |
+
- hit counts,
|
| 370 |
+
- label distribution,
|
| 371 |
+
- lowest-confidence outliers,
|
| 372 |
+
- relevant constraints,
|
| 373 |
+
- summary string.
|
| 374 |
+
|
| 375 |
+
### `POST /api/jobs/{job_id}/undo`
|
| 376 |
+
|
| 377 |
+
Restores the previous semantic state snapshot if available.
|
| 378 |
+
|
| 379 |
+
```bash
|
| 380 |
+
curl -X POST http://127.0.0.1:7860/api/jobs/<job-id>/undo
|
| 381 |
+
```
|
docs/FEATURES.md
CHANGED
|
@@ -60,3 +60,25 @@ Turn an input audio file into a practical drum sample pack: detected hits, group
|
|
| 60 |
- Realtime Demucs. It is not realistic for this use-case and should remain offline/cached.
|
| 61 |
- Perfect source separation. Stem quality depends on model choice and input material.
|
| 62 |
- Full DAW/sample-editor UX. This pass creates the workstation foundation; detailed editing is next.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
- Realtime Demucs. It is not realistic for this use-case and should remain offline/cached.
|
| 61 |
- Perfect source separation. Stem quality depends on model choice and input material.
|
| 62 |
- Full DAW/sample-editor UX. This pass creates the workstation foundation; detailed editing is next.
|
| 63 |
+
|
| 64 |
+
## Interactive supervised UX features
|
| 65 |
+
|
| 66 |
+
| Area | Feature | Status | Notes |
|
| 67 |
+
|---|---|---|---|
|
| 68 |
+
| Supervision | Supplied UX docs embedded | Implemented | Added and aligned under `docs/interactive-ux/`. |
|
| 69 |
+
| Supervision | Persistent semantic state | Implemented | `supervision_state.json` is created beside each run manifest. |
|
| 70 |
+
| Supervision | Hit/cluster state model | Implemented | State tracks current cluster assignment, confidence, suppression, favorite/review flags, and representatives. |
|
| 71 |
+
| Supervision | Constraint store | Implemented | Stores `force-cluster`, `must-link`, `cannot-link`, `lock-cluster`, `suppress-pattern`, and `pin-representative`. |
|
| 72 |
+
| Supervision | Event log | Implemented | State changes append replay/audit events. |
|
| 73 |
+
| Supervision | Undo stack | Implemented | Last semantic edit can be undone. |
|
| 74 |
+
| Supervision | Confidence scoring | Partial | Heuristic and deterministic; does not yet use cached mel/transient feature margins. |
|
| 75 |
+
| Supervision | Outlier-first review queue | Implemented | UI prioritizes low-confidence/singleton/unstable hits. |
|
| 76 |
+
| Supervision | Move hit to cluster | Implemented | Creates supervision constraints and may produce suggestions. |
|
| 77 |
+
| Supervision | Pull hit into new cluster | Implemented | Creates a user cluster and cannot-link/force-cluster constraints. |
|
| 78 |
+
| Supervision | Lock cluster | Implemented | Lock state persists and updates confidence/UI. |
|
| 79 |
+
| Supervision | Suppress hit as bleed | Implemented | Marks hit suppressed, stores suppress-pattern, may suggest similar suppressions. |
|
| 80 |
+
| Supervision | Favorite representative | Partial | Pins semantic representative; supervised export does not yet honor it. |
|
| 81 |
+
| Supervision | Suggestion inbox | Partial | Move/split/suppress suggestions can be accepted/rejected; exact diff preview is not implemented. |
|
| 82 |
+
| Supervision | Cluster explanation | Implemented | Backend and UI show confidence reasons, label distribution, outliers, and constraints. |
|
| 83 |
+
| Supervision | Edited artifact re-export | Not implemented | Semantic edits do not yet regenerate sample WAVs, MIDI, reconstruction, or ZIP. |
|
| 84 |
+
| Supervision | Force-onset from waveform | Not implemented | Waveform click currently auditions nearest existing hit only. |
|
docs/PROGRESS.md
CHANGED
|
@@ -81,3 +81,67 @@ Completed in this pass:
|
|
| 81 |
Outcome:
|
| 82 |
|
| 83 |
The app now supports a real review loop for inspecting what the onset detector and clustering produced. Users can audition individual detected slices, representative samples, stem audio, and reconstruction audio from one screen. Progress updates are lower-latency and less wasteful via SSE while still remaining robust in browsers that need polling fallback.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
Outcome:
|
| 82 |
|
| 83 |
The app now supports a real review loop for inspecting what the onset detector and clustering produced. Users can audition individual detected slices, representative samples, stem audio, and reconstruction audio from one screen. Progress updates are lower-latency and less wasteful via SSE while still remaining robust in browsers that need polling fallback.
|
| 84 |
+
|
| 85 |
+
## Pass 4: interactive supervised UX foundation
|
| 86 |
+
|
| 87 |
+
Completed in this pass:
|
| 88 |
+
|
| 89 |
+
1. Added the supplied interactive UX document set under `docs/interactive-ux/`.
|
| 90 |
+
2. Read and aligned the UX documents with the project as currently implemented.
|
| 91 |
+
3. Added `supervised_state.py` for persistent semantic state beside each completed run manifest.
|
| 92 |
+
4. Added `supervision_state.json` generation after each successful extraction.
|
| 93 |
+
5. Added state schema for hits, clusters, constraints, events, suggestions, confidence, review queue, and undo snapshots.
|
| 94 |
+
6. Added supervised editing endpoints:
|
| 95 |
+
- `GET /api/jobs/{job_id}/state`
|
| 96 |
+
- `POST /api/jobs/{job_id}/hits/{hit_id}/move`
|
| 97 |
+
- `POST /api/jobs/{job_id}/hits/{hit_id}/pull-out`
|
| 98 |
+
- `POST /api/jobs/{job_id}/hits/{hit_id}/suppress`
|
| 99 |
+
- `POST /api/jobs/{job_id}/hits/{hit_id}/review`
|
| 100 |
+
- `POST /api/jobs/{job_id}/clusters/{cluster_id}/lock`
|
| 101 |
+
- `GET /api/jobs/{job_id}/suggestions`
|
| 102 |
+
- `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/accept`
|
| 103 |
+
- `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/reject`
|
| 104 |
+
- `GET /api/jobs/{job_id}/explain/cluster/{cluster_id}`
|
| 105 |
+
- `POST /api/jobs/{job_id}/undo`
|
| 106 |
+
7. Added an interactive supervision UI panel with:
|
| 107 |
+
- state summary,
|
| 108 |
+
- selected-hit actions,
|
| 109 |
+
- target cluster picker,
|
| 110 |
+
- outlier-first review queue,
|
| 111 |
+
- cluster board,
|
| 112 |
+
- suggestion inbox,
|
| 113 |
+
- constraint/event log,
|
| 114 |
+
- cluster explanation drawer.
|
| 115 |
+
8. Added `scripts/test_interactive_supervision.py` to verify the supervised API loop.
|
| 116 |
+
|
| 117 |
+
Outcome:
|
| 118 |
+
|
| 119 |
+
The app is now an extraction and supervised-review workstation at the semantic-state level. User corrections are persisted as constraints/events and can be inspected, suggested from, and undone. The next required step is edited-state export so these decisions affect downloadable artifacts.
|
| 120 |
+
|
| 121 |
+
## Current assessment after Pass 4
|
| 122 |
+
|
| 123 |
+
The project now satisfies the first interactive UX milestone for replayable supervised state:
|
| 124 |
+
|
| 125 |
+
```text
|
| 126 |
+
analyze audio
|
| 127 |
+
→ inspect hits/clusters
|
| 128 |
+
→ move/pull/suppress/favorite/lock
|
| 129 |
+
→ persist constraints/events
|
| 130 |
+
→ update confidence and review queue
|
| 131 |
+
→ generate/accept/reject suggestions
|
| 132 |
+
→ explain clusters
|
| 133 |
+
→ undo semantic edits
|
| 134 |
+
→ reload completed run with decisions intact
|
| 135 |
+
```
|
| 136 |
+
|
| 137 |
+
It does not yet satisfy the full workstation loop because edited semantic state is not yet rendered into updated sample WAVs, MIDI, reconstruction, or ZIP output.
|
| 138 |
+
|
| 139 |
+
## Next recommended pass after Pass 4
|
| 140 |
+
|
| 141 |
+
1. Add supervised re-export endpoint.
|
| 142 |
+
2. Exclude suppressed hits from supervised exports.
|
| 143 |
+
3. Honor favorite/pinned representatives in supervised sample WAVs.
|
| 144 |
+
4. Add force-onset endpoint using cached `stem.wav`.
|
| 145 |
+
5. Add add-onset mode to the waveform UI.
|
| 146 |
+
6. Add restore suppressed hit and batch restore.
|
| 147 |
+
7. Add feature-vector cache for true local reclustering.
|
docs/REMAINING_WORK.md
CHANGED
|
@@ -33,3 +33,24 @@ The project is now a usable extraction workstation, not a complete interactive s
|
|
| 33 |
5. Add lower-level progress hooks inside expensive stages where practical.
|
| 34 |
6. Convert frontend to TypeScript and add UI tests.
|
| 35 |
7. Add an in-app benchmark/parameter profile panel.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
5. Add lower-level progress hooks inside expensive stages where practical.
|
| 34 |
6. Convert frontend to TypeScript and add UI tests.
|
| 35 |
7. Add an in-app benchmark/parameter profile panel.
|
| 36 |
+
|
| 37 |
+
## Remaining after interactive UX foundation
|
| 38 |
+
|
| 39 |
+
Completed since the previous remaining-work snapshot:
|
| 40 |
+
|
| 41 |
+
- Supplied `docs/interactive-ux` document set embedded and aligned.
|
| 42 |
+
- Persistent supervised state added via `supervised_state.py` and `supervision_state.json`.
|
| 43 |
+
- Constraint store and event log added.
|
| 44 |
+
- Hit/cluster confidence and outlier-first review queue added.
|
| 45 |
+
- Move hit, pull-out, lock/unlock, suppress, review/favorite, suggestions, explanations, and undo endpoints added.
|
| 46 |
+
- Interactive supervision UI panel added.
|
| 47 |
+
|
| 48 |
+
Highest-priority remaining work now:
|
| 49 |
+
|
| 50 |
+
1. **Supervised artifact export**: regenerate edited sample WAVs, MIDI, reconstruction, manifest, and ZIP from `supervision_state.json` without rerunning Demucs/onset detection.
|
| 51 |
+
2. **Force-onset correction**: add an onset by clicking/shift-clicking the waveform, slice from cached `stem.wav`, classify, assign, and store a `force-onset` constraint.
|
| 52 |
+
3. **Suppression restore**: restore suppressed hits individually and in batches.
|
| 53 |
+
4. **Real constrained local reclustering**: cache hit feature vectors and recompute affected neighborhoods after edits.
|
| 54 |
+
5. **Suggestion diff preview**: show exact before/after membership changes before accepting a suggestion.
|
| 55 |
+
6. **Constraint violation detection**: explicitly report conflicting user constraints.
|
| 56 |
+
7. **Frontend tests and TypeScript migration**: harden the increasingly stateful UI.
|
docs/TASKS.md
CHANGED
|
@@ -58,3 +58,32 @@ Last updated: 2026-05-12
|
|
| 58 |
- [ ] Add side-by-side run comparison.
|
| 59 |
- [ ] Convert frontend to TypeScript with a small Vite build once UX stabilizes.
|
| 60 |
- [ ] Add automated browser-level UI tests.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
- [ ] Add side-by-side run comparison.
|
| 59 |
- [ ] Convert frontend to TypeScript with a small Vite build once UX stabilizes.
|
| 60 |
- [ ] Add automated browser-level UI tests.
|
| 61 |
+
|
| 62 |
+
## Interactive UX continuation tasks
|
| 63 |
+
|
| 64 |
+
| Task | Status | Evidence |
|
| 65 |
+
|---|---:|---|
|
| 66 |
+
| Add supplied interactive UX docs under `docs/interactive-ux/` | Done | `docs/interactive-ux/*.md`. |
|
| 67 |
+
| Read and align UX docs with current implementation | Done | Status sections updated in every interactive UX document. |
|
| 68 |
+
| Add persistent semantic job state | Done | `supervised_state.py`, `supervision_state.json`. |
|
| 69 |
+
| Add event log and constraint store | Done | `supervised_state.py`; tested by `scripts/test_interactive_supervision.py`. |
|
| 70 |
+
| Add hit/cluster confidence and review queue | Done/Partial | Heuristic confidence and review queue implemented; feature-margin confidence remains open. |
|
| 71 |
+
| Add move hit to cluster | Done | `POST /api/jobs/{job_id}/hits/{hit_id}/move`. |
|
| 72 |
+
| Add pull hit into new cluster | Done | `POST /api/jobs/{job_id}/hits/{hit_id}/pull-out`. |
|
| 73 |
+
| Add cluster lock/unlock | Done | `POST /api/jobs/{job_id}/clusters/{cluster_id}/lock`. |
|
| 74 |
+
| Add suppress hit as bleed/noise | Done | `POST /api/jobs/{job_id}/hits/{hit_id}/suppress`. |
|
| 75 |
+
| Add accept/favorite hit action | Done/Partial | `POST /api/jobs/{job_id}/hits/{hit_id}/review`; artifact re-export still open. |
|
| 76 |
+
| Add suggestion inbox | Done/Partial | UI/API supports accept/reject; exact diff preview still open. |
|
| 77 |
+
| Add cluster explanation drawer | Done | `GET /api/jobs/{job_id}/explain/cluster/{cluster_id}` plus UI drawer. |
|
| 78 |
+
| Add semantic undo | Done | `POST /api/jobs/{job_id}/undo`. |
|
| 79 |
+
| Add supervised export from edited state | Todo | Needed so corrections affect ZIP/MIDI/WAV outputs. |
|
| 80 |
+
| Add click-to-add missed onset | Todo | Needed for `force-onset` constraints and direct onset correction. |
|
| 81 |
+
| Add suppressed-hit restore | Todo | Needed as the safety counterpart to suppression. |
|
| 82 |
+
| Add true local feature-neighborhood reclustering | Todo | Requires cached feature vectors and constraint-aware assignment. |
|
| 83 |
+
|
| 84 |
+
## Latest validation tasks
|
| 85 |
+
|
| 86 |
+
- [x] `python3 -m py_compile app.py pipeline_runner.py sample_extractor.py supervised_state.py scripts/*.py`
|
| 87 |
+
- [x] `node --check web/app.js`
|
| 88 |
+
- [x] `python3 scripts/test_sse_and_review_hits.py`
|
| 89 |
+
- [x] `python3 scripts/test_interactive_supervision.py`
|
docs/interactive-ux/ARCHITECTURE_NOTES.md
ADDED
|
@@ -0,0 +1,204 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Architecture notes for supervised interactive extraction
|
| 2 |
+
|
| 3 |
+
## Required shift
|
| 4 |
+
|
| 5 |
+
Original extraction flow:
|
| 6 |
+
|
| 7 |
+
```text
|
| 8 |
+
audio → pipeline → result artifacts
|
| 9 |
+
```
|
| 10 |
+
|
| 11 |
+
Current interactive foundation:
|
| 12 |
+
|
| 13 |
+
```text
|
| 14 |
+
audio/cache → immutable manifest/artifacts → supervision_state.json → reactive UI → user constraints/events/suggestions
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
+
The current implementation deliberately keeps the batch extraction artifacts immutable. Interactive edits mutate `supervision_state.json`, not the original `manifest.json`, hit WAVs, representative WAVs, MIDI, reconstruction, or ZIP. This keeps edits cheap and reversible, but supervised re-export is the next architectural step.
|
| 18 |
+
|
| 19 |
+
## Implemented modules
|
| 20 |
+
|
| 21 |
+
| Module/file | Responsibility |
|
| 22 |
+
|---|---|
|
| 23 |
+
| `pipeline_runner.py` | Batch extraction, timing, manifests, review-hit WAV exports |
|
| 24 |
+
| `sample_extractor.py` | Audio analysis, classification, batch/online clustering, export helpers |
|
| 25 |
+
| `supervised_state.py` | Persistent semantic job state, constraints, events, confidence, suggestions, undo |
|
| 26 |
+
| `app.py` | FastAPI endpoints for batch jobs and supervised state mutations |
|
| 27 |
+
| `web/app.js` | Browser state rendering, review queue, cluster board, suggestions, actions |
|
| 28 |
+
| `web/index.html` | Workstation layout and interactive supervision panel |
|
| 29 |
+
| `web/styles.css` | Visual treatment for low confidence, suppression, locks, panels |
|
| 30 |
+
|
| 31 |
+
## Core entities as implemented
|
| 32 |
+
|
| 33 |
+
`supervised_state.py` stores JSON dictionaries equivalent to these shapes:
|
| 34 |
+
|
| 35 |
+
```ts
|
| 36 |
+
type Hit = {
|
| 37 |
+
id: string;
|
| 38 |
+
index: number;
|
| 39 |
+
label: string;
|
| 40 |
+
cluster_id: string;
|
| 41 |
+
original_cluster_id: string;
|
| 42 |
+
cluster_label: string;
|
| 43 |
+
onset_sec: number;
|
| 44 |
+
duration_ms: number;
|
| 45 |
+
rms_energy: number;
|
| 46 |
+
spectral_centroid_hz: number;
|
| 47 |
+
file: string;
|
| 48 |
+
is_representative: boolean;
|
| 49 |
+
source: "detected" | "forced";
|
| 50 |
+
suppressed: boolean;
|
| 51 |
+
favorite: boolean;
|
| 52 |
+
review_status: "unreviewed" | "accepted" | "favorite" | "suppressed";
|
| 53 |
+
confidence: number;
|
| 54 |
+
confidence_reasons: string[];
|
| 55 |
+
explicit: boolean;
|
| 56 |
+
};
|
| 57 |
+
|
| 58 |
+
type Cluster = {
|
| 59 |
+
id: string;
|
| 60 |
+
label: string;
|
| 61 |
+
classification: string;
|
| 62 |
+
hit_ids: string[];
|
| 63 |
+
representative_hit_id: string | null;
|
| 64 |
+
locked: boolean;
|
| 65 |
+
user_named: boolean;
|
| 66 |
+
confidence: number;
|
| 67 |
+
confidence_reasons: string[];
|
| 68 |
+
suppressed_count: number;
|
| 69 |
+
original_id: string | null;
|
| 70 |
+
};
|
| 71 |
+
|
| 72 |
+
type Constraint =
|
| 73 |
+
| { id: string; type: "must-link"; a: string; b: string; source: string }
|
| 74 |
+
| { id: string; type: "cannot-link"; a: string; b: string; source: string }
|
| 75 |
+
| { id: string; type: "force-cluster"; hit_id: string; cluster_id: string; source: string }
|
| 76 |
+
| { id: string; type: "lock-cluster"; cluster_id: string; locked: boolean; source: string }
|
| 77 |
+
| { id: string; type: "suppress-pattern"; example_hit_id: string; reason: string; source: string }
|
| 78 |
+
| { id: string; type: "pin-representative"; hit_id: string; cluster_id: string; source: string };
|
| 79 |
+
```
|
| 80 |
+
|
| 81 |
+
## Current state file
|
| 82 |
+
|
| 83 |
+
Each completed run now gets:
|
| 84 |
+
|
| 85 |
+
```text
|
| 86 |
+
.runs/<job_id>/output/manifest.json
|
| 87 |
+
.runs/<job_id>/output/supervision_state.json
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
`manifest.json` is the immutable batch result. `supervision_state.json` is the mutable, replayable semantic state.
|
| 91 |
+
|
| 92 |
+
## Implemented API additions
|
| 93 |
+
|
| 94 |
+
```text
|
| 95 |
+
GET /api/jobs/{job_id}/state
|
| 96 |
+
POST /api/jobs/{job_id}/hits/{hit_id}/move
|
| 97 |
+
POST /api/jobs/{job_id}/hits/{hit_id}/pull-out
|
| 98 |
+
POST /api/jobs/{job_id}/hits/{hit_id}/suppress
|
| 99 |
+
POST /api/jobs/{job_id}/hits/{hit_id}/review
|
| 100 |
+
POST /api/jobs/{job_id}/clusters/{cluster_id}/lock
|
| 101 |
+
GET /api/jobs/{job_id}/suggestions
|
| 102 |
+
POST /api/jobs/{job_id}/suggestions/{suggestion_id}/accept
|
| 103 |
+
POST /api/jobs/{job_id}/suggestions/{suggestion_id}/reject
|
| 104 |
+
GET /api/jobs/{job_id}/explain/cluster/{cluster_id}
|
| 105 |
+
POST /api/jobs/{job_id}/undo
|
| 106 |
+
```
|
| 107 |
+
|
| 108 |
+
## Current confidence scoring
|
| 109 |
+
|
| 110 |
+
Initial confidence is heuristic and deterministic. It combines:
|
| 111 |
+
|
| 112 |
+
- cluster size,
|
| 113 |
+
- cluster label purity,
|
| 114 |
+
- representative presence,
|
| 115 |
+
- lock state,
|
| 116 |
+
- hit label agreement with cluster label,
|
| 117 |
+
- energy rank,
|
| 118 |
+
- rough duration reasonableness,
|
| 119 |
+
- representative/favorite/explicit assignment state,
|
| 120 |
+
- suppression state.
|
| 121 |
+
|
| 122 |
+
This is good enough to drive the review queue but should be replaced or supplemented by cached feature-vector margins.
|
| 123 |
+
|
| 124 |
+
## Current suggestion engine
|
| 125 |
+
|
| 126 |
+
Implemented suggestion types:
|
| 127 |
+
|
| 128 |
+
```ts
|
| 129 |
+
type Suggestion =
|
| 130 |
+
| { type: "move-hits"; hit_ids: string[]; target_cluster_id: string; confidence: number; reason: string }
|
| 131 |
+
| { type: "split-hits"; hit_ids: string[]; source_cluster_id: string; target_cluster_id: string; confidence: number; reason: string }
|
| 132 |
+
| { type: "suppress-hits"; hit_ids: string[]; confidence: number; reason: string };
|
| 133 |
+
```
|
| 134 |
+
|
| 135 |
+
Suggestion generation currently uses label, spectral centroid, and RMS-energy similarity. Accepted suggestions become explicit constraints/examples.
|
| 136 |
+
|
| 137 |
+
## Event log
|
| 138 |
+
|
| 139 |
+
Examples now emitted:
|
| 140 |
+
|
| 141 |
+
```text
|
| 142 |
+
job.state.created
|
| 143 |
+
constraint.created
|
| 144 |
+
hit.moved
|
| 145 |
+
hit.pulled_out
|
| 146 |
+
cluster.locked
|
| 147 |
+
cluster.unlocked
|
| 148 |
+
hit.suppressed
|
| 149 |
+
hit.reviewed
|
| 150 |
+
suggestion.created
|
| 151 |
+
suggestion.accepted
|
| 152 |
+
suggestion.rejected
|
| 153 |
+
state.undo
|
| 154 |
+
```
|
| 155 |
+
|
| 156 |
+
The UI renders recent events and constraints in the supervision panel.
|
| 157 |
+
|
| 158 |
+
## Local recomputation boundary
|
| 159 |
+
|
| 160 |
+
Implemented now:
|
| 161 |
+
|
| 162 |
+
```text
|
| 163 |
+
semantic edit
|
| 164 |
+
→ update hit/cluster membership in supervision_state.json
|
| 165 |
+
→ append constraints/events
|
| 166 |
+
→ generate heuristic suggestions
|
| 167 |
+
→ recompute hit/cluster confidence and review queue
|
| 168 |
+
```
|
| 169 |
+
|
| 170 |
+
Not implemented yet:
|
| 171 |
+
|
| 172 |
+
```text
|
| 173 |
+
semantic edit
|
| 174 |
+
→ load cached feature vectors
|
| 175 |
+
→ choose affected neighborhood
|
| 176 |
+
→ run constrained local reclustering
|
| 177 |
+
→ create preview diff
|
| 178 |
+
→ optionally apply/re-export artifacts
|
| 179 |
+
```
|
| 180 |
+
|
| 181 |
+
## UI state implications
|
| 182 |
+
|
| 183 |
+
Implemented panels:
|
| 184 |
+
|
| 185 |
+
- waveform + onset audition,
|
| 186 |
+
- representative samples,
|
| 187 |
+
- detected hit review,
|
| 188 |
+
- outlier-first review queue,
|
| 189 |
+
- cluster board,
|
| 190 |
+
- suggestion inbox,
|
| 191 |
+
- constraint/history inspector,
|
| 192 |
+
- cluster explanation drawer,
|
| 193 |
+
- export/download panel for the original batch run.
|
| 194 |
+
|
| 195 |
+
Still missing:
|
| 196 |
+
|
| 197 |
+
- edited export panel,
|
| 198 |
+
- force-onset mode,
|
| 199 |
+
- suppression restore UI,
|
| 200 |
+
- side-by-side before/after diff preview.
|
| 201 |
+
|
| 202 |
+
## Implementation warning
|
| 203 |
+
|
| 204 |
+
Automatic propagation must stay conservative. The current implementation follows this by creating suggestions rather than silently moving/suppressing batches. Every semantic edit is undoable.
|
docs/interactive-ux/FEASIBILITY_MATRIX.md
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Feature feasibility matrix
|
| 2 |
+
|
| 3 |
+
## Scoring convention
|
| 4 |
+
|
| 5 |
+
All scores are between `0.0` and `1.0`.
|
| 6 |
+
|
| 7 |
+
For positive dimensions, higher is better:
|
| 8 |
+
|
| 9 |
+
- **Feasibility**: likelihood this can be implemented robustly with the current project direction.
|
| 10 |
+
- **Value UX**: how much nicer/faster the application feels.
|
| 11 |
+
- **Value quality**: expected improvement in extraction correctness.
|
| 12 |
+
- **Realtime fit**: how well the interaction can update from cached features without full reruns.
|
| 13 |
+
- **MVP fit**: whether it belongs in the first serious interactive version.
|
| 14 |
+
|
| 15 |
+
For cost dimensions, higher is harder/more expensive:
|
| 16 |
+
|
| 17 |
+
- **Complexity**: algorithmic/product complexity.
|
| 18 |
+
- **Effort**: implementation effort.
|
| 19 |
+
|
| 20 |
+
## Matrix
|
| 21 |
+
|
| 22 |
+
| Interaction | Feasibility | Complexity | Effort | Value UX | Value quality | Realtime fit | MVP fit | Verdict |
|
| 23 |
+
|---|---:|---:|---:|---:|---:|---:|---:|---|
|
| 24 |
+
| Move sample to cluster → auto-tune clustering | 0.90 | 0.55 | 0.45 | 0.95 | 0.90 | 0.85 | 0.95 | Build early |
|
| 25 |
+
| Pull sample out → protect distinction | 0.90 | 0.55 | 0.45 | 0.90 | 0.90 | 0.85 | 0.95 | Build early |
|
| 26 |
+
| Lock confirmed cluster identity | 0.95 | 0.25 | 0.20 | 0.80 | 0.75 | 0.95 | 1.00 | Build immediately |
|
| 27 |
+
| Outlier-first review queue | 0.95 | 0.30 | 0.25 | 0.90 | 0.80 | 0.95 | 1.00 | Build immediately |
|
| 28 |
+
| Low-confidence visual emphasis | 0.95 | 0.25 | 0.20 | 0.85 | 0.65 | 0.95 | 0.95 | Build immediately |
|
| 29 |
+
| Bleed brush/suppression | 0.85 | 0.55 | 0.50 | 0.90 | 0.85 | 0.80 | 0.85 | Build early |
|
| 30 |
+
| Click-to-add missed onset | 0.90 | 0.45 | 0.40 | 0.90 | 0.80 | 0.90 | 0.90 | Build early |
|
| 31 |
+
| Cluster naming as reusable semantic hint | 0.75 | 0.65 | 0.55 | 0.75 | 0.65 | 0.70 | 0.40 | Useful, not first |
|
| 32 |
+
| Star/favorite sample → optimize around it | 0.95 | 0.35 | 0.30 | 0.75 | 0.70 | 0.90 | 0.75 | Build early |
|
| 33 |
+
| Explain this cluster | 0.85 | 0.50 | 0.45 | 0.80 | 0.65 | 0.85 | 0.80 | Build early |
|
| 34 |
+
| Live counterfactual parameter previews | 0.80 | 0.70 | 0.65 | 0.90 | 0.75 | 0.75 | 0.55 | High value, later |
|
| 35 |
+
| Temporal pattern supervision | 0.75 | 0.70 | 0.65 | 0.80 | 0.85 | 0.70 | 0.45 | Later |
|
| 36 |
+
| Reconstruction-error-driven correction | 0.70 | 0.80 | 0.75 | 0.85 | 0.90 | 0.55 | 0.35 | Later, powerful |
|
| 37 |
+
| Multi-resolution semantic clustering | 0.80 | 0.70 | 0.65 | 0.85 | 0.80 | 0.75 | 0.50 | Later |
|
| 38 |
+
| Auto-clean this family | 0.80 | 0.60 | 0.55 | 0.75 | 0.80 | 0.75 | 0.50 | Later |
|
| 39 |
+
| Drag clusters in semantic space | 0.70 | 0.75 | 0.75 | 0.90 | 0.65 | 0.65 | 0.30 | Cool, not first |
|
| 40 |
+
| Cluster gravity / physics metaphor | 0.60 | 0.80 | 0.80 | 0.80 | 0.50 | 0.60 | 0.20 | Risky novelty |
|
| 41 |
+
| Context-aware classification | 0.65 | 0.80 | 0.75 | 0.65 | 0.80 | 0.55 | 0.25 | Researchy |
|
| 42 |
+
| Teach mode across songs | 0.70 | 0.85 | 0.80 | 0.85 | 0.85 | 0.60 | 0.25 | Later platform feature |
|
| 43 |
+
| Predictive batch questions | 0.85 | 0.55 | 0.50 | 0.90 | 0.80 | 0.80 | 0.70 | Build after uncertainty scoring |
|
| 44 |
+
|
| 45 |
+
## Highest ROI set
|
| 46 |
+
|
| 47 |
+
| Rank | Feature | Why |
|
| 48 |
+
|---:|---|---|
|
| 49 |
+
| 1 | Lock confirmed cluster identity | Easy and prevents frustrating reclustering drift |
|
| 50 |
+
| 2 | Outlier-first review queue | Huge UX gain from simple uncertainty ranking |
|
| 51 |
+
| 3 | Move sample to cluster as supervision | Core differentiator; directly improves results |
|
| 52 |
+
| 4 | Pull sample out / protect distinction | Required counterpart to positive supervision |
|
| 53 |
+
| 5 | Click-to-add missed onset | Direct correction beats indirect threshold tweaking |
|
| 54 |
+
| 6 | Bleed brush | Removes many false positives quickly |
|
| 55 |
+
| 7 | Explain cluster | Makes the system debuggable and trustworthy |
|
| 56 |
+
| 8 | Predictive batch questions | Multiplies the effect of each correction |
|
| 57 |
+
| 9 | Counterfactual previews | Makes advanced tuning understandable |
|
| 58 |
+
| 10 | Reconstruction-error correction | Very powerful, but architecturally heavier |
|
| 59 |
+
|
| 60 |
+
## Technical conclusion
|
| 61 |
+
|
| 62 |
+
The most feasible "magic" is not heavy ML. It is constraint-aware clustering, cached feature vectors, uncertainty scoring, and local recomputation.
|
| 63 |
+
|
| 64 |
+
That foundation should be implemented before adding higher-risk semantic-space or personalized-model features.
|
| 65 |
+
|
| 66 |
+
## Implementation alignment as of 2026-05-12
|
| 67 |
+
|
| 68 |
+
| Interaction | Current status | Notes |
|
| 69 |
+
|---|---|---|
|
| 70 |
+
| Move sample to cluster → auto-tune clustering | partial | Implemented as semantic state mutation with `force-cluster`/`must-link` constraints and heuristic move suggestions. True constrained local reclustering is still open. |
|
| 71 |
+
| Pull sample out → protect distinction | partial | Implemented as a new user cluster plus `cannot-link` and split suggestions. True constrained reclustering is still open. |
|
| 72 |
+
| Lock confirmed cluster identity | done | Lock/unlock persists in `supervision_state.json` and appears in the UI. Replay into future reruns is still open. |
|
| 73 |
+
| Outlier-first review queue | done | Implemented with heuristic confidence/priority. Feature-margin and reconstruction-impact ranking remain open. |
|
| 74 |
+
| Low-confidence visual emphasis | done | Low-confidence and suppressed hits are visually distinguished in the hit table and review queue. |
|
| 75 |
+
| Bleed brush/suppression | partial | Suppress selected hit and similar suppression suggestions are implemented. Region brush and restore are still open. |
|
| 76 |
+
| Click-to-add missed onset | todo | Waveform click currently auditions nearest existing hit only. |
|
| 77 |
+
| Cluster naming as reusable semantic hint | todo | User clusters receive generated labels; explicit rename/semantic hinting is not implemented. |
|
| 78 |
+
| Star/favorite sample → optimize around it | partial | Favorite pins representative in semantic state; artifact re-export does not yet honor it. |
|
| 79 |
+
| Explain this cluster | done | Explanation endpoint and UI drawer are implemented. |
|
| 80 |
+
| Predictive batch questions | partial | Suggestion inbox exists; exact diff previews and richer question phrasing are open. |
|
| 81 |
+
| Live counterfactual parameter previews | todo | Not implemented. |
|
| 82 |
+
| Reconstruction-error-driven correction | todo | Not implemented. |
|
| 83 |
+
| Multi-resolution semantic clustering | todo | Not implemented. |
|
| 84 |
+
| Auto-clean this family | todo | Not implemented. |
|
| 85 |
+
| Drag clusters in semantic space | backlog | Not implemented. |
|
| 86 |
+
| Cluster gravity / physics metaphor | backlog | Not implemented. |
|
| 87 |
+
| Context-aware classification | backlog | Not implemented. |
|
| 88 |
+
| Teach mode across songs | backlog | Not implemented. |
|
| 89 |
+
|
| 90 |
+
## Revised highest-ROI next set
|
| 91 |
+
|
| 92 |
+
| Rank | Feature | Why now |
|
| 93 |
+
|---:|---|---|
|
| 94 |
+
| 1 | Supervised re-export | Makes current semantic edits affect the downloadable sample pack. |
|
| 95 |
+
| 2 | Force-onset from waveform | Adds the missing direct correction primitive for missed hits. |
|
| 96 |
+
| 3 | Suppression restore | Required safety counterpart to suppression. |
|
| 97 |
+
| 4 | Cached feature refs | Unlocks real local reclustering and better confidence. |
|
| 98 |
+
| 5 | Diff preview for suggestions | Makes batch suggestions safer and more trustworthy. |
|
| 99 |
+
| 6 | Constraint violation detection | Prevents silent conflicts once constraints become richer. |
|
| 100 |
+
| 7 | Browser tests | Protects the increasingly stateful UI from regressions. |
|
docs/interactive-ux/FEATURE_REQUIREMENTS.md
ADDED
|
@@ -0,0 +1,162 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Interactive UX feature requirements
|
| 2 |
+
|
| 3 |
+
## Goal
|
| 4 |
+
|
| 5 |
+
Add interactions that make extraction faster and more accurate by converting user actions into reusable supervision signals.
|
| 6 |
+
|
| 7 |
+
The application should progressively converge toward the user's intended drum vocabulary with minimal manual cleanup.
|
| 8 |
+
|
| 9 |
+
## Success criteria
|
| 10 |
+
|
| 11 |
+
- User corrections affect more than the single edited item when safe.
|
| 12 |
+
- Explicit user intent is preserved across reclustering and reloads.
|
| 13 |
+
- The system surfaces uncertain/high-leverage items before stable ones.
|
| 14 |
+
- The user can understand why items are grouped or separated.
|
| 15 |
+
- Cached stem/source analysis can be reused while downstream parameters update quickly.
|
| 16 |
+
- The UI supports fast audition, correction, locking, and export loops.
|
| 17 |
+
|
| 18 |
+
## Current implementation summary
|
| 19 |
+
|
| 20 |
+
Implemented now:
|
| 21 |
+
|
| 22 |
+
- Persistent semantic state per run: `supervision_state.json`.
|
| 23 |
+
- Event log and constraint store.
|
| 24 |
+
- Confidence-weighted hit/cluster state.
|
| 25 |
+
- Outlier-first review queue.
|
| 26 |
+
- Move hit to cluster.
|
| 27 |
+
- Pull hit into new cluster.
|
| 28 |
+
- Lock/unlock cluster.
|
| 29 |
+
- Suppress hit as bleed/noise.
|
| 30 |
+
- Accept/favorite selected hit.
|
| 31 |
+
- Suggestion inbox with accept/reject.
|
| 32 |
+
- Cluster explanation endpoint and UI drawer.
|
| 33 |
+
- Undo for the last semantic edit.
|
| 34 |
+
|
| 35 |
+
Partially implemented:
|
| 36 |
+
|
| 37 |
+
- Local recomputation is currently semantic-state recomputation, not full feature-neighborhood reclustering.
|
| 38 |
+
- Suggestions are heuristic and preview-count based, not full diff previews.
|
| 39 |
+
- Favorite/pin changes semantic representative but does not yet regenerate the sample pack.
|
| 40 |
+
- Confidence scoring is heuristic, not feature-margin/stability based.
|
| 41 |
+
|
| 42 |
+
Not implemented yet:
|
| 43 |
+
|
| 44 |
+
- Click-to-add missed onset.
|
| 45 |
+
- Restore suppressed hit.
|
| 46 |
+
- Supervised re-export from edited state.
|
| 47 |
+
- Counterfactual parameter previews.
|
| 48 |
+
- Reconstruction-error correction.
|
| 49 |
+
- Teach mode across songs.
|
| 50 |
+
|
| 51 |
+
## Functional requirements and status
|
| 52 |
+
|
| 53 |
+
### FR-001: Cluster move as positive supervision
|
| 54 |
+
|
| 55 |
+
Status: **implemented as semantic-state edit**.
|
| 56 |
+
|
| 57 |
+
When a user moves a hit/sample into a cluster, the backend creates `force-cluster` and, when a representative exists, `must-link`. State confidence is recomputed and similar hit suggestions may be generated.
|
| 58 |
+
|
| 59 |
+
Remaining gap: true local feature-neighborhood reclustering and exact before/after diff preview.
|
| 60 |
+
|
| 61 |
+
### FR-002: Pull-out as negative supervision
|
| 62 |
+
|
| 63 |
+
Status: **implemented as semantic-state edit**.
|
| 64 |
+
|
| 65 |
+
Pulling a hit out creates a new user cluster, records a `cannot-link` to the source representative when possible, and stores a `force-cluster` assignment for the new cluster.
|
| 66 |
+
|
| 67 |
+
Remaining gap: automatic split suggestions are heuristic and do not yet run constrained reclustering.
|
| 68 |
+
|
| 69 |
+
### FR-003: Lock confirmed cluster identity
|
| 70 |
+
|
| 71 |
+
Status: **implemented**.
|
| 72 |
+
|
| 73 |
+
Clusters can be locked/unlocked through the API/UI. Lock state is persisted and influences confidence.
|
| 74 |
+
|
| 75 |
+
Remaining gap: future full reruns do not yet replay locks into batch clustering.
|
| 76 |
+
|
| 77 |
+
### FR-004: Outlier-first review queue
|
| 78 |
+
|
| 79 |
+
Status: **implemented**.
|
| 80 |
+
|
| 81 |
+
The backend returns a `review_queue` sorted by low confidence, singleton status, review status, and suppression state. The UI renders the queue and lets the user select/audition items.
|
| 82 |
+
|
| 83 |
+
Remaining gap: expected-impact ranking should eventually use feature margin and reconstruction contribution.
|
| 84 |
+
|
| 85 |
+
### FR-005: Confidence-weighted visual emphasis
|
| 86 |
+
|
| 87 |
+
Status: **implemented**.
|
| 88 |
+
|
| 89 |
+
Hit rows display confidence and flags. Low-confidence rows get emphasis; suppressed rows visually recede.
|
| 90 |
+
|
| 91 |
+
### FR-006: Click-to-add missed onset
|
| 92 |
+
|
| 93 |
+
Status: **not implemented**.
|
| 94 |
+
|
| 95 |
+
Current waveform click auditions the nearest existing hit only.
|
| 96 |
+
|
| 97 |
+
Required next behavior:
|
| 98 |
+
|
| 99 |
+
- Add `force-onset` constraint at selected time.
|
| 100 |
+
- Slice candidate hit from cached `stem.wav`.
|
| 101 |
+
- Classify and assign locally.
|
| 102 |
+
- Store it as a forced hit in `supervision_state.json`.
|
| 103 |
+
|
| 104 |
+
### FR-007: Bleed brush / false-positive suppression
|
| 105 |
+
|
| 106 |
+
Status: **implemented for selected hits; brush region not implemented**.
|
| 107 |
+
|
| 108 |
+
Selected hits can be suppressed as bleed/noise. The system stores `suppress-pattern` and proposes similar suppressions.
|
| 109 |
+
|
| 110 |
+
Remaining gap: region brush, restore, and supervised export exclusion.
|
| 111 |
+
|
| 112 |
+
### FR-008: Favorite/pin sample optimization
|
| 113 |
+
|
| 114 |
+
Status: **partial**.
|
| 115 |
+
|
| 116 |
+
Favorite action records `pin-representative` and updates the semantic representative. Exported WAV/ZIP does not yet change.
|
| 117 |
+
|
| 118 |
+
### FR-009: Explain cluster
|
| 119 |
+
|
| 120 |
+
Status: **implemented**.
|
| 121 |
+
|
| 122 |
+
Cluster explanation includes representative, hit counts, confidence reasons, label distribution, outliers, and relevant constraints.
|
| 123 |
+
|
| 124 |
+
### FR-010: Predictive batch questions
|
| 125 |
+
|
| 126 |
+
Status: **partial**.
|
| 127 |
+
|
| 128 |
+
Suggestions exist for move/split/suppress patterns and can be accepted/rejected. They do not yet show exact diff previews.
|
| 129 |
+
|
| 130 |
+
### FR-011: Live counterfactual parameter preview
|
| 131 |
+
|
| 132 |
+
Status: **not implemented**.
|
| 133 |
+
|
| 134 |
+
### FR-012: Reconstruction-error correction
|
| 135 |
+
|
| 136 |
+
Status: **not implemented**.
|
| 137 |
+
|
| 138 |
+
## Non-functional requirements and status
|
| 139 |
+
|
| 140 |
+
### NFR-001: Reversibility
|
| 141 |
+
|
| 142 |
+
Status: **implemented for semantic edits** via undo stack.
|
| 143 |
+
|
| 144 |
+
### NFR-002: Explainability
|
| 145 |
+
|
| 146 |
+
Status: **partial**. Events, constraints, confidence reasons, and cluster explanations are visible. Suggestion diff previews are not yet implemented.
|
| 147 |
+
|
| 148 |
+
### NFR-003: Local recomputation first
|
| 149 |
+
|
| 150 |
+
Status: **partial**. Current recomputation is cheap semantic-state recomputation. True local feature reclustering remains open.
|
| 151 |
+
|
| 152 |
+
### NFR-004: Cached preprocessing
|
| 153 |
+
|
| 154 |
+
Status: **partial**. Stems/source loads are cached and hit audio is exported. Feature-vector caching is still open.
|
| 155 |
+
|
| 156 |
+
### NFR-005: Deterministic replay
|
| 157 |
+
|
| 158 |
+
Status: **partial**. Constraints/events persist and can be reloaded. A dedicated replay command/export pipeline is still open.
|
| 159 |
+
|
| 160 |
+
### NFR-006: No silent override of explicit user intent
|
| 161 |
+
|
| 162 |
+
Status: **implemented in current semantic layer**. Explicit moves, locks, suppressions, and favorites persist unless undone or explicitly changed.
|
docs/interactive-ux/PROGRESS.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Interactive UX progress
|
| 2 |
+
|
| 3 |
+
## Last updated
|
| 4 |
+
|
| 5 |
+
2026-05-12
|
| 6 |
+
|
| 7 |
+
## Current phase
|
| 8 |
+
|
| 9 |
+
Implementation is now in **Phase 1–3 foundation**: persistent state, constraints, events, confidence/review queue, and supervised cluster interactions are implemented at the semantic-state layer.
|
| 10 |
+
|
| 11 |
+
## Completed in this pass
|
| 12 |
+
|
| 13 |
+
| Item | Status | Notes |
|
| 14 |
+
|---|---|---|
|
| 15 |
+
| Add supplied docs to `docs/interactive-ux/` | done | All supplied Markdown docs were copied into this directory and aligned with the implemented project. |
|
| 16 |
+
| Persistent job state | done | `supervision_state.json` is created beside `manifest.json` for each completed run. |
|
| 17 |
+
| Hit/cluster state schema | done | Implemented in `supervised_state.py` using JSON-serializable dictionaries. |
|
| 18 |
+
| Event log | done | State mutations append `job.state.created`, `constraint.created`, `hit.moved`, `hit.pulled_out`, `cluster.locked`, `hit.suppressed`, `suggestion.created`, etc. |
|
| 19 |
+
| Constraint store | done | Supports `force-cluster`, `must-link`, `cannot-link`, `lock-cluster`, `suppress-pattern`, and `pin-representative`. |
|
| 20 |
+
| Confidence scoring | partial | Heuristic scores based on cluster size, label agreement, energy rank, representative/favorite/explicit state, suppression, and lock state. Feature-vector margin scoring is not implemented yet. |
|
| 21 |
+
| Outlier-first review queue | done | Backend computes `review_queue`; UI renders it and lets the user jump to the selected hit. |
|
| 22 |
+
| Move hit to cluster | done | Endpoint creates constraints and updates state; it also proposes similar move suggestions. |
|
| 23 |
+
| Pull hit into new cluster | done | Endpoint creates `cannot-link` and `force-cluster` constraints and a user cluster. |
|
| 24 |
+
| Lock cluster | done | Endpoint toggles lock state and records a lock constraint. |
|
| 25 |
+
| Suppress hit as bleed/noise | done | Endpoint creates `suppress-pattern`, marks the hit suppressed, and proposes similar suppressions. |
|
| 26 |
+
| Favorite sample / pin representative | partial | Endpoint supports `review` status `favorite`, records `pin-representative`, and updates the representative hit in semantic state. Audio artifact selection is not re-exported yet. |
|
| 27 |
+
| Suggestion inbox | partial | Open suggestions render in the UI and can be accepted/rejected. Suggestion generation is heuristic and limited to move/split/suppress patterns. |
|
| 28 |
+
| Cluster explanation drawer | done | Endpoint and UI show representative, confidence reasons, outliers, relevant constraints, and label distribution. |
|
| 29 |
+
| Undo | done | Last semantic edit can be restored using an undo snapshot stack. |
|
| 30 |
+
| Validation script | done | Added `scripts/test_interactive_supervision.py`. |
|
| 31 |
+
|
| 32 |
+
## Not yet implemented
|
| 33 |
+
|
| 34 |
+
- Real cached feature-vector store for local reclustering.
|
| 35 |
+
- Artifact re-export after semantic edits.
|
| 36 |
+
- Waveform click-to-add missed onset.
|
| 37 |
+
- Restore suppressed hit/batch restore.
|
| 38 |
+
- Real local neighborhood reclustering that changes assignments beyond explicit move/suggestion acceptance.
|
| 39 |
+
- Constraint violation detection and reporting.
|
| 40 |
+
- Predictive diff preview before accepting suggestions.
|
| 41 |
+
- Reconstruction-error-driven correction.
|
| 42 |
+
- Multi-resolution/hierarchical clusters.
|
| 43 |
+
- User correction profiles / teach mode across songs.
|
| 44 |
+
- Frontend TypeScript migration and browser automation tests.
|
| 45 |
+
|
| 46 |
+
## Current risks
|
| 47 |
+
|
| 48 |
+
| Risk | Impact | Mitigation |
|
| 49 |
+
|---|---|---|
|
| 50 |
+
| Semantic edits do not rewrite exports yet | User may expect moved/suppressed hits to affect ZIP/MIDI immediately | Next task should be edited-state export. |
|
| 51 |
+
| Confidence scores are heuristic | Review queue may sometimes prioritize the wrong hits | Add cached mel/transient features and margin-to-next-cluster scoring. |
|
| 52 |
+
| Suggestions are simple | May over-suggest or under-suggest | Keep them previewable, explicit, and undoable; never silently apply. |
|
| 53 |
+
| Locks are semantic only | Batch reruns do not yet replay constraints | Add deterministic replay/local recluster using constraints. |
|
| 54 |
+
| No browser tests | UI regressions are easy | Add Playwright or lightweight DOM tests. |
|
| 55 |
+
|
| 56 |
+
## Next implementation milestone
|
| 57 |
+
|
| 58 |
+
Milestone: **edited-state export and force-onset correction**.
|
| 59 |
+
|
| 60 |
+
Minimum deliverables:
|
| 61 |
+
|
| 62 |
+
1. Add `POST /api/jobs/{job_id}/export/supervised` that creates a ZIP/MIDI/manifest from `supervision_state.json`.
|
| 63 |
+
2. Exclude suppressed hits/clusters from the supervised export.
|
| 64 |
+
3. Honor favorite/pinned representatives in exported samples.
|
| 65 |
+
4. Add force-onset endpoint that slices a new hit from cached `stem.wav`.
|
| 66 |
+
5. Add waveform shift-click or add-onset mode in the UI.
|
| 67 |
+
6. Add tests proving semantic edits change the supervised export without rerunning stem extraction.
|
| 68 |
+
|
| 69 |
+
## Definition of done for the current foundation
|
| 70 |
+
|
| 71 |
+
This loop now works:
|
| 72 |
+
|
| 73 |
+
```text
|
| 74 |
+
analyze audio
|
| 75 |
+
→ inspect clusters
|
| 76 |
+
→ load semantic state
|
| 77 |
+
→ move one wrong hit
|
| 78 |
+
→ store constraints/events
|
| 79 |
+
→ see confidence/review queue update
|
| 80 |
+
→ lock corrected cluster
|
| 81 |
+
→ suppress bleed
|
| 82 |
+
→ inspect explanations and suggestions
|
| 83 |
+
→ undo semantic edits
|
| 84 |
+
→ reload the job and preserve explicit decisions
|
| 85 |
+
```
|
| 86 |
+
|
| 87 |
+
The remaining missing piece is that edited semantic state is not yet reflected in a regenerated sample pack.
|
docs/interactive-ux/README.md
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Interactive supervised extraction UX
|
| 2 |
+
|
| 3 |
+
This directory contains the supplied interactive-UX design documents, aligned with the implementation as of 2026-05-12.
|
| 4 |
+
|
| 5 |
+
The product direction is to turn drum sample extraction from a one-shot batch process into an interactive, supervised, progressively improving workflow. User edits should become constraints, preferences, and examples that improve clustering, review priority, labeling, and cleanup instead of only changing a visible table row.
|
| 6 |
+
|
| 7 |
+
## Current implementation status
|
| 8 |
+
|
| 9 |
+
The project now has a first supervised-editing foundation layered on top of the immutable extraction manifest:
|
| 10 |
+
|
| 11 |
+
- `supervised_state.py` persists `supervision_state.json` beside each completed run manifest.
|
| 12 |
+
- The state contains hits, clusters, confidence scores, review queue entries, constraints, events, suggestions, and undo snapshots.
|
| 13 |
+
- The FastAPI backend exposes state, move, pull-out, lock, suppress, review/favorite, suggestion, explanation, and undo endpoints.
|
| 14 |
+
- The browser UI includes an interactive supervision panel with a review queue, cluster board, suggestion inbox, constraint/event log, and cluster explanation drawer.
|
| 15 |
+
- The current supervised layer updates semantic state only. It does not yet rewrite sample WAVs, MIDI, reconstruction audio, or the ZIP after edits.
|
| 16 |
+
|
| 17 |
+
## Documents
|
| 18 |
+
|
| 19 |
+
| Document | Purpose | Alignment status |
|
| 20 |
+
|---|---|---|
|
| 21 |
+
| [`FEATURE_REQUIREMENTS.md`](./FEATURE_REQUIREMENTS.md) | Functional requirements for interaction-driven quality improvements | Updated with implemented/partial/not-started status |
|
| 22 |
+
| [`SCOPE.md`](./SCOPE.md) | MVP, v2, and research/backlog boundaries | Updated to reflect delivered MVP foundation |
|
| 23 |
+
| [`FEASIBILITY_MATRIX.md`](./FEASIBILITY_MATRIX.md) | Feasibility, complexity, effort, UX value, quality value table | Kept as design prioritization with implementation notes |
|
| 24 |
+
| [`TASKS.md`](./TASKS.md) | Implementation task breakdown with dependencies and acceptance criteria | Updated with current task statuses |
|
| 25 |
+
| [`PROGRESS.md`](./PROGRESS.md) | Current status, completed work, open work, next actions | Updated after this development pass |
|
| 26 |
+
| [`ARCHITECTURE_NOTES.md`](./ARCHITECTURE_NOTES.md) | Required state model and technical approach | Updated with actual module/API mapping |
|
| 27 |
+
|
| 28 |
+
## Principle
|
| 29 |
+
|
| 30 |
+
Good interaction:
|
| 31 |
+
|
| 32 |
+
```text
|
| 33 |
+
user moves one hit into another cluster
|
| 34 |
+
→ system creates force-cluster/must-link constraints
|
| 35 |
+
→ semantic state recomputes confidence and review priority
|
| 36 |
+
→ similar ambiguous hits are proposed as suggestions
|
| 37 |
+
→ the UI explains the state/event/constraint changes
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
Bad interaction:
|
| 41 |
+
|
| 42 |
+
```text
|
| 43 |
+
user moves one hit
|
| 44 |
+
→ only that one visible DOM row changes
|
| 45 |
+
→ future clustering ignores the correction
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
## Current build recommendation
|
| 49 |
+
|
| 50 |
+
Next implementation should close the gap between semantic edits and artifact output:
|
| 51 |
+
|
| 52 |
+
1. Re-export edited sample pack from `supervision_state.json` without rerunning Demucs/onset detection.
|
| 53 |
+
2. Add waveform click-to-add missed onset backed by `force-onset` constraints.
|
| 54 |
+
3. Add restore for suppressed hits and batch accepted suggestions.
|
| 55 |
+
4. Make move/pull-out trigger a real local reclustering pass using cached feature vectors.
|
| 56 |
+
5. Add visible diff previews before accepting grouped suggestions.
|
| 57 |
+
6. Add browser-level tests for the interactive supervision panel.
|
| 58 |
+
|
| 59 |
+
The strongest technical foundation remains constraint-aware clustering plus uncertainty-driven review. The current pass implements the persistent state and UI/API shell needed for that foundation.
|
docs/interactive-ux/SCOPE.md
ADDED
|
@@ -0,0 +1,173 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Interactive UX scope
|
| 2 |
+
|
| 3 |
+
## Scope model
|
| 4 |
+
|
| 5 |
+
Features are grouped by implementation risk and product leverage.
|
| 6 |
+
|
| 7 |
+
- **MVP**: should be built first; high value, technically feasible, foundational.
|
| 8 |
+
- **V2**: valuable after the core supervised editing loop exists.
|
| 9 |
+
- **Research/backlog**: plausible, but higher risk or dependent on stronger state/model infrastructure.
|
| 10 |
+
|
| 11 |
+
## MVP scope and status
|
| 12 |
+
|
| 13 |
+
### 1. Constraint-aware cluster editing
|
| 14 |
+
|
| 15 |
+
Status: **partial / foundation implemented**.
|
| 16 |
+
|
| 17 |
+
Implemented:
|
| 18 |
+
|
| 19 |
+
- Move hit/sample to cluster.
|
| 20 |
+
- Pull hit/sample out into a new cluster.
|
| 21 |
+
- Create `must-link`, `cannot-link`, and `force-cluster` constraints.
|
| 22 |
+
- Preserve explicit edits in `supervision_state.json` across reload.
|
| 23 |
+
- Generate basic suggestions after edits.
|
| 24 |
+
|
| 25 |
+
Remaining:
|
| 26 |
+
|
| 27 |
+
- True local neighborhood reclustering from cached feature vectors.
|
| 28 |
+
- Constraint violation detection.
|
| 29 |
+
- Edited artifact re-export.
|
| 30 |
+
|
| 31 |
+
### 2. Lock confirmed clusters
|
| 32 |
+
|
| 33 |
+
Status: **implemented**.
|
| 34 |
+
|
| 35 |
+
Implemented:
|
| 36 |
+
|
| 37 |
+
- Lock/unlock cluster.
|
| 38 |
+
- Persist lock state.
|
| 39 |
+
- Show lock state in the cluster board and target cluster control.
|
| 40 |
+
- Confidence scoring accounts for locked state.
|
| 41 |
+
|
| 42 |
+
Remaining:
|
| 43 |
+
|
| 44 |
+
- Prevent future batch reruns from violating locked cluster identity through deterministic replay.
|
| 45 |
+
|
| 46 |
+
### 3. Outlier-first review queue
|
| 47 |
+
|
| 48 |
+
Status: **implemented**.
|
| 49 |
+
|
| 50 |
+
Implemented:
|
| 51 |
+
|
| 52 |
+
- Confidence score per hit and cluster.
|
| 53 |
+
- Review list sorted by uncertainty priority.
|
| 54 |
+
- Quick actions: accept, favorite, move, pull out, suppress.
|
| 55 |
+
|
| 56 |
+
Remaining:
|
| 57 |
+
|
| 58 |
+
- Split/restore/batch actions.
|
| 59 |
+
- Expected-impact scoring using feature margin and reconstruction contribution.
|
| 60 |
+
|
| 61 |
+
### 4. Confidence-weighted UI
|
| 62 |
+
|
| 63 |
+
Status: **implemented**.
|
| 64 |
+
|
| 65 |
+
Implemented:
|
| 66 |
+
|
| 67 |
+
- Confidence per hit and cluster.
|
| 68 |
+
- Low-confidence row emphasis.
|
| 69 |
+
- Suppressed rows recede.
|
| 70 |
+
- State summary badges.
|
| 71 |
+
|
| 72 |
+
### 5. Click-to-add missed onset
|
| 73 |
+
|
| 74 |
+
Status: **not implemented**.
|
| 75 |
+
|
| 76 |
+
Current waveform clicks audition existing hits. Add-onset mode is the next direct correction primitive.
|
| 77 |
+
|
| 78 |
+
### 6. Bleed suppression examples
|
| 79 |
+
|
| 80 |
+
Status: **partial**.
|
| 81 |
+
|
| 82 |
+
Implemented:
|
| 83 |
+
|
| 84 |
+
- Mark selected hit as bleed/noise.
|
| 85 |
+
- Store `suppress-pattern`.
|
| 86 |
+
- Generate similar suppression suggestions.
|
| 87 |
+
|
| 88 |
+
Remaining:
|
| 89 |
+
|
| 90 |
+
- Region brush.
|
| 91 |
+
- Restore suppressed hits.
|
| 92 |
+
- Exclude suppressed hits from supervised export.
|
| 93 |
+
|
| 94 |
+
### 7. Explain this cluster
|
| 95 |
+
|
| 96 |
+
Status: **implemented**.
|
| 97 |
+
|
| 98 |
+
Implemented:
|
| 99 |
+
|
| 100 |
+
- Representative ID.
|
| 101 |
+
- Members/outliers.
|
| 102 |
+
- Confidence reasons.
|
| 103 |
+
- Label distribution.
|
| 104 |
+
- Relevant constraints.
|
| 105 |
+
- Locked/suppressed counts.
|
| 106 |
+
|
| 107 |
+
## V2 scope
|
| 108 |
+
|
| 109 |
+
### 1. Predictive batch questions
|
| 110 |
+
|
| 111 |
+
Status: **partial**.
|
| 112 |
+
|
| 113 |
+
Basic suggestions exist. Exact diff previews, grouped approval UX, and richer reasoning are not complete.
|
| 114 |
+
|
| 115 |
+
### 2. Live counterfactual previews
|
| 116 |
+
|
| 117 |
+
Status: **not implemented**.
|
| 118 |
+
|
| 119 |
+
### 3. Auto-clean this family
|
| 120 |
+
|
| 121 |
+
Status: **not implemented**.
|
| 122 |
+
|
| 123 |
+
### 4. Multi-resolution semantic clustering
|
| 124 |
+
|
| 125 |
+
Status: **not implemented**.
|
| 126 |
+
|
| 127 |
+
### 5. Run-to-run preference memory
|
| 128 |
+
|
| 129 |
+
Status: **not implemented**.
|
| 130 |
+
|
| 131 |
+
## Research/backlog scope
|
| 132 |
+
|
| 133 |
+
Unchanged from design intent:
|
| 134 |
+
|
| 135 |
+
- Reconstruction-error-driven correction.
|
| 136 |
+
- Drag clusters in semantic space.
|
| 137 |
+
- Cluster gravity / physics metaphor.
|
| 138 |
+
- Context-aware classification.
|
| 139 |
+
- Teach mode across songs.
|
| 140 |
+
|
| 141 |
+
## Explicitly out of scope for MVP
|
| 142 |
+
|
| 143 |
+
- Realtime Demucs.
|
| 144 |
+
- Training neural models from scratch.
|
| 145 |
+
- Cloud multi-user collaboration.
|
| 146 |
+
- DAW plugin integration.
|
| 147 |
+
- Full Ableton/Logic export format support.
|
| 148 |
+
- Automatic perfect drum transcription claims.
|
| 149 |
+
|
| 150 |
+
## MVP acceptance criteria status
|
| 151 |
+
|
| 152 |
+
Target loop:
|
| 153 |
+
|
| 154 |
+
```text
|
| 155 |
+
cached stem is analyzed
|
| 156 |
+
→ clusters appear
|
| 157 |
+
→ uncertain items are prioritized
|
| 158 |
+
→ user moves one wrong hit
|
| 159 |
+
→ system stores a constraint
|
| 160 |
+
→ affected state recomputes
|
| 161 |
+
→ related mistakes are suggested
|
| 162 |
+
→ locked clusters remain stable in state
|
| 163 |
+
→ semantic decisions reload reproducibly
|
| 164 |
+
```
|
| 165 |
+
|
| 166 |
+
Status: **mostly achieved for semantic state**.
|
| 167 |
+
|
| 168 |
+
Not achieved yet:
|
| 169 |
+
|
| 170 |
+
```text
|
| 171 |
+
→ edited state can be exported reproducibly as updated WAV/MIDI/ZIP artifacts
|
| 172 |
+
→ local feature reclustering updates related hits without manually accepting suggestions
|
| 173 |
+
```
|
docs/interactive-ux/TASKS.md
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Interactive UX tasks
|
| 2 |
+
|
| 3 |
+
## Status legend
|
| 4 |
+
|
| 5 |
+
| Status | Meaning |
|
| 6 |
+
|---|---|
|
| 7 |
+
| `todo` | Not started |
|
| 8 |
+
| `doing` | In progress |
|
| 9 |
+
| `partial` | Implemented as a useful first version but not complete |
|
| 10 |
+
| `done` | Completed for the current architecture |
|
| 11 |
+
| `blocked` | Waiting on prerequisite decision/work |
|
| 12 |
+
|
| 13 |
+
## Phase 0: Documentation and design capture
|
| 14 |
+
|
| 15 |
+
| ID | Task | Status | Acceptance criteria |
|
| 16 |
+
|---|---|---|---|
|
| 17 |
+
| UX-000 | Capture feature requirements | done | Requirements documented in `FEATURE_REQUIREMENTS.md` |
|
| 18 |
+
| UX-001 | Capture scope boundaries | done | MVP/V2/backlog documented in `SCOPE.md` |
|
| 19 |
+
| UX-002 | Capture feasibility matrix | done | Scores documented in `FEASIBILITY_MATRIX.md` |
|
| 20 |
+
| UX-003 | Capture implementation tasks | done | Task backlog documented here |
|
| 21 |
+
| UX-004 | Capture progress | done | Progress documented in `PROGRESS.md` |
|
| 22 |
+
| UX-005 | Align supplied docs with implemented project | done | All documents updated to reflect current code/API/UI behavior |
|
| 23 |
+
|
| 24 |
+
## Phase 1: State model and persistence
|
| 25 |
+
|
| 26 |
+
| ID | Task | Status | Acceptance criteria |
|
| 27 |
+
|---|---|---|---|
|
| 28 |
+
| UX-101 | Define persisted job state schema | done | `supervised_state.py` creates `supervision_state.json` with hits, clusters, constraints, events, suggestions, and undo state |
|
| 29 |
+
| UX-102 | Add event log | done | User/system changes append events and the event log is stored with job output |
|
| 30 |
+
| UX-103 | Add constraint store | done | `must-link`, `cannot-link`, `force-cluster`, `lock-cluster`, `suppress-pattern`, and `pin-representative` are saved |
|
| 31 |
+
| UX-104 | Add artifact/cache references | partial | State stores hit audio file refs and manifest fingerprint; feature refs and content-addressed feature cache are not implemented yet |
|
| 32 |
+
| UX-105 | Add deterministic replay command | todo | A job can be regenerated from source digest, params, and constraints |
|
| 33 |
+
| UX-106 | Add semantic undo stack | done | `POST /api/jobs/{job_id}/undo` restores the previous semantic state snapshot |
|
| 34 |
+
|
| 35 |
+
## Phase 2: Confidence and review queue
|
| 36 |
+
|
| 37 |
+
| ID | Task | Status | Acceptance criteria |
|
| 38 |
+
|---|---|---|---|
|
| 39 |
+
| UX-201 | Compute hit assignment confidence | partial | Each hit has confidence from cluster confidence, label agreement, energy rank, representative/explicit/suppressed state; feature-vector margin is not implemented |
|
| 40 |
+
| UX-202 | Compute cluster confidence | partial | Each cluster has confidence from purity, size, representative, lock state; true feature cohesion/stability is not implemented |
|
| 41 |
+
| UX-203 | Add uncertainty ranking | done | Backend returns `review_queue` sorted by priority |
|
| 42 |
+
| UX-204 | Add low-confidence UI emphasis | done | Hit rows and review queue emphasize low-confidence and suppressed items |
|
| 43 |
+
| UX-205 | Add review quick actions | partial | Accept, favorite, move, pull-out, suppress are available; split/restore/batch actions are still incomplete |
|
| 44 |
+
|
| 45 |
+
## Phase 3: Constraint-aware editing
|
| 46 |
+
|
| 47 |
+
| ID | Task | Status | Acceptance criteria |
|
| 48 |
+
|---|---|---|---|
|
| 49 |
+
| UX-301 | Move hit to cluster endpoint | done | Moving a hit creates `force-cluster` and usually `must-link` constraints |
|
| 50 |
+
| UX-302 | Pull hit into new cluster endpoint | done | Pulling a hit creates a new user cluster plus `cannot-link`/`force-cluster` constraints |
|
| 51 |
+
| UX-303 | Lock cluster endpoint | done | Locked cluster state persists and is shown in the UI |
|
| 52 |
+
| UX-304 | Local neighborhood recomputation | partial | Semantic state and confidence are recomputed; true cached feature-neighborhood reclustering is not implemented |
|
| 53 |
+
| UX-305 | Constraint violation detection | todo | Backend reports attempted changes that violate user constraints |
|
| 54 |
+
| UX-306 | Undo last interaction | done | User can reverse the previous semantic edit |
|
| 55 |
+
| UX-307 | Suggest similar moves from a hit move | partial | Heuristic suggestions are generated using label, centroid, and energy similarity |
|
| 56 |
+
|
| 57 |
+
## Phase 4: Missed onset and bleed interactions
|
| 58 |
+
|
| 59 |
+
| ID | Task | Status | Acceptance criteria |
|
| 60 |
+
|---|---|---|---|
|
| 61 |
+
| UX-401 | Add force-onset endpoint | todo | User can add onset by time; hit is sliced/classified/clustered |
|
| 62 |
+
| UX-402 | Add waveform click interaction | partial | Existing waveform clicks audition nearest hit; add-onset mode is not implemented |
|
| 63 |
+
| UX-403 | Add bleed suppression endpoint | done | User can mark hit as bleed/noise and create a suppress-pattern example |
|
| 64 |
+
| UX-404 | Implement similar-false-positive search | partial | Heuristic suppress suggestions are generated from energy/centroid/label similarity |
|
| 65 |
+
| UX-405 | Add suppression restore | todo | Suppressed hits can be restored individually or in batches |
|
| 66 |
+
|
| 67 |
+
## Phase 5: Suggestions and explanations
|
| 68 |
+
|
| 69 |
+
| ID | Task | Status | Acceptance criteria |
|
| 70 |
+
|---|---|---|---|
|
| 71 |
+
| UX-501 | Add suggestion model | done | Suggestions have type, confidence, reason, preview count, status, accept/reject state |
|
| 72 |
+
| UX-502 | Generate move suggestions from cluster edits | partial | Basic suggestions generated after move edits |
|
| 73 |
+
| UX-503 | Generate split suggestions from cannot-link edits | partial | Basic split suggestions generated after pull-out edits |
|
| 74 |
+
| UX-504 | Generate suppression suggestions | partial | Basic suppression suggestions generated after suppress edits |
|
| 75 |
+
| UX-505 | Add suggestion inbox UI | done | Suggestions are visible and can be accepted/rejected |
|
| 76 |
+
| UX-506 | Add explain-cluster endpoint | done | Backend returns representative, confidence reasons, outliers, constraints, label distribution |
|
| 77 |
+
| UX-507 | Add explanation drawer UI | done | UI renders explanation JSON for the selected cluster |
|
| 78 |
+
| UX-508 | Add diff previews before suggestion acceptance | todo | Suggestions show exact before/after cluster membership changes before acceptance |
|
| 79 |
+
|
| 80 |
+
## Phase 6: Counterfactuals and advanced quality features
|
| 81 |
+
|
| 82 |
+
| ID | Task | Status | Acceptance criteria |
|
| 83 |
+
|---|---|---|---|
|
| 84 |
+
| UX-601 | Add parameter diff estimator | todo | UI can preview approximate effect of parameter changes |
|
| 85 |
+
| UX-602 | Add cached local parameter recompute | todo | Parameter changes reuse hit features where possible |
|
| 86 |
+
| UX-603 | Add reconstruction error map | todo | Backend computes original-vs-reconstruction mismatch by region |
|
| 87 |
+
| UX-604 | Add reconstruction correction workflow | todo | User can select bad region and get likely causes/fixes |
|
| 88 |
+
| UX-605 | Add multi-resolution cluster hierarchy | todo | Clusters can be browsed coarse-to-fine |
|
| 89 |
+
|
| 90 |
+
## Phase 7: Preference memory / teach mode
|
| 91 |
+
|
| 92 |
+
| ID | Task | Status | Acceptance criteria |
|
| 93 |
+
|---|---|---|---|
|
| 94 |
+
| UX-701 | Persist user correction profiles | todo | Reusable preference profile stores accepted correction patterns |
|
| 95 |
+
| UX-702 | Apply profile to new jobs | todo | New jobs can opt into prior preferences |
|
| 96 |
+
| UX-703 | Add profile safety controls | todo | User can inspect, disable, delete, and scope learned preferences |
|
| 97 |
+
| UX-704 | Evaluate profile impact | todo | Benchmarks compare profile-on vs profile-off results |
|
| 98 |
+
|
| 99 |
+
## Current recommended next task
|
| 100 |
+
|
| 101 |
+
Start with `UX-401` plus supervised export.
|
| 102 |
+
|
| 103 |
+
Reason:
|
| 104 |
+
|
| 105 |
+
The project now has a replayable state/events/constraints foundation. The largest UX gap is that semantic edits do not yet regenerate edited artifacts. Force-onset is the next direct correction primitive after move/pull/lock/suppress.
|
scripts/test_interactive_supervision.py
ADDED
|
@@ -0,0 +1,112 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Smoke-test manifest-backed interactive supervision endpoints."""
|
| 3 |
+
|
| 4 |
+
from __future__ import annotations
|
| 5 |
+
|
| 6 |
+
import io
|
| 7 |
+
import json
|
| 8 |
+
import sys
|
| 9 |
+
import time
|
| 10 |
+
from pathlib import Path
|
| 11 |
+
from urllib.parse import quote
|
| 12 |
+
|
| 13 |
+
import soundfile as sf
|
| 14 |
+
from fastapi.testclient import TestClient
|
| 15 |
+
|
| 16 |
+
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
| 17 |
+
|
| 18 |
+
from app import app # noqa: E402
|
| 19 |
+
from synth_generator import generate_test_song # noqa: E402
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def wait_for_job(client: TestClient, job_id: str) -> dict:
|
| 23 |
+
for _ in range(80):
|
| 24 |
+
payload = client.get(f"/api/jobs/{job_id}").json()
|
| 25 |
+
if payload["status"] in {"complete", "error"}:
|
| 26 |
+
return payload
|
| 27 |
+
time.sleep(0.15)
|
| 28 |
+
raise TimeoutError(job_id)
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def post_json(client: TestClient, path: str, body: dict | None = None) -> dict:
|
| 32 |
+
response = client.post(path, json=body or {})
|
| 33 |
+
response.raise_for_status()
|
| 34 |
+
return response.json()
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def main() -> int:
|
| 38 |
+
song = generate_test_song(pattern_name="funk", bars=1, bpm=124, add_bass=False)
|
| 39 |
+
buf = io.BytesIO()
|
| 40 |
+
sf.write(buf, song.drums_only, song.sr, format="WAV")
|
| 41 |
+
buf.seek(0)
|
| 42 |
+
|
| 43 |
+
client = TestClient(app)
|
| 44 |
+
response = client.post(
|
| 45 |
+
"/api/jobs",
|
| 46 |
+
files={"file": ("interactive.wav", buf, "audio/wav")},
|
| 47 |
+
data={"params": json.dumps({"stem": "all", "clustering_mode": "online_preview", "target_min": 3, "target_max": 10})},
|
| 48 |
+
)
|
| 49 |
+
response.raise_for_status()
|
| 50 |
+
job_id = response.json()["id"]
|
| 51 |
+
job = wait_for_job(client, job_id)
|
| 52 |
+
assert job["status"] == "complete", job.get("error")
|
| 53 |
+
|
| 54 |
+
state = client.get(f"/api/jobs/{job_id}/state").json()
|
| 55 |
+
assert state["summary"]["hit_count"] > 0
|
| 56 |
+
assert state["summary"]["cluster_count"] > 0
|
| 57 |
+
assert state["review_queue"], "expected uncertainty review queue"
|
| 58 |
+
|
| 59 |
+
hit_id = state["hits"][0]["id"]
|
| 60 |
+
cluster_id = state["clusters"][0]["id"]
|
| 61 |
+
q_hit = quote(hit_id, safe="")
|
| 62 |
+
q_cluster = quote(cluster_id, safe="")
|
| 63 |
+
|
| 64 |
+
state = post_json(client, f"/api/jobs/{job_id}/clusters/{q_cluster}/lock", {"locked": True})
|
| 65 |
+
assert state["summary"]["locked_cluster_count"] >= 1
|
| 66 |
+
|
| 67 |
+
state = post_json(client, f"/api/jobs/{job_id}/hits/{q_hit}/review", {"status": "favorite"})
|
| 68 |
+
assert state["summary"]["constraint_count"] >= 1
|
| 69 |
+
|
| 70 |
+
explanation = client.get(f"/api/jobs/{job_id}/explain/cluster/{q_cluster}")
|
| 71 |
+
explanation.raise_for_status()
|
| 72 |
+
assert explanation.json()["cluster_id"] == cluster_id
|
| 73 |
+
|
| 74 |
+
state = post_json(client, f"/api/jobs/{job_id}/hits/{q_hit}/pull-out", {})
|
| 75 |
+
assert state["summary"]["cluster_count"] >= 1
|
| 76 |
+
assert state["summary"]["undo_available"] is True
|
| 77 |
+
assert any(c["type"] in {"cannot-link", "force-cluster"} for c in state["constraints"])
|
| 78 |
+
|
| 79 |
+
state = post_json(client, f"/api/jobs/{job_id}/undo", {})
|
| 80 |
+
assert state["summary"]["hit_count"] > 0
|
| 81 |
+
|
| 82 |
+
if len(state["clusters"]) > 1:
|
| 83 |
+
target = next(c for c in state["clusters"] if c["id"] != state["hits"][0]["cluster_id"])
|
| 84 |
+
state = post_json(
|
| 85 |
+
client,
|
| 86 |
+
f"/api/jobs/{job_id}/hits/{q_hit}/move",
|
| 87 |
+
{"target_cluster_id": target["id"]},
|
| 88 |
+
)
|
| 89 |
+
assert any(c["type"] == "force-cluster" for c in state["constraints"])
|
| 90 |
+
|
| 91 |
+
if len(state["hits"]) > 1:
|
| 92 |
+
suppress_hit = quote(state["hits"][1]["id"], safe="")
|
| 93 |
+
state = post_json(client, f"/api/jobs/{job_id}/hits/{suppress_hit}/suppress", {"reason": "bleed"})
|
| 94 |
+
assert state["summary"]["suppressed_hit_count"] >= 1
|
| 95 |
+
|
| 96 |
+
suggestions = client.get(f"/api/jobs/{job_id}/suggestions")
|
| 97 |
+
suggestions.raise_for_status()
|
| 98 |
+
|
| 99 |
+
print(json.dumps({
|
| 100 |
+
"status": "ok",
|
| 101 |
+
"job_id": job_id,
|
| 102 |
+
"hit_count": state["summary"]["hit_count"],
|
| 103 |
+
"cluster_count": state["summary"]["cluster_count"],
|
| 104 |
+
"constraints": state["summary"]["constraint_count"],
|
| 105 |
+
"events": state["summary"]["event_count"],
|
| 106 |
+
"suggestions": state["summary"]["open_suggestion_count"],
|
| 107 |
+
}, indent=2))
|
| 108 |
+
return 0
|
| 109 |
+
|
| 110 |
+
|
| 111 |
+
if __name__ == "__main__":
|
| 112 |
+
raise SystemExit(main())
|
supervised_state.py
ADDED
|
@@ -0,0 +1,675 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Persistent supervised-editing state for interactive extraction jobs.
|
| 3 |
+
|
| 4 |
+
The extraction pipeline produces immutable audio artifacts and a batch manifest.
|
| 5 |
+
This module layers replayable semantic state on top of that manifest: hits,
|
| 6 |
+
clusters, constraints, events, suggestions, confidence, and undo snapshots.
|
| 7 |
+
|
| 8 |
+
The first implementation intentionally avoids rewriting audio artifacts. It makes
|
| 9 |
+
supervised edits cheap, explicit, inspectable, and reproducible, then leaves
|
| 10 |
+
artifact re-export as a later step.
|
| 11 |
+
"""
|
| 12 |
+
|
| 13 |
+
from __future__ import annotations
|
| 14 |
+
|
| 15 |
+
import copy
|
| 16 |
+
import json
|
| 17 |
+
import math
|
| 18 |
+
import time
|
| 19 |
+
import uuid
|
| 20 |
+
from pathlib import Path
|
| 21 |
+
from typing import Any, Callable
|
| 22 |
+
|
| 23 |
+
STATE_VERSION = "interactive-state-v1"
|
| 24 |
+
STATE_FILENAME = "supervision_state.json"
|
| 25 |
+
MAX_UNDO = 30
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def now() -> float:
|
| 29 |
+
return round(time.time(), 6)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def state_path(output_dir: str | Path) -> Path:
|
| 33 |
+
return Path(output_dir) / STATE_FILENAME
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def manifest_path(output_dir: str | Path) -> Path:
|
| 37 |
+
return Path(output_dir) / "manifest.json"
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def load_manifest(output_dir: str | Path) -> dict[str, Any]:
|
| 41 |
+
path = manifest_path(output_dir)
|
| 42 |
+
if not path.exists():
|
| 43 |
+
raise FileNotFoundError(f"manifest.json not found in {Path(output_dir)}")
|
| 44 |
+
return json.loads(path.read_text(encoding="utf-8"))
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def _hit_id(hit: dict[str, Any]) -> str:
|
| 48 |
+
return f"hit:{int(hit.get('index', 0)):05d}"
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def _cluster_id(raw: Any) -> str:
|
| 52 |
+
return f"cluster:{raw}"
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def _base_label(label: str) -> str:
|
| 56 |
+
text = str(label or "other")
|
| 57 |
+
return text.rsplit("_", 1)[0] if "_" in text else text
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def _new_id(prefix: str) -> str:
|
| 61 |
+
return f"{prefix}:{uuid.uuid4().hex[:10]}"
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def _safe_float(value: Any, default: float = 0.0) -> float:
|
| 65 |
+
try:
|
| 66 |
+
out = float(value)
|
| 67 |
+
if math.isfinite(out):
|
| 68 |
+
return out
|
| 69 |
+
except Exception:
|
| 70 |
+
pass
|
| 71 |
+
return default
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def _snapshot(state: dict[str, Any]) -> dict[str, Any]:
|
| 75 |
+
snap = copy.deepcopy(state)
|
| 76 |
+
snap["undo_stack"] = []
|
| 77 |
+
return snap
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def _push_undo(state: dict[str, Any]) -> None:
|
| 81 |
+
stack = list(state.get("undo_stack") or [])
|
| 82 |
+
stack.append(_snapshot(state))
|
| 83 |
+
del stack[:-MAX_UNDO]
|
| 84 |
+
state["undo_stack"] = stack
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def _event(state: dict[str, Any], event_type: str, payload: dict[str, Any] | None = None, source: str = "system") -> dict[str, Any]:
|
| 88 |
+
event = {
|
| 89 |
+
"id": _new_id("event"),
|
| 90 |
+
"type": event_type,
|
| 91 |
+
"source": source,
|
| 92 |
+
"created_at": now(),
|
| 93 |
+
"payload": payload or {},
|
| 94 |
+
}
|
| 95 |
+
state.setdefault("events", []).append(event)
|
| 96 |
+
return event
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def _constraint(state: dict[str, Any], constraint_type: str, payload: dict[str, Any], source: str = "user") -> dict[str, Any]:
|
| 100 |
+
constraint = {
|
| 101 |
+
"id": _new_id("constraint"),
|
| 102 |
+
"type": constraint_type,
|
| 103 |
+
"source": source,
|
| 104 |
+
"created_at": now(),
|
| 105 |
+
**payload,
|
| 106 |
+
}
|
| 107 |
+
state.setdefault("constraints", []).append(constraint)
|
| 108 |
+
_event(state, "constraint.created", {"constraint_id": constraint["id"], "type": constraint_type}, source=source)
|
| 109 |
+
return constraint
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
def _write_state(output_dir: str | Path, state: dict[str, Any]) -> dict[str, Any]:
|
| 113 |
+
state["updated_at"] = now()
|
| 114 |
+
path = state_path(output_dir)
|
| 115 |
+
path.write_text(json.dumps(state, indent=2, sort_keys=True), encoding="utf-8")
|
| 116 |
+
return state
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
def _cluster_label_for_hit(hit: dict[str, Any]) -> str:
|
| 120 |
+
return str(hit.get("cluster_label") or f"{hit.get('label', 'other')}_{hit.get('cluster_id', '0')}")
|
| 121 |
+
|
| 122 |
+
|
| 123 |
+
def build_initial_state(job_id: str, manifest: dict[str, Any]) -> dict[str, Any]:
|
| 124 |
+
hits_by_id: dict[str, dict[str, Any]] = {}
|
| 125 |
+
clusters: dict[str, dict[str, Any]] = {}
|
| 126 |
+
|
| 127 |
+
raw_hits = list(manifest.get("hits") or [])
|
| 128 |
+
if not raw_hits:
|
| 129 |
+
# Older manifests may only contain samples. Keep state valid even then.
|
| 130 |
+
raw_hits = []
|
| 131 |
+
|
| 132 |
+
for hit in raw_hits:
|
| 133 |
+
hid = _hit_id(hit)
|
| 134 |
+
cid = _cluster_id(hit.get("cluster_id", "unclustered"))
|
| 135 |
+
cluster_label = _cluster_label_for_hit(hit)
|
| 136 |
+
hits_by_id[hid] = {
|
| 137 |
+
"id": hid,
|
| 138 |
+
"index": int(hit.get("index", len(hits_by_id))),
|
| 139 |
+
"label": str(hit.get("label") or "other"),
|
| 140 |
+
"cluster_id": cid,
|
| 141 |
+
"original_cluster_id": cid,
|
| 142 |
+
"cluster_label": cluster_label,
|
| 143 |
+
"onset_sec": _safe_float(hit.get("onset_sec")),
|
| 144 |
+
"duration_ms": _safe_float(hit.get("duration_ms")),
|
| 145 |
+
"rms_energy": _safe_float(hit.get("rms_energy")),
|
| 146 |
+
"spectral_centroid_hz": _safe_float(hit.get("spectral_centroid_hz")),
|
| 147 |
+
"file": hit.get("file"),
|
| 148 |
+
"is_representative": bool(hit.get("is_representative")),
|
| 149 |
+
"source": "detected",
|
| 150 |
+
"suppressed": False,
|
| 151 |
+
"favorite": False,
|
| 152 |
+
"review_status": "unreviewed",
|
| 153 |
+
"confidence": 0.0,
|
| 154 |
+
"confidence_reasons": [],
|
| 155 |
+
"explicit": False,
|
| 156 |
+
}
|
| 157 |
+
clusters.setdefault(
|
| 158 |
+
cid,
|
| 159 |
+
{
|
| 160 |
+
"id": cid,
|
| 161 |
+
"label": cluster_label,
|
| 162 |
+
"classification": _base_label(cluster_label),
|
| 163 |
+
"hit_ids": [],
|
| 164 |
+
"representative_hit_id": None,
|
| 165 |
+
"locked": False,
|
| 166 |
+
"user_named": False,
|
| 167 |
+
"confidence": 0.0,
|
| 168 |
+
"confidence_reasons": [],
|
| 169 |
+
"suppressed_count": 0,
|
| 170 |
+
"original_id": cid,
|
| 171 |
+
},
|
| 172 |
+
)["hit_ids"].append(hid)
|
| 173 |
+
if bool(hit.get("is_representative")):
|
| 174 |
+
clusters[cid]["representative_hit_id"] = hid
|
| 175 |
+
|
| 176 |
+
for cid, cluster in clusters.items():
|
| 177 |
+
if cluster["representative_hit_id"] is None and cluster["hit_ids"]:
|
| 178 |
+
cluster["representative_hit_id"] = cluster["hit_ids"][0]
|
| 179 |
+
|
| 180 |
+
state = {
|
| 181 |
+
"version": STATE_VERSION,
|
| 182 |
+
"job_id": job_id,
|
| 183 |
+
"created_at": now(),
|
| 184 |
+
"updated_at": now(),
|
| 185 |
+
"manifest_fingerprint": _manifest_fingerprint(manifest),
|
| 186 |
+
"hits": hits_by_id,
|
| 187 |
+
"clusters": clusters,
|
| 188 |
+
"constraints": [],
|
| 189 |
+
"events": [],
|
| 190 |
+
"suggestions": [],
|
| 191 |
+
"undo_stack": [],
|
| 192 |
+
"counters": {"user_clusters": 0},
|
| 193 |
+
}
|
| 194 |
+
recompute_scores(state)
|
| 195 |
+
_event(
|
| 196 |
+
state,
|
| 197 |
+
"job.state.created",
|
| 198 |
+
{
|
| 199 |
+
"hit_count": len(hits_by_id),
|
| 200 |
+
"cluster_count": len(clusters),
|
| 201 |
+
"manifest_fingerprint": state["manifest_fingerprint"],
|
| 202 |
+
},
|
| 203 |
+
)
|
| 204 |
+
return state
|
| 205 |
+
|
| 206 |
+
|
| 207 |
+
def _manifest_fingerprint(manifest: dict[str, Any]) -> str:
|
| 208 |
+
import hashlib
|
| 209 |
+
|
| 210 |
+
payload = {
|
| 211 |
+
"params": manifest.get("params"),
|
| 212 |
+
"hit_count": manifest.get("hit_count"),
|
| 213 |
+
"cluster_count": manifest.get("cluster_count"),
|
| 214 |
+
"files": manifest.get("files"),
|
| 215 |
+
"hits": [
|
| 216 |
+
{
|
| 217 |
+
"index": h.get("index"),
|
| 218 |
+
"cluster_id": h.get("cluster_id"),
|
| 219 |
+
"file": h.get("file"),
|
| 220 |
+
"onset_sec": h.get("onset_sec"),
|
| 221 |
+
}
|
| 222 |
+
for h in manifest.get("hits", [])
|
| 223 |
+
],
|
| 224 |
+
}
|
| 225 |
+
return hashlib.sha256(json.dumps(payload, sort_keys=True).encode("utf-8")).hexdigest()
|
| 226 |
+
|
| 227 |
+
|
| 228 |
+
def load_or_create_state(job_id: str, output_dir: str | Path) -> dict[str, Any]:
|
| 229 |
+
path = state_path(output_dir)
|
| 230 |
+
if path.exists():
|
| 231 |
+
state = json.loads(path.read_text(encoding="utf-8"))
|
| 232 |
+
if state.get("version") != STATE_VERSION:
|
| 233 |
+
raise ValueError(f"Unsupported supervision state version: {state.get('version')}")
|
| 234 |
+
return state
|
| 235 |
+
manifest = load_manifest(output_dir)
|
| 236 |
+
state = build_initial_state(job_id, manifest)
|
| 237 |
+
return _write_state(output_dir, state)
|
| 238 |
+
|
| 239 |
+
|
| 240 |
+
def _active_hits(state: dict[str, Any], cluster: dict[str, Any]) -> list[dict[str, Any]]:
|
| 241 |
+
hits = state.get("hits", {})
|
| 242 |
+
return [hits[hid] for hid in cluster.get("hit_ids", []) if hid in hits and not hits[hid].get("suppressed")]
|
| 243 |
+
|
| 244 |
+
|
| 245 |
+
def recompute_scores(state: dict[str, Any]) -> None:
|
| 246 |
+
hits = state.get("hits", {})
|
| 247 |
+
clusters = state.get("clusters", {})
|
| 248 |
+
energies = sorted(_safe_float(hit.get("rms_energy")) for hit in hits.values())
|
| 249 |
+
|
| 250 |
+
def energy_rank(value: float) -> float:
|
| 251 |
+
if not energies:
|
| 252 |
+
return 0.5
|
| 253 |
+
less = sum(1 for item in energies if item <= value)
|
| 254 |
+
return less / max(1, len(energies))
|
| 255 |
+
|
| 256 |
+
for cluster in clusters.values():
|
| 257 |
+
members = [hits[hid] for hid in cluster.get("hit_ids", []) if hid in hits]
|
| 258 |
+
active = [hit for hit in members if not hit.get("suppressed")]
|
| 259 |
+
if not members:
|
| 260 |
+
confidence = 0.15
|
| 261 |
+
reasons = ["empty cluster"]
|
| 262 |
+
else:
|
| 263 |
+
labels: dict[str, int] = {}
|
| 264 |
+
for hit in active:
|
| 265 |
+
labels[hit.get("label", "other")] = labels.get(hit.get("label", "other"), 0) + 1
|
| 266 |
+
majority = max(labels.values()) if labels else 0
|
| 267 |
+
purity = majority / max(1, len(active))
|
| 268 |
+
size_score = min(1.0, math.log2(len(active) + 1) / 4.0)
|
| 269 |
+
representative_bonus = 0.12 if cluster.get("representative_hit_id") in cluster.get("hit_ids", []) else 0.0
|
| 270 |
+
lock_bonus = 0.12 if cluster.get("locked") else 0.0
|
| 271 |
+
confidence = (0.42 * purity) + (0.34 * size_score) + representative_bonus + lock_bonus
|
| 272 |
+
reasons = []
|
| 273 |
+
if len(active) <= 1:
|
| 274 |
+
reasons.append("singleton cluster")
|
| 275 |
+
if purity < 0.75:
|
| 276 |
+
reasons.append("mixed labels")
|
| 277 |
+
if cluster.get("locked"):
|
| 278 |
+
reasons.append("user locked")
|
| 279 |
+
if representative_bonus:
|
| 280 |
+
reasons.append("has representative")
|
| 281 |
+
cluster["confidence"] = round(max(0.0, min(1.0, confidence)), 4)
|
| 282 |
+
cluster["confidence_reasons"] = reasons or ["cohesive cluster"]
|
| 283 |
+
cluster["suppressed_count"] = sum(1 for hit in members if hit.get("suppressed"))
|
| 284 |
+
|
| 285 |
+
for hit in hits.values():
|
| 286 |
+
cluster = clusters.get(hit.get("cluster_id"), {})
|
| 287 |
+
active_count = len(_active_hits(state, cluster)) if cluster else 0
|
| 288 |
+
label_match = _base_label(str(cluster.get("label", ""))) == str(hit.get("label", ""))
|
| 289 |
+
energy = energy_rank(_safe_float(hit.get("rms_energy")))
|
| 290 |
+
duration_ms = _safe_float(hit.get("duration_ms"))
|
| 291 |
+
duration_score = 0.65 if duration_ms <= 0 else max(0.0, min(1.0, 1.0 - abs(duration_ms - 180.0) / 700.0))
|
| 292 |
+
cluster_conf = _safe_float(cluster.get("confidence"), 0.2)
|
| 293 |
+
confidence = (0.42 * cluster_conf) + (0.18 * min(1.0, active_count / 4.0)) + (0.18 if label_match else 0.0) + (0.12 * energy) + (0.10 * duration_score)
|
| 294 |
+
reasons = []
|
| 295 |
+
if active_count <= 1:
|
| 296 |
+
reasons.append("singleton")
|
| 297 |
+
if not label_match:
|
| 298 |
+
reasons.append("label differs from cluster")
|
| 299 |
+
if energy < 0.2:
|
| 300 |
+
reasons.append("low energy")
|
| 301 |
+
if hit.get("is_representative"):
|
| 302 |
+
confidence += 0.08
|
| 303 |
+
reasons.append("representative")
|
| 304 |
+
if hit.get("explicit"):
|
| 305 |
+
confidence += 0.10
|
| 306 |
+
reasons.append("explicit user assignment")
|
| 307 |
+
if hit.get("suppressed"):
|
| 308 |
+
confidence = min(confidence, 0.25)
|
| 309 |
+
reasons.append("suppressed")
|
| 310 |
+
hit["confidence"] = round(max(0.0, min(1.0, confidence)), 4)
|
| 311 |
+
hit["confidence_reasons"] = reasons or ["consistent assignment"]
|
| 312 |
+
hit["cluster_label"] = cluster.get("label", hit.get("cluster_label", "unclustered"))
|
| 313 |
+
|
| 314 |
+
|
| 315 |
+
def review_queue(state: dict[str, Any], limit: int = 30) -> list[dict[str, Any]]:
|
| 316 |
+
rows = []
|
| 317 |
+
clusters = state.get("clusters", {})
|
| 318 |
+
for hit in state.get("hits", {}).values():
|
| 319 |
+
cluster = clusters.get(hit.get("cluster_id"), {})
|
| 320 |
+
score = 1.0 - _safe_float(hit.get("confidence"), 0.0)
|
| 321 |
+
if len(cluster.get("hit_ids", [])) <= 1:
|
| 322 |
+
score += 0.15
|
| 323 |
+
if hit.get("suppressed"):
|
| 324 |
+
score -= 0.35
|
| 325 |
+
if hit.get("review_status") == "accepted":
|
| 326 |
+
score -= 0.25
|
| 327 |
+
rows.append(
|
| 328 |
+
{
|
| 329 |
+
"hit_id": hit["id"],
|
| 330 |
+
"hit_index": hit.get("index"),
|
| 331 |
+
"label": hit.get("label"),
|
| 332 |
+
"cluster_id": hit.get("cluster_id"),
|
| 333 |
+
"cluster_label": cluster.get("label"),
|
| 334 |
+
"confidence": hit.get("confidence", 0.0),
|
| 335 |
+
"priority": round(max(0.0, score), 4),
|
| 336 |
+
"reasons": hit.get("confidence_reasons", []),
|
| 337 |
+
"suppressed": bool(hit.get("suppressed")),
|
| 338 |
+
"file": hit.get("file"),
|
| 339 |
+
}
|
| 340 |
+
)
|
| 341 |
+
rows.sort(key=lambda item: (-item["priority"], item["hit_index"] or 0))
|
| 342 |
+
return rows[: max(1, min(int(limit), 200))]
|
| 343 |
+
|
| 344 |
+
|
| 345 |
+
def _find_similar_hits(state: dict[str, Any], hit_id: str, *, exclude_cluster: str | None = None, include_suppressed: bool = False, limit: int = 12) -> list[tuple[dict[str, Any], float]]:
|
| 346 |
+
hits = state.get("hits", {})
|
| 347 |
+
src = hits[hit_id]
|
| 348 |
+
src_centroid = _safe_float(src.get("spectral_centroid_hz"))
|
| 349 |
+
src_energy = _safe_float(src.get("rms_energy"))
|
| 350 |
+
scored: list[tuple[dict[str, Any], float]] = []
|
| 351 |
+
for candidate in hits.values():
|
| 352 |
+
if candidate["id"] == hit_id:
|
| 353 |
+
continue
|
| 354 |
+
if exclude_cluster and candidate.get("cluster_id") == exclude_cluster:
|
| 355 |
+
continue
|
| 356 |
+
if candidate.get("suppressed") and not include_suppressed:
|
| 357 |
+
continue
|
| 358 |
+
label_score = 1.0 if candidate.get("label") == src.get("label") else 0.35
|
| 359 |
+
centroid_delta = abs(_safe_float(candidate.get("spectral_centroid_hz")) - src_centroid)
|
| 360 |
+
centroid_score = max(0.0, 1.0 - centroid_delta / 6000.0)
|
| 361 |
+
energy_delta = abs(_safe_float(candidate.get("rms_energy")) - src_energy)
|
| 362 |
+
energy_score = max(0.0, 1.0 - energy_delta / max(src_energy, 1e-4, _safe_float(candidate.get("rms_energy"))))
|
| 363 |
+
score = (0.48 * label_score) + (0.34 * centroid_score) + (0.18 * energy_score)
|
| 364 |
+
if score >= 0.62:
|
| 365 |
+
scored.append((candidate, round(score, 4)))
|
| 366 |
+
scored.sort(key=lambda item: (-item[1], item[0].get("index", 0)))
|
| 367 |
+
return scored[:limit]
|
| 368 |
+
|
| 369 |
+
|
| 370 |
+
def _add_suggestion(state: dict[str, Any], suggestion_type: str, payload: dict[str, Any], confidence: float, reason: str) -> dict[str, Any]:
|
| 371 |
+
suggestion = {
|
| 372 |
+
"id": _new_id("suggestion"),
|
| 373 |
+
"type": suggestion_type,
|
| 374 |
+
"status": "open",
|
| 375 |
+
"created_at": now(),
|
| 376 |
+
"confidence": round(max(0.0, min(1.0, confidence)), 4),
|
| 377 |
+
"reason": reason,
|
| 378 |
+
**payload,
|
| 379 |
+
}
|
| 380 |
+
state.setdefault("suggestions", []).append(suggestion)
|
| 381 |
+
_event(state, "suggestion.created", {"suggestion_id": suggestion["id"], "type": suggestion_type, "reason": reason})
|
| 382 |
+
return suggestion
|
| 383 |
+
|
| 384 |
+
|
| 385 |
+
def _rebuild_cluster_labels(state: dict[str, Any]) -> None:
|
| 386 |
+
hits = state.get("hits", {})
|
| 387 |
+
for cluster in state.get("clusters", {}).values():
|
| 388 |
+
for hid in cluster.get("hit_ids", []):
|
| 389 |
+
if hid in hits:
|
| 390 |
+
hits[hid]["cluster_label"] = cluster.get("label", "unclustered")
|
| 391 |
+
|
| 392 |
+
|
| 393 |
+
def move_hit(output_dir: str | Path, job_id: str, hit_id: str, target_cluster_id: str, source: str = "user") -> dict[str, Any]:
|
| 394 |
+
state = load_or_create_state(job_id, output_dir)
|
| 395 |
+
hits = state.get("hits", {})
|
| 396 |
+
clusters = state.get("clusters", {})
|
| 397 |
+
if hit_id not in hits:
|
| 398 |
+
raise KeyError(f"Unknown hit: {hit_id}")
|
| 399 |
+
if target_cluster_id not in clusters:
|
| 400 |
+
raise KeyError(f"Unknown cluster: {target_cluster_id}")
|
| 401 |
+
hit = hits[hit_id]
|
| 402 |
+
source_cluster_id = hit.get("cluster_id")
|
| 403 |
+
if source_cluster_id == target_cluster_id:
|
| 404 |
+
hit["review_status"] = "accepted"
|
| 405 |
+
recompute_scores(state)
|
| 406 |
+
return _write_state(output_dir, state)
|
| 407 |
+
|
| 408 |
+
_push_undo(state)
|
| 409 |
+
if source_cluster_id in clusters:
|
| 410 |
+
clusters[source_cluster_id]["hit_ids"] = [hid for hid in clusters[source_cluster_id].get("hit_ids", []) if hid != hit_id]
|
| 411 |
+
clusters[target_cluster_id].setdefault("hit_ids", [])
|
| 412 |
+
if hit_id not in clusters[target_cluster_id]["hit_ids"]:
|
| 413 |
+
clusters[target_cluster_id]["hit_ids"].append(hit_id)
|
| 414 |
+
hit["cluster_id"] = target_cluster_id
|
| 415 |
+
hit["cluster_label"] = clusters[target_cluster_id].get("label", target_cluster_id)
|
| 416 |
+
hit["explicit"] = True
|
| 417 |
+
hit["review_status"] = "accepted"
|
| 418 |
+
target_rep = clusters[target_cluster_id].get("representative_hit_id")
|
| 419 |
+
_constraint(state, "force-cluster", {"hit_id": hit_id, "cluster_id": target_cluster_id}, source=source)
|
| 420 |
+
if target_rep and target_rep != hit_id:
|
| 421 |
+
_constraint(state, "must-link", {"a": hit_id, "b": target_rep}, source=source)
|
| 422 |
+
_event(state, "hit.moved", {"hit_id": hit_id, "from_cluster_id": source_cluster_id, "to_cluster_id": target_cluster_id}, source=source)
|
| 423 |
+
|
| 424 |
+
similar = _find_similar_hits(state, hit_id, exclude_cluster=target_cluster_id, limit=10)
|
| 425 |
+
suggested_ids = [item[0]["id"] for item in similar if item[1] >= 0.72]
|
| 426 |
+
if suggested_ids:
|
| 427 |
+
avg = sum(score for _, score in similar if _["id"] in suggested_ids) / len(suggested_ids)
|
| 428 |
+
_add_suggestion(
|
| 429 |
+
state,
|
| 430 |
+
"move-hits",
|
| 431 |
+
{"hit_ids": suggested_ids, "target_cluster_id": target_cluster_id, "preview_count": len(suggested_ids)},
|
| 432 |
+
avg,
|
| 433 |
+
f"Similar label/spectral/energy profile to {hit_id}",
|
| 434 |
+
)
|
| 435 |
+
_rebuild_cluster_labels(state)
|
| 436 |
+
recompute_scores(state)
|
| 437 |
+
return _write_state(output_dir, state)
|
| 438 |
+
|
| 439 |
+
|
| 440 |
+
def pull_hit_to_new_cluster(output_dir: str | Path, job_id: str, hit_id: str, label: str | None = None, source: str = "user") -> dict[str, Any]:
|
| 441 |
+
state = load_or_create_state(job_id, output_dir)
|
| 442 |
+
hits = state.get("hits", {})
|
| 443 |
+
clusters = state.get("clusters", {})
|
| 444 |
+
if hit_id not in hits:
|
| 445 |
+
raise KeyError(f"Unknown hit: {hit_id}")
|
| 446 |
+
hit = hits[hit_id]
|
| 447 |
+
source_cluster_id = hit.get("cluster_id")
|
| 448 |
+
source_rep = clusters.get(source_cluster_id, {}).get("representative_hit_id")
|
| 449 |
+
_push_undo(state)
|
| 450 |
+
state.setdefault("counters", {})["user_clusters"] = int(state.get("counters", {}).get("user_clusters", 0)) + 1
|
| 451 |
+
base = label or f"{hit.get('label', 'hit')}_user_{state['counters']['user_clusters']}"
|
| 452 |
+
new_cluster_id = _new_id("cluster:user")
|
| 453 |
+
if source_cluster_id in clusters:
|
| 454 |
+
clusters[source_cluster_id]["hit_ids"] = [hid for hid in clusters[source_cluster_id].get("hit_ids", []) if hid != hit_id]
|
| 455 |
+
clusters[new_cluster_id] = {
|
| 456 |
+
"id": new_cluster_id,
|
| 457 |
+
"label": base,
|
| 458 |
+
"classification": _base_label(base),
|
| 459 |
+
"hit_ids": [hit_id],
|
| 460 |
+
"representative_hit_id": hit_id,
|
| 461 |
+
"locked": False,
|
| 462 |
+
"user_named": bool(label),
|
| 463 |
+
"confidence": 0.0,
|
| 464 |
+
"confidence_reasons": [],
|
| 465 |
+
"suppressed_count": 0,
|
| 466 |
+
"original_id": None,
|
| 467 |
+
}
|
| 468 |
+
hit["cluster_id"] = new_cluster_id
|
| 469 |
+
hit["cluster_label"] = base
|
| 470 |
+
hit["explicit"] = True
|
| 471 |
+
hit["review_status"] = "accepted"
|
| 472 |
+
if source_rep and source_rep != hit_id:
|
| 473 |
+
_constraint(state, "cannot-link", {"a": hit_id, "b": source_rep}, source=source)
|
| 474 |
+
_constraint(state, "force-cluster", {"hit_id": hit_id, "cluster_id": new_cluster_id}, source=source)
|
| 475 |
+
_event(state, "hit.pulled_out", {"hit_id": hit_id, "from_cluster_id": source_cluster_id, "to_cluster_id": new_cluster_id}, source=source)
|
| 476 |
+
|
| 477 |
+
similar = _find_similar_hits(state, hit_id, exclude_cluster=new_cluster_id, limit=8)
|
| 478 |
+
split_ids = [item[0]["id"] for item in similar if item[0].get("cluster_id") == source_cluster_id and item[1] >= 0.70]
|
| 479 |
+
if split_ids:
|
| 480 |
+
_add_suggestion(
|
| 481 |
+
state,
|
| 482 |
+
"split-hits",
|
| 483 |
+
{"hit_ids": split_ids, "source_cluster_id": source_cluster_id, "target_cluster_id": new_cluster_id, "preview_count": len(split_ids)},
|
| 484 |
+
0.76,
|
| 485 |
+
f"Similar to pulled-out hit {hit_id}; preview split from original cluster",
|
| 486 |
+
)
|
| 487 |
+
_rebuild_cluster_labels(state)
|
| 488 |
+
recompute_scores(state)
|
| 489 |
+
return _write_state(output_dir, state)
|
| 490 |
+
|
| 491 |
+
|
| 492 |
+
def lock_cluster(output_dir: str | Path, job_id: str, cluster_id: str, locked: bool = True, source: str = "user") -> dict[str, Any]:
|
| 493 |
+
state = load_or_create_state(job_id, output_dir)
|
| 494 |
+
clusters = state.get("clusters", {})
|
| 495 |
+
if cluster_id not in clusters:
|
| 496 |
+
raise KeyError(f"Unknown cluster: {cluster_id}")
|
| 497 |
+
_push_undo(state)
|
| 498 |
+
clusters[cluster_id]["locked"] = bool(locked)
|
| 499 |
+
_constraint(state, "lock-cluster", {"cluster_id": cluster_id, "locked": bool(locked)}, source=source)
|
| 500 |
+
_event(state, "cluster.locked" if locked else "cluster.unlocked", {"cluster_id": cluster_id}, source=source)
|
| 501 |
+
recompute_scores(state)
|
| 502 |
+
return _write_state(output_dir, state)
|
| 503 |
+
|
| 504 |
+
|
| 505 |
+
def suppress_hit(output_dir: str | Path, job_id: str, hit_id: str, reason: str = "bleed", source: str = "user") -> dict[str, Any]:
|
| 506 |
+
state = load_or_create_state(job_id, output_dir)
|
| 507 |
+
hits = state.get("hits", {})
|
| 508 |
+
if hit_id not in hits:
|
| 509 |
+
raise KeyError(f"Unknown hit: {hit_id}")
|
| 510 |
+
_push_undo(state)
|
| 511 |
+
hit = hits[hit_id]
|
| 512 |
+
hit["suppressed"] = True
|
| 513 |
+
hit["review_status"] = "suppressed"
|
| 514 |
+
hit["explicit"] = True
|
| 515 |
+
_constraint(state, "suppress-pattern", {"example_hit_id": hit_id, "reason": reason}, source=source)
|
| 516 |
+
_event(state, "hit.suppressed", {"hit_id": hit_id, "reason": reason}, source=source)
|
| 517 |
+
similar = _find_similar_hits(state, hit_id, include_suppressed=False, limit=16)
|
| 518 |
+
suggested_ids = [item[0]["id"] for item in similar if item[1] >= 0.72 and _safe_float(item[0].get("rms_energy")) <= _safe_float(hit.get("rms_energy")) * 1.35]
|
| 519 |
+
if suggested_ids:
|
| 520 |
+
_add_suggestion(
|
| 521 |
+
state,
|
| 522 |
+
"suppress-hits",
|
| 523 |
+
{"hit_ids": suggested_ids, "reason_code": reason, "preview_count": len(suggested_ids)},
|
| 524 |
+
0.74,
|
| 525 |
+
f"Similar low-energy profile to suppressed {reason} example {hit_id}",
|
| 526 |
+
)
|
| 527 |
+
recompute_scores(state)
|
| 528 |
+
return _write_state(output_dir, state)
|
| 529 |
+
|
| 530 |
+
|
| 531 |
+
def set_hit_review_status(output_dir: str | Path, job_id: str, hit_id: str, status: str = "accepted", source: str = "user") -> dict[str, Any]:
|
| 532 |
+
if status not in {"unreviewed", "accepted", "favorite"}:
|
| 533 |
+
raise ValueError("status must be unreviewed, accepted, or favorite")
|
| 534 |
+
state = load_or_create_state(job_id, output_dir)
|
| 535 |
+
if hit_id not in state.get("hits", {}):
|
| 536 |
+
raise KeyError(f"Unknown hit: {hit_id}")
|
| 537 |
+
_push_undo(state)
|
| 538 |
+
hit = state["hits"][hit_id]
|
| 539 |
+
hit["review_status"] = status
|
| 540 |
+
if status == "favorite":
|
| 541 |
+
hit["favorite"] = True
|
| 542 |
+
cid = hit.get("cluster_id")
|
| 543 |
+
if cid in state.get("clusters", {}):
|
| 544 |
+
state["clusters"][cid]["representative_hit_id"] = hit_id
|
| 545 |
+
_constraint(state, "pin-representative", {"hit_id": hit_id, "cluster_id": cid}, source=source)
|
| 546 |
+
_event(state, "hit.reviewed", {"hit_id": hit_id, "status": status}, source=source)
|
| 547 |
+
recompute_scores(state)
|
| 548 |
+
return _write_state(output_dir, state)
|
| 549 |
+
|
| 550 |
+
|
| 551 |
+
def accept_suggestion(output_dir: str | Path, job_id: str, suggestion_id: str) -> dict[str, Any]:
|
| 552 |
+
state = load_or_create_state(job_id, output_dir)
|
| 553 |
+
suggestion = next((s for s in state.get("suggestions", []) if s.get("id") == suggestion_id), None)
|
| 554 |
+
if not suggestion:
|
| 555 |
+
raise KeyError(f"Unknown suggestion: {suggestion_id}")
|
| 556 |
+
if suggestion.get("status") != "open":
|
| 557 |
+
return state
|
| 558 |
+
_push_undo(state)
|
| 559 |
+
stype = suggestion.get("type")
|
| 560 |
+
if stype in {"move-hits", "split-hits"}:
|
| 561 |
+
target = suggestion.get("target_cluster_id")
|
| 562 |
+
for hid in suggestion.get("hit_ids", []):
|
| 563 |
+
if hid in state.get("hits", {}) and target in state.get("clusters", {}):
|
| 564 |
+
current = state["hits"][hid].get("cluster_id")
|
| 565 |
+
if current in state["clusters"]:
|
| 566 |
+
state["clusters"][current]["hit_ids"] = [x for x in state["clusters"][current].get("hit_ids", []) if x != hid]
|
| 567 |
+
state["clusters"][target].setdefault("hit_ids", [])
|
| 568 |
+
if hid not in state["clusters"][target]["hit_ids"]:
|
| 569 |
+
state["clusters"][target]["hit_ids"].append(hid)
|
| 570 |
+
state["hits"][hid]["cluster_id"] = target
|
| 571 |
+
state["hits"][hid]["explicit"] = True
|
| 572 |
+
_constraint(state, "force-cluster", {"hit_id": hid, "cluster_id": target}, source="accepted-suggestion")
|
| 573 |
+
elif stype == "suppress-hits":
|
| 574 |
+
for hid in suggestion.get("hit_ids", []):
|
| 575 |
+
if hid in state.get("hits", {}):
|
| 576 |
+
state["hits"][hid]["suppressed"] = True
|
| 577 |
+
state["hits"][hid]["review_status"] = "suppressed"
|
| 578 |
+
_constraint(state, "suppress-pattern", {"example_hit_id": hid, "reason": suggestion.get("reason_code", "bleed")}, source="accepted-suggestion")
|
| 579 |
+
else:
|
| 580 |
+
raise ValueError(f"Unsupported suggestion type: {stype}")
|
| 581 |
+
suggestion["status"] = "accepted"
|
| 582 |
+
suggestion["resolved_at"] = now()
|
| 583 |
+
_event(state, "suggestion.accepted", {"suggestion_id": suggestion_id, "type": stype}, source="user")
|
| 584 |
+
_rebuild_cluster_labels(state)
|
| 585 |
+
recompute_scores(state)
|
| 586 |
+
return _write_state(output_dir, state)
|
| 587 |
+
|
| 588 |
+
|
| 589 |
+
def reject_suggestion(output_dir: str | Path, job_id: str, suggestion_id: str) -> dict[str, Any]:
|
| 590 |
+
state = load_or_create_state(job_id, output_dir)
|
| 591 |
+
suggestion = next((s for s in state.get("suggestions", []) if s.get("id") == suggestion_id), None)
|
| 592 |
+
if not suggestion:
|
| 593 |
+
raise KeyError(f"Unknown suggestion: {suggestion_id}")
|
| 594 |
+
_push_undo(state)
|
| 595 |
+
suggestion["status"] = "rejected"
|
| 596 |
+
suggestion["resolved_at"] = now()
|
| 597 |
+
_event(state, "suggestion.rejected", {"suggestion_id": suggestion_id, "type": suggestion.get("type")}, source="user")
|
| 598 |
+
return _write_state(output_dir, state)
|
| 599 |
+
|
| 600 |
+
|
| 601 |
+
def undo_last(output_dir: str | Path, job_id: str) -> dict[str, Any]:
|
| 602 |
+
state = load_or_create_state(job_id, output_dir)
|
| 603 |
+
stack = list(state.get("undo_stack") or [])
|
| 604 |
+
if not stack:
|
| 605 |
+
return state
|
| 606 |
+
restored = stack.pop()
|
| 607 |
+
restored["undo_stack"] = stack
|
| 608 |
+
_event(restored, "state.undo", {"restored_for_job_id": job_id}, source="user")
|
| 609 |
+
recompute_scores(restored)
|
| 610 |
+
return _write_state(output_dir, restored)
|
| 611 |
+
|
| 612 |
+
|
| 613 |
+
def explain_cluster(state: dict[str, Any], cluster_id: str) -> dict[str, Any]:
|
| 614 |
+
clusters = state.get("clusters", {})
|
| 615 |
+
hits = state.get("hits", {})
|
| 616 |
+
if cluster_id not in clusters:
|
| 617 |
+
raise KeyError(f"Unknown cluster: {cluster_id}")
|
| 618 |
+
cluster = clusters[cluster_id]
|
| 619 |
+
members = [hits[hid] for hid in cluster.get("hit_ids", []) if hid in hits]
|
| 620 |
+
active = [h for h in members if not h.get("suppressed")]
|
| 621 |
+
constraints = [c for c in state.get("constraints", []) if c.get("cluster_id") == cluster_id or c.get("hit_id") in cluster.get("hit_ids", []) or c.get("a") in cluster.get("hit_ids", []) or c.get("b") in cluster.get("hit_ids", [])]
|
| 622 |
+
outliers = sorted(active, key=lambda h: h.get("confidence", 0.0))[:8]
|
| 623 |
+
labels: dict[str, int] = {}
|
| 624 |
+
for hit in active:
|
| 625 |
+
labels[hit.get("label", "other")] = labels.get(hit.get("label", "other"), 0) + 1
|
| 626 |
+
return {
|
| 627 |
+
"cluster_id": cluster_id,
|
| 628 |
+
"label": cluster.get("label"),
|
| 629 |
+
"locked": bool(cluster.get("locked")),
|
| 630 |
+
"confidence": cluster.get("confidence"),
|
| 631 |
+
"confidence_reasons": cluster.get("confidence_reasons", []),
|
| 632 |
+
"representative_hit_id": cluster.get("representative_hit_id"),
|
| 633 |
+
"hit_count": len(members),
|
| 634 |
+
"active_hit_count": len(active),
|
| 635 |
+
"suppressed_count": sum(1 for hit in members if hit.get("suppressed")),
|
| 636 |
+
"label_distribution": labels,
|
| 637 |
+
"outliers": [{"hit_id": h["id"], "hit_index": h.get("index"), "confidence": h.get("confidence"), "reasons": h.get("confidence_reasons", [])} for h in outliers],
|
| 638 |
+
"constraints": constraints[-20:],
|
| 639 |
+
"summary": f"{cluster.get('label')} has {len(active)} active hits, confidence {cluster.get('confidence')}, and {len(constraints)} relevant constraints.",
|
| 640 |
+
}
|
| 641 |
+
|
| 642 |
+
|
| 643 |
+
def public_state(state: dict[str, Any], url_for: Callable[[str], str] | None = None, review_limit: int = 30) -> dict[str, Any]:
|
| 644 |
+
recompute_scores(state)
|
| 645 |
+
hits = copy.deepcopy(list(state.get("hits", {}).values()))
|
| 646 |
+
clusters = copy.deepcopy(list(state.get("clusters", {}).values()))
|
| 647 |
+
for hit in hits:
|
| 648 |
+
if url_for and hit.get("file"):
|
| 649 |
+
hit["url"] = url_for(hit["file"])
|
| 650 |
+
clusters.sort(key=lambda c: (-len(c.get("hit_ids", [])), c.get("label", "")))
|
| 651 |
+
hits.sort(key=lambda h: h.get("index", 0))
|
| 652 |
+
open_suggestions = [s for s in state.get("suggestions", []) if s.get("status") == "open"]
|
| 653 |
+
open_suggestions.sort(key=lambda s: (-_safe_float(s.get("confidence")), s.get("created_at", 0)))
|
| 654 |
+
return {
|
| 655 |
+
"version": state.get("version"),
|
| 656 |
+
"job_id": state.get("job_id"),
|
| 657 |
+
"created_at": state.get("created_at"),
|
| 658 |
+
"updated_at": state.get("updated_at"),
|
| 659 |
+
"summary": {
|
| 660 |
+
"hit_count": len(hits),
|
| 661 |
+
"cluster_count": len(clusters),
|
| 662 |
+
"constraint_count": len(state.get("constraints", [])),
|
| 663 |
+
"event_count": len(state.get("events", [])),
|
| 664 |
+
"open_suggestion_count": len(open_suggestions),
|
| 665 |
+
"suppressed_hit_count": sum(1 for h in hits if h.get("suppressed")),
|
| 666 |
+
"locked_cluster_count": sum(1 for c in clusters if c.get("locked")),
|
| 667 |
+
"undo_available": bool(state.get("undo_stack")),
|
| 668 |
+
},
|
| 669 |
+
"hits": hits,
|
| 670 |
+
"clusters": clusters,
|
| 671 |
+
"constraints": state.get("constraints", [])[-100:],
|
| 672 |
+
"events": state.get("events", [])[-120:],
|
| 673 |
+
"suggestions": open_suggestions[:50],
|
| 674 |
+
"review_queue": review_queue(state, review_limit),
|
| 675 |
+
}
|
web/app.js
CHANGED
|
@@ -12,6 +12,8 @@ let selectedFile = null;
|
|
| 12 |
let activePoll = null;
|
| 13 |
let activeEvents = null;
|
| 14 |
let lastResult = null;
|
|
|
|
|
|
|
| 15 |
let selectedHitIndex = null;
|
| 16 |
|
| 17 |
function esc(value) {
|
|
@@ -47,6 +49,60 @@ async function api(path, options = {}) {
|
|
| 47 |
return response.json();
|
| 48 |
}
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
function setSelectOptions(select, values, labels = null) {
|
| 51 |
select.innerHTML = "";
|
| 52 |
for (const value of values) {
|
|
@@ -159,15 +215,19 @@ function playAudio(el, url) {
|
|
| 159 |
|
| 160 |
function selectHit(index, shouldPlay = true) {
|
| 161 |
if (!lastResult) return;
|
| 162 |
-
const
|
| 163 |
-
if (!
|
|
|
|
| 164 |
selectedHitIndex = hit.index;
|
| 165 |
-
|
|
|
|
|
|
|
| 166 |
if (shouldPlay) playAudio($("hitAudio"), hit.url);
|
| 167 |
for (const row of document.querySelectorAll("[data-hit-index]")) {
|
| 168 |
row.classList.toggle("selected", Number(row.dataset.hitIndex) === Number(hit.index));
|
| 169 |
}
|
| 170 |
drawWaveform(lastResult.overview);
|
|
|
|
| 171 |
}
|
| 172 |
|
| 173 |
function auditionSample(sample) {
|
|
@@ -200,19 +260,26 @@ function renderSamples(result) {
|
|
| 200 |
|
| 201 |
function renderHits(result) {
|
| 202 |
const tbody = $("hitsTable").querySelector("tbody");
|
| 203 |
-
const hits = result.hits ?? [];
|
| 204 |
-
tbody.innerHTML = hits.map((hit) =>
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
<
|
| 210 |
-
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 216 |
for (const row of tbody.querySelectorAll("[data-hit-index]")) {
|
| 217 |
row.addEventListener("click", () => selectHit(row.dataset.hitIndex));
|
| 218 |
}
|
|
@@ -225,9 +292,143 @@ function renderHits(result) {
|
|
| 225 |
if (hits.length && selectedHitIndex === null) selectHit(hits[0].index, false);
|
| 226 |
}
|
| 227 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 228 |
function renderResult(job) {
|
| 229 |
const result = job.result;
|
| 230 |
if (!result) return;
|
|
|
|
| 231 |
lastResult = result;
|
| 232 |
if (!(result.hits ?? []).some((hit) => Number(hit.index) === Number(selectedHitIndex))) {
|
| 233 |
selectedHitIndex = (result.hits ?? [])[0]?.index ?? null;
|
|
@@ -245,6 +446,9 @@ function renderResult(job) {
|
|
| 245 |
renderSamples(result);
|
| 246 |
renderHits(result);
|
| 247 |
drawWaveform(result.overview);
|
|
|
|
|
|
|
|
|
|
| 248 |
}
|
| 249 |
|
| 250 |
function renderJob(job) {
|
|
@@ -345,6 +549,8 @@ async function runExtraction() {
|
|
| 345 |
if (!selectedFile) return;
|
| 346 |
selectedHitIndex = null;
|
| 347 |
lastResult = null;
|
|
|
|
|
|
|
| 348 |
$("runButton").disabled = true;
|
| 349 |
$("jobPill").textContent = "uploading";
|
| 350 |
$("logs").textContent = "Uploading source and starting extraction…";
|
|
@@ -426,6 +632,16 @@ $("clearCacheButton").addEventListener("click", async () => {
|
|
| 426 |
$("logs").textContent = error.message;
|
| 427 |
}
|
| 428 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 429 |
$("waveform").addEventListener("click", selectNearestWaveformHit);
|
| 430 |
|
| 431 |
const dropzone = $("dropzone");
|
|
|
|
| 12 |
let activePoll = null;
|
| 13 |
let activeEvents = null;
|
| 14 |
let lastResult = null;
|
| 15 |
+
let lastSupervisionState = null;
|
| 16 |
+
let activeJobId = null;
|
| 17 |
let selectedHitIndex = null;
|
| 18 |
|
| 19 |
function esc(value) {
|
|
|
|
| 49 |
return response.json();
|
| 50 |
}
|
| 51 |
|
| 52 |
+
async function jsonApi(path, body = {}, method = "POST") {
|
| 53 |
+
return api(path, {
|
| 54 |
+
method,
|
| 55 |
+
headers: { "Content-Type": "application/json" },
|
| 56 |
+
body: JSON.stringify(body),
|
| 57 |
+
});
|
| 58 |
+
}
|
| 59 |
+
|
| 60 |
+
function hitIdFromIndex(index) {
|
| 61 |
+
if (index === null || index === undefined) return null;
|
| 62 |
+
return `hit:${String(Number(index)).padStart(5, "0")}`;
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
function stateHitByIndex(index) {
|
| 66 |
+
const id = hitIdFromIndex(index);
|
| 67 |
+
return (lastSupervisionState?.hits ?? []).find((hit) => hit.id === id) ?? null;
|
| 68 |
+
}
|
| 69 |
+
|
| 70 |
+
function decorateHit(hit) {
|
| 71 |
+
const stateHit = stateHitByIndex(hit.index);
|
| 72 |
+
return {
|
| 73 |
+
...hit,
|
| 74 |
+
state_hit_id: stateHit?.id ?? hitIdFromIndex(hit.index),
|
| 75 |
+
cluster_ref: stateHit?.cluster_id ?? `cluster:${hit.cluster_id}`,
|
| 76 |
+
cluster_label: stateHit?.cluster_label ?? hit.cluster_label,
|
| 77 |
+
confidence: stateHit?.confidence,
|
| 78 |
+
confidence_reasons: stateHit?.confidence_reasons ?? [],
|
| 79 |
+
suppressed: Boolean(stateHit?.suppressed),
|
| 80 |
+
favorite: Boolean(stateHit?.favorite),
|
| 81 |
+
review_status: stateHit?.review_status ?? "unreviewed",
|
| 82 |
+
};
|
| 83 |
+
}
|
| 84 |
+
|
| 85 |
+
function currentTargetCluster() {
|
| 86 |
+
const id = $("targetClusterSelect")?.value;
|
| 87 |
+
return (lastSupervisionState?.clusters ?? []).find((cluster) => cluster.id === id) ?? null;
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
function setActionButtons() {
|
| 91 |
+
const hasState = Boolean(activeJobId && lastSupervisionState);
|
| 92 |
+
const hasHit = hasState && selectedHitIndex !== null;
|
| 93 |
+
for (const id of ["moveHitButton", "pullHitButton", "acceptHitButton", "favoriteHitButton", "suppressHitButton"]) {
|
| 94 |
+
const button = $(id);
|
| 95 |
+
if (button) button.disabled = !hasHit;
|
| 96 |
+
}
|
| 97 |
+
for (const id of ["refreshStateButton", "undoButton", "lockClusterButton", "explainClusterButton"]) {
|
| 98 |
+
const button = $(id);
|
| 99 |
+
if (button) button.disabled = !hasState;
|
| 100 |
+
}
|
| 101 |
+
const target = currentTargetCluster();
|
| 102 |
+
if ($("lockClusterButton")) $("lockClusterButton").textContent = target?.locked ? "Unlock target cluster" : "Lock target cluster";
|
| 103 |
+
if ($("undoButton") && lastSupervisionState) $("undoButton").disabled = !lastSupervisionState.summary?.undo_available;
|
| 104 |
+
}
|
| 105 |
+
|
| 106 |
function setSelectOptions(select, values, labels = null) {
|
| 107 |
select.innerHTML = "";
|
| 108 |
for (const value of values) {
|
|
|
|
| 215 |
|
| 216 |
function selectHit(index, shouldPlay = true) {
|
| 217 |
if (!lastResult) return;
|
| 218 |
+
const rawHit = (lastResult.hits ?? []).find((item) => Number(item.index) === Number(index));
|
| 219 |
+
if (!rawHit) return;
|
| 220 |
+
const hit = decorateHit(rawHit);
|
| 221 |
selectedHitIndex = hit.index;
|
| 222 |
+
const confidence = hit.confidence === undefined ? "—" : `${Math.round(Number(hit.confidence) * 100)}%`;
|
| 223 |
+
const flags = [hit.is_representative ? "representative" : null, hit.favorite ? "favorite" : null, hit.suppressed ? "suppressed" : null, hit.review_status !== "unreviewed" ? hit.review_status : null].filter(Boolean).join(" · ");
|
| 224 |
+
$("selectedHitMeta").textContent = `#${hit.index} · ${hit.label} · ${hit.cluster_label} · ${hit.onset_sec}s · ${hit.duration_ms} ms · confidence ${confidence}${flags ? ` · ${flags}` : ""}`;
|
| 225 |
if (shouldPlay) playAudio($("hitAudio"), hit.url);
|
| 226 |
for (const row of document.querySelectorAll("[data-hit-index]")) {
|
| 227 |
row.classList.toggle("selected", Number(row.dataset.hitIndex) === Number(hit.index));
|
| 228 |
}
|
| 229 |
drawWaveform(lastResult.overview);
|
| 230 |
+
setActionButtons();
|
| 231 |
}
|
| 232 |
|
| 233 |
function auditionSample(sample) {
|
|
|
|
| 260 |
|
| 261 |
function renderHits(result) {
|
| 262 |
const tbody = $("hitsTable").querySelector("tbody");
|
| 263 |
+
const hits = (result.hits ?? []).map(decorateHit);
|
| 264 |
+
tbody.innerHTML = hits.map((hit) => {
|
| 265 |
+
const confidence = hit.confidence === undefined ? "—" : `${Math.round(Number(hit.confidence) * 100)}%`;
|
| 266 |
+
const flags = [hit.is_representative ? "rep" : null, hit.favorite ? "fav" : null, hit.suppressed ? "suppressed" : null, hit.review_status !== "unreviewed" ? hit.review_status : null].filter(Boolean);
|
| 267 |
+
const classes = [Number(hit.index) === Number(selectedHitIndex) ? "selected" : "", hit.suppressed ? "suppressed" : "", Number(hit.confidence ?? 1) < 0.55 ? "low-confidence" : ""].filter(Boolean).join(" ");
|
| 268 |
+
return `
|
| 269 |
+
<tr data-hit-index="${esc(hit.index)}" class="${esc(classes)}">
|
| 270 |
+
<td><button class="mini-button" type="button" data-hit-audition="${esc(hit.index)}">Audition</button></td>
|
| 271 |
+
<td>${esc(hit.index)}</td>
|
| 272 |
+
<td>${esc(hit.label)}${hit.is_representative || hit.favorite ? " ★" : ""}</td>
|
| 273 |
+
<td>${esc(hit.cluster_label)}</td>
|
| 274 |
+
<td>${esc(confidence)}</td>
|
| 275 |
+
<td>${esc(flags.join(", ") || "—")}</td>
|
| 276 |
+
<td>${esc(hit.onset_sec)} s</td>
|
| 277 |
+
<td>${esc(hit.duration_ms)} ms</td>
|
| 278 |
+
<td>${esc(hit.rms_energy)}</td>
|
| 279 |
+
<td><a href="${esc(hit.url)}" download>WAV</a></td>
|
| 280 |
+
</tr>
|
| 281 |
+
`;
|
| 282 |
+
}).join("");
|
| 283 |
for (const row of tbody.querySelectorAll("[data-hit-index]")) {
|
| 284 |
row.addEventListener("click", () => selectHit(row.dataset.hitIndex));
|
| 285 |
}
|
|
|
|
| 292 |
if (hits.length && selectedHitIndex === null) selectHit(hits[0].index, false);
|
| 293 |
}
|
| 294 |
|
| 295 |
+
function renderSupervisionState(state) {
|
| 296 |
+
lastSupervisionState = state;
|
| 297 |
+
const summary = state.summary ?? {};
|
| 298 |
+
$("supervisionSummary").innerHTML = `
|
| 299 |
+
<span>${esc(summary.hit_count ?? 0)} hits</span>
|
| 300 |
+
<span>${esc(summary.cluster_count ?? 0)} clusters</span>
|
| 301 |
+
<span>${esc(summary.constraint_count ?? 0)} constraints</span>
|
| 302 |
+
<span>${esc(summary.open_suggestion_count ?? 0)} suggestions</span>
|
| 303 |
+
<span>${esc(summary.suppressed_hit_count ?? 0)} suppressed</span>
|
| 304 |
+
<span>${esc(summary.locked_cluster_count ?? 0)} locked</span>
|
| 305 |
+
`;
|
| 306 |
+
|
| 307 |
+
const currentTarget = $("targetClusterSelect").value;
|
| 308 |
+
$("targetClusterSelect").innerHTML = (state.clusters ?? []).map((cluster) => `
|
| 309 |
+
<option value="${esc(cluster.id)}">${cluster.locked ? "locked · " : ""}${esc(cluster.label)} · ${esc(cluster.hit_ids?.length ?? 0)} hits · ${Math.round(Number(cluster.confidence ?? 0) * 100)}%</option>
|
| 310 |
+
`).join("");
|
| 311 |
+
if ((state.clusters ?? []).some((cluster) => cluster.id === currentTarget)) $("targetClusterSelect").value = currentTarget;
|
| 312 |
+
|
| 313 |
+
$("reviewQueue").innerHTML = (state.review_queue ?? []).slice(0, 14).map((item) => `
|
| 314 |
+
<button class="compact-row ${item.suppressed ? "suppressed" : ""}" type="button" data-review-hit="${esc(item.hit_index)}">
|
| 315 |
+
<span><strong>#${esc(item.hit_index)} · ${esc(item.label)}</strong><small>${esc(item.cluster_label)} · ${Math.round(Number(item.confidence ?? 0) * 100)}% · ${esc((item.reasons ?? []).join(", "))}</small></span>
|
| 316 |
+
<span>${Math.round(Number(item.priority ?? 0) * 100)}</span>
|
| 317 |
+
</button>
|
| 318 |
+
`).join("") || `<p class="empty">No review items.</p>`;
|
| 319 |
+
for (const button of $("reviewQueue").querySelectorAll("[data-review-hit]")) {
|
| 320 |
+
button.addEventListener("click", () => selectHit(button.dataset.reviewHit));
|
| 321 |
+
}
|
| 322 |
+
|
| 323 |
+
$("clusterBoard").innerHTML = (state.clusters ?? []).map((cluster) => `
|
| 324 |
+
<button class="compact-row ${cluster.locked ? "locked" : ""}" type="button" data-cluster-select="${esc(cluster.id)}">
|
| 325 |
+
<span><strong>${cluster.locked ? "Locked · " : ""}${esc(cluster.label)}</strong><small>${esc(cluster.hit_ids?.length ?? 0)} hits · ${Math.round(Number(cluster.confidence ?? 0) * 100)}% · ${esc((cluster.confidence_reasons ?? []).join(", "))}</small></span>
|
| 326 |
+
<span>${esc(cluster.suppressed_count ?? 0)} suppr.</span>
|
| 327 |
+
</button>
|
| 328 |
+
`).join("") || `<p class="empty">No clusters.</p>`;
|
| 329 |
+
for (const button of $("clusterBoard").querySelectorAll("[data-cluster-select]")) {
|
| 330 |
+
button.addEventListener("click", () => {
|
| 331 |
+
$("targetClusterSelect").value = button.dataset.clusterSelect;
|
| 332 |
+
setActionButtons();
|
| 333 |
+
explainTargetCluster().catch(() => {});
|
| 334 |
+
});
|
| 335 |
+
}
|
| 336 |
+
|
| 337 |
+
$("suggestionInbox").innerHTML = (state.suggestions ?? []).map((suggestion) => `
|
| 338 |
+
<div class="suggestion-row">
|
| 339 |
+
<div><strong>${esc(suggestion.type)}</strong><small>${esc(suggestion.reason)} · ${Math.round(Number(suggestion.confidence ?? 0) * 100)}% · ${esc(suggestion.preview_count ?? suggestion.hit_ids?.length ?? 0)} hits</small></div>
|
| 340 |
+
<div class="row-actions">
|
| 341 |
+
<button class="mini-button" type="button" data-accept-suggestion="${esc(suggestion.id)}">Accept</button>
|
| 342 |
+
<button class="mini-button" type="button" data-reject-suggestion="${esc(suggestion.id)}">Reject</button>
|
| 343 |
+
</div>
|
| 344 |
+
</div>
|
| 345 |
+
`).join("") || `<p class="empty">No open suggestions.</p>`;
|
| 346 |
+
for (const button of $("suggestionInbox").querySelectorAll("[data-accept-suggestion]")) {
|
| 347 |
+
button.addEventListener("click", () => acceptSuggestion(button.dataset.acceptSuggestion));
|
| 348 |
+
}
|
| 349 |
+
for (const button of $("suggestionInbox").querySelectorAll("[data-reject-suggestion]")) {
|
| 350 |
+
button.addEventListener("click", () => rejectSuggestion(button.dataset.rejectSuggestion));
|
| 351 |
+
}
|
| 352 |
+
|
| 353 |
+
const eventRows = (state.events ?? []).slice(-12).reverse().map((event) => `<div class="log-row"><strong>${esc(event.type)}</strong><small>${esc(event.source)} · ${fmtDate(event.created_at)}</small></div>`);
|
| 354 |
+
const constraintRows = (state.constraints ?? []).slice(-8).reverse().map((constraint) => `<div class="log-row constraint"><strong>${esc(constraint.type)}</strong><small>${esc(constraint.source)} · ${fmtDate(constraint.created_at)}</small></div>`);
|
| 355 |
+
$("stateLog").innerHTML = [...eventRows, ...constraintRows].join("") || `<p class="empty">No state events yet.</p>`;
|
| 356 |
+
|
| 357 |
+
setActionButtons();
|
| 358 |
+
if (lastResult) renderHits(lastResult);
|
| 359 |
+
}
|
| 360 |
+
|
| 361 |
+
async function fetchState() {
|
| 362 |
+
if (!activeJobId) return null;
|
| 363 |
+
const state = await api(`/api/jobs/${encodeURIComponent(activeJobId)}/state`);
|
| 364 |
+
renderSupervisionState(state);
|
| 365 |
+
return state;
|
| 366 |
+
}
|
| 367 |
+
|
| 368 |
+
async function applyStateAction(path, body = {}) {
|
| 369 |
+
if (!activeJobId) return;
|
| 370 |
+
const state = await jsonApi(path, body);
|
| 371 |
+
renderSupervisionState(state);
|
| 372 |
+
}
|
| 373 |
+
|
| 374 |
+
async function moveSelectedHit() {
|
| 375 |
+
const hitId = hitIdFromIndex(selectedHitIndex);
|
| 376 |
+
const target = $("targetClusterSelect").value;
|
| 377 |
+
if (!activeJobId || !hitId || !target) return;
|
| 378 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/move`, { target_cluster_id: target });
|
| 379 |
+
}
|
| 380 |
+
|
| 381 |
+
async function pullSelectedHit() {
|
| 382 |
+
const hitId = hitIdFromIndex(selectedHitIndex);
|
| 383 |
+
if (!activeJobId || !hitId) return;
|
| 384 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/pull-out`, {});
|
| 385 |
+
}
|
| 386 |
+
|
| 387 |
+
async function suppressSelectedHit() {
|
| 388 |
+
const hitId = hitIdFromIndex(selectedHitIndex);
|
| 389 |
+
if (!activeJobId || !hitId) return;
|
| 390 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/suppress`, { reason: "bleed" });
|
| 391 |
+
}
|
| 392 |
+
|
| 393 |
+
async function reviewSelectedHit(status) {
|
| 394 |
+
const hitId = hitIdFromIndex(selectedHitIndex);
|
| 395 |
+
if (!activeJobId || !hitId) return;
|
| 396 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/review`, { status });
|
| 397 |
+
}
|
| 398 |
+
|
| 399 |
+
async function toggleTargetClusterLock() {
|
| 400 |
+
const target = currentTargetCluster();
|
| 401 |
+
if (!activeJobId || !target) return;
|
| 402 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/clusters/${encodeURIComponent(target.id)}/lock`, { locked: !target.locked });
|
| 403 |
+
}
|
| 404 |
+
|
| 405 |
+
async function explainTargetCluster() {
|
| 406 |
+
const target = currentTargetCluster();
|
| 407 |
+
if (!activeJobId || !target) return;
|
| 408 |
+
const explanation = await api(`/api/jobs/${encodeURIComponent(activeJobId)}/explain/cluster/${encodeURIComponent(target.id)}`);
|
| 409 |
+
$("clusterExplanation").classList.remove("empty");
|
| 410 |
+
$("clusterExplanation").textContent = JSON.stringify(explanation, null, 2);
|
| 411 |
+
}
|
| 412 |
+
|
| 413 |
+
async function acceptSuggestion(id) {
|
| 414 |
+
if (!activeJobId) return;
|
| 415 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/suggestions/${encodeURIComponent(id)}/accept`, {});
|
| 416 |
+
}
|
| 417 |
+
|
| 418 |
+
async function rejectSuggestion(id) {
|
| 419 |
+
if (!activeJobId) return;
|
| 420 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/suggestions/${encodeURIComponent(id)}/reject`, {});
|
| 421 |
+
}
|
| 422 |
+
|
| 423 |
+
async function undoLastEdit() {
|
| 424 |
+
if (!activeJobId) return;
|
| 425 |
+
await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/undo`, {});
|
| 426 |
+
}
|
| 427 |
+
|
| 428 |
function renderResult(job) {
|
| 429 |
const result = job.result;
|
| 430 |
if (!result) return;
|
| 431 |
+
activeJobId = job.id;
|
| 432 |
lastResult = result;
|
| 433 |
if (!(result.hits ?? []).some((hit) => Number(hit.index) === Number(selectedHitIndex))) {
|
| 434 |
selectedHitIndex = (result.hits ?? [])[0]?.index ?? null;
|
|
|
|
| 446 |
renderSamples(result);
|
| 447 |
renderHits(result);
|
| 448 |
drawWaveform(result.overview);
|
| 449 |
+
if (activeJobId) fetchState().catch((error) => {
|
| 450 |
+
$("supervisionSummary").textContent = error.message;
|
| 451 |
+
});
|
| 452 |
}
|
| 453 |
|
| 454 |
function renderJob(job) {
|
|
|
|
| 549 |
if (!selectedFile) return;
|
| 550 |
selectedHitIndex = null;
|
| 551 |
lastResult = null;
|
| 552 |
+
lastSupervisionState = null;
|
| 553 |
+
activeJobId = null;
|
| 554 |
$("runButton").disabled = true;
|
| 555 |
$("jobPill").textContent = "uploading";
|
| 556 |
$("logs").textContent = "Uploading source and starting extraction…";
|
|
|
|
| 632 |
$("logs").textContent = error.message;
|
| 633 |
}
|
| 634 |
});
|
| 635 |
+
$("refreshStateButton").addEventListener("click", () => fetchState().catch((error) => { $("supervisionSummary").textContent = error.message; }));
|
| 636 |
+
$("undoButton").addEventListener("click", () => undoLastEdit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 637 |
+
$("moveHitButton").addEventListener("click", () => moveSelectedHit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 638 |
+
$("pullHitButton").addEventListener("click", () => pullSelectedHit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 639 |
+
$("acceptHitButton").addEventListener("click", () => reviewSelectedHit("accepted").catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 640 |
+
$("favoriteHitButton").addEventListener("click", () => reviewSelectedHit("favorite").catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 641 |
+
$("suppressHitButton").addEventListener("click", () => suppressSelectedHit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 642 |
+
$("lockClusterButton").addEventListener("click", () => toggleTargetClusterLock().catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 643 |
+
$("explainClusterButton").addEventListener("click", () => explainTargetCluster().catch((error) => { $("clusterExplanation").textContent = error.message; }));
|
| 644 |
+
$("targetClusterSelect").addEventListener("change", setActionButtons);
|
| 645 |
$("waveform").addEventListener("click", selectNearestWaveformHit);
|
| 646 |
|
| 647 |
const dropzone = $("dropzone");
|
web/index.html
CHANGED
|
@@ -187,6 +187,51 @@
|
|
| 187 |
<audio id="sampleAudio" controls></audio>
|
| 188 |
</article>
|
| 189 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 190 |
<div class="result-columns">
|
| 191 |
<section>
|
| 192 |
<h3>Representative samples</h3>
|
|
@@ -208,7 +253,7 @@
|
|
| 208 |
<table id="hitsTable">
|
| 209 |
<thead>
|
| 210 |
<tr>
|
| 211 |
-
<th>Audition</th><th>#</th><th>Label</th><th>Cluster</th><th>Onset</th><th>Duration</th><th>Energy</th><th>File</th>
|
| 212 |
</tr>
|
| 213 |
</thead>
|
| 214 |
<tbody></tbody>
|
|
|
|
| 187 |
<audio id="sampleAudio" controls></audio>
|
| 188 |
</article>
|
| 189 |
</div>
|
| 190 |
+
|
| 191 |
+
<section class="supervision-panel" aria-live="polite">
|
| 192 |
+
<div class="supervision-header">
|
| 193 |
+
<div>
|
| 194 |
+
<h3>Interactive supervision</h3>
|
| 195 |
+
<p class="subtle">Moves, locks, suppressions, favorites, and accepted suggestions are saved as replayable semantic state next to the run manifest.</p>
|
| 196 |
+
</div>
|
| 197 |
+
<div class="supervision-actions">
|
| 198 |
+
<button id="refreshStateButton" class="ghost-button" type="button">Refresh state</button>
|
| 199 |
+
<button id="undoButton" class="ghost-button" type="button" disabled>Undo edit</button>
|
| 200 |
+
</div>
|
| 201 |
+
</div>
|
| 202 |
+
<div id="supervisionSummary" class="state-summary">No interactive state loaded.</div>
|
| 203 |
+
<div class="supervision-tools">
|
| 204 |
+
<label>Target cluster
|
| 205 |
+
<select id="targetClusterSelect"></select>
|
| 206 |
+
</label>
|
| 207 |
+
<button id="moveHitButton" class="secondary-button" type="button" disabled>Move selected hit</button>
|
| 208 |
+
<button id="pullHitButton" class="secondary-button" type="button" disabled>Pull into new cluster</button>
|
| 209 |
+
<button id="acceptHitButton" class="secondary-button" type="button" disabled>Accept hit</button>
|
| 210 |
+
<button id="favoriteHitButton" class="secondary-button" type="button" disabled>Favorite as representative</button>
|
| 211 |
+
<button id="suppressHitButton" class="secondary-button danger-button" type="button" disabled>Suppress as bleed</button>
|
| 212 |
+
<button id="lockClusterButton" class="secondary-button" type="button" disabled>Lock target cluster</button>
|
| 213 |
+
<button id="explainClusterButton" class="secondary-button" type="button" disabled>Explain target cluster</button>
|
| 214 |
+
</div>
|
| 215 |
+
<div class="supervision-grid">
|
| 216 |
+
<article>
|
| 217 |
+
<h4>Outlier-first review queue</h4>
|
| 218 |
+
<div id="reviewQueue" class="compact-list"></div>
|
| 219 |
+
</article>
|
| 220 |
+
<article>
|
| 221 |
+
<h4>Cluster board</h4>
|
| 222 |
+
<div id="clusterBoard" class="compact-list"></div>
|
| 223 |
+
</article>
|
| 224 |
+
<article>
|
| 225 |
+
<h4>Suggestion inbox</h4>
|
| 226 |
+
<div id="suggestionInbox" class="compact-list"></div>
|
| 227 |
+
</article>
|
| 228 |
+
<article>
|
| 229 |
+
<h4>Constraint / event log</h4>
|
| 230 |
+
<div id="stateLog" class="compact-list"></div>
|
| 231 |
+
</article>
|
| 232 |
+
</div>
|
| 233 |
+
<pre id="clusterExplanation" class="explanation empty">Select a cluster and click Explain.</pre>
|
| 234 |
+
</section>
|
| 235 |
<div class="result-columns">
|
| 236 |
<section>
|
| 237 |
<h3>Representative samples</h3>
|
|
|
|
| 253 |
<table id="hitsTable">
|
| 254 |
<thead>
|
| 255 |
<tr>
|
| 256 |
+
<th>Audition</th><th>#</th><th>Label</th><th>Cluster</th><th>Confidence</th><th>Flags</th><th>Onset</th><th>Duration</th><th>Energy</th><th>File</th>
|
| 257 |
</tr>
|
| 258 |
</thead>
|
| 259 |
<tbody></tbody>
|
web/styles.css
CHANGED
|
@@ -99,3 +99,24 @@ tr.selected td { background: rgba(139,211,255,.12); }
|
|
| 99 |
tr[data-hit-index] { cursor: pointer; }
|
| 100 |
tr[data-hit-index]:hover td { background: rgba(255,255,255,.045); }
|
| 101 |
@media (max-width: 760px) { .review-grid { grid-template-columns: 1fr; } }
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
tr[data-hit-index] { cursor: pointer; }
|
| 100 |
tr[data-hit-index]:hover td { background: rgba(255,255,255,.045); }
|
| 101 |
@media (max-width: 760px) { .review-grid { grid-template-columns: 1fr; } }
|
| 102 |
+
.supervision-panel { border: 1px solid var(--line); border-radius: 24px; background: rgba(0,0,0,.14); padding: 16px; margin: 0 0 20px; }
|
| 103 |
+
.supervision-header { display: flex; align-items: flex-start; justify-content: space-between; gap: 16px; margin-bottom: 14px; }
|
| 104 |
+
.supervision-header h3, .supervision-grid h4 { margin: 0; }
|
| 105 |
+
.supervision-actions, .row-actions { display: flex; gap: 8px; flex-wrap: wrap; justify-content: flex-end; }
|
| 106 |
+
.state-summary { display: flex; flex-wrap: wrap; gap: 8px; margin-bottom: 14px; color: #dbe5f7; }
|
| 107 |
+
.state-summary span { border: 1px solid var(--line); border-radius: 999px; background: rgba(255,255,255,.06); padding: 7px 10px; font-size: 12px; font-weight: 800; }
|
| 108 |
+
.supervision-tools { display: grid; grid-template-columns: minmax(220px, 1fr) repeat(7, auto); gap: 10px; align-items: end; margin-bottom: 16px; }
|
| 109 |
+
.danger-button { border-color: rgba(255,109,122,.35); color: #ffd4d8; }
|
| 110 |
+
.supervision-grid { display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 12px; }
|
| 111 |
+
.compact-list { display: grid; gap: 8px; max-height: 300px; overflow: auto; }
|
| 112 |
+
.compact-row, .suggestion-row, .log-row { width: 100%; display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 10px; align-items: center; border: 1px solid var(--line); border-radius: 14px; padding: 10px; background: rgba(0,0,0,.14); color: var(--text); text-align: left; }
|
| 113 |
+
.compact-row strong, .suggestion-row strong, .log-row strong { display: block; font-size: 13px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
| 114 |
+
.compact-row small, .suggestion-row small, .log-row small { display: block; color: var(--muted); font-size: 11px; margin-top: 3px; line-height: 1.35; }
|
| 115 |
+
.compact-row.locked { border-color: rgba(85,230,165,.45); background: rgba(85,230,165,.08); }
|
| 116 |
+
.compact-row.suppressed, tr.suppressed td { opacity: .62; text-decoration: line-through; }
|
| 117 |
+
.log-row.constraint { border-color: rgba(200,165,255,.26); }
|
| 118 |
+
.explanation { min-height: 120px; max-height: 320px; overflow: auto; border: 1px solid var(--line); border-radius: 16px; background: #05070b; color: #b9d7e9; padding: 12px; margin: 14px 0 0; font-size: 12px; line-height: 1.45; }
|
| 119 |
+
tr.low-confidence td { background: rgba(255,202,107,.06); }
|
| 120 |
+
tr.low-confidence.selected td { background: rgba(139,211,255,.15); }
|
| 121 |
+
@media (max-width: 1320px) { .supervision-tools { grid-template-columns: repeat(3, minmax(0, 1fr)); } .supervision-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } }
|
| 122 |
+
@media (max-width: 760px) { .supervision-header { display: block; } .supervision-actions { justify-content: flex-start; margin-top: 10px; } .supervision-tools, .supervision-grid { grid-template-columns: 1fr; } }
|