ChatGPT commited on
Commit
03d531b
·
1 Parent(s): 3703c4e

feat: add supervised interactive editing state

Browse files
README.md CHANGED
@@ -10,18 +10,18 @@ pinned: false
10
 
11
  # Drum Sample Extractor
12
 
13
- A custom FastAPI + browser workstation for extracting reusable drum samples from an audio file.
14
 
15
- The pipeline can isolate a stem with Demucs, detect onsets, classify hits, cluster similar transients, choose representative samples, optionally synthesize alternate samples, and export WAVs, MIDI, reconstruction audio, manifests, and a complete ZIP sample pack.
16
 
17
  ## Current status
18
 
19
  The project is usable as a local/Hugging Face Space application. Gradio is no longer the active UI; the active app is a custom FastAPI backend plus a no-build browser frontend.
20
 
21
- Implemented in the current development pass:
22
 
23
  - Custom web frontend in `web/`, served by `app.py`.
24
- - FastAPI job API with upload, polling, safe artifact downloads, config, health, cache clearing, and run-history listing.
25
  - Timed pipeline runner in `pipeline_runner.py`.
26
  - Per-stage timing in every `manifest.json`.
27
  - Two clustering modes:
@@ -31,15 +31,34 @@ Implemented in the current development pass:
31
  - Run history panel indexing `.runs/*/output/manifest.json`.
32
  - Individual review WAVs for every detected hit under `review/hits/`.
33
  - Click-to-audition workflow for waveform onsets, detected hit rows, and representative sample rows.
34
- - Server-sent-events progress endpoint with frontend `EventSource` support and polling fallback.
35
- - Documentation for features, progress, tasks, API, timing, hit review, realtime suitability, UI, and remaining work.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  - Legacy Gradio apps preserved in `legacy/` for reference only.
37
 
38
  Not fully complete yet:
39
 
40
- - No interactive waveform editing of onsets/clusters.
41
- - No interactive onset/cluster editing yet.
42
- - No frontend TypeScript build/test harness.
 
 
43
  - Demucs remains offline/batch by design.
44
 
45
  See:
@@ -47,7 +66,8 @@ See:
47
  - `docs/FEATURES.md`
48
  - `docs/TASKS.md`
49
  - `docs/PROGRESS.md`
50
- - `docs/HIT_REVIEW_AND_STREAMING.md`
 
51
  - `docs/REMAINING_WORK.md`
52
 
53
  ## Run locally
@@ -68,11 +88,19 @@ For fast iteration, set:
68
 
69
  That bypasses Demucs and uses the near-realtime clustering path.
70
 
 
 
 
 
 
 
 
 
 
71
  ## Run benchmarks
72
 
73
  ```bash
74
  python3 scripts/benchmark_subprocesses.py --runs 2 --bars 4 --output docs/benchmark-subprocesses.json
75
- python3 scripts/test_sse_and_review_hits.py
76
  ```
77
 
78
  The benchmark uses synthetic drum fixtures and `stem=all` so the DSP stages are measured without Demucs model download/runtime noise.
@@ -93,6 +121,20 @@ Then poll the returned job id:
93
  curl http://127.0.0.1:7860/api/jobs/<job-id>
94
  ```
95
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96
  List active/completed runs:
97
 
98
  ```bash
@@ -103,11 +145,14 @@ curl http://127.0.0.1:7860/api/jobs
103
 
104
  | Path | Purpose |
105
  |---|---|
106
- | `app.py` | FastAPI app, static UI serving, job API, run history, artifact downloads |
107
  | `pipeline_runner.py` | Timed extraction pipeline, disk stem/source cache, batch/online clustering routing |
108
  | `sample_extractor.py` | Core DSP/sample extraction implementation |
109
- | `web/` | Custom no-build browser frontend with waveform, hit review, and sample audition |
 
110
  | `scripts/benchmark_subprocesses.py` | Synthetic benchmark runner for stage timings |
 
 
111
  | `docs/` | Review, timing, API, UI, feature, task, progress, and remaining-work documentation |
112
  | `legacy/` | Previous Gradio apps retained for reference |
113
 
@@ -122,6 +167,7 @@ Each run is stored under `.runs/<job-id>/output/`:
122
  - `samples/*.wav`
123
  - `review/hits/*.wav`
124
  - `manifest.json`
 
125
 
126
  Generated runtime directories are ignored by git:
127
 
 
10
 
11
  # Drum Sample Extractor
12
 
13
+ A custom FastAPI + browser workstation for extracting, reviewing, and now semantically supervising reusable drum samples from an audio file.
14
 
15
+ The pipeline can isolate a stem with Demucs, detect onsets, classify hits, cluster similar transients, choose representative samples, optionally synthesize alternate samples, and export WAVs, MIDI, reconstruction audio, manifests, and a complete ZIP sample pack. The interactive layer stores user corrections as replayable semantic state beside each run manifest.
16
 
17
  ## Current status
18
 
19
  The project is usable as a local/Hugging Face Space application. Gradio is no longer the active UI; the active app is a custom FastAPI backend plus a no-build browser frontend.
20
 
21
+ Implemented:
22
 
23
  - Custom web frontend in `web/`, served by `app.py`.
24
+ - FastAPI job API with upload, polling, safe artifact downloads, config, health, cache clearing, run history, and SSE progress.
25
  - Timed pipeline runner in `pipeline_runner.py`.
26
  - Per-stage timing in every `manifest.json`.
27
  - Two clustering modes:
 
31
  - Run history panel indexing `.runs/*/output/manifest.json`.
32
  - Individual review WAVs for every detected hit under `review/hits/`.
33
  - Click-to-audition workflow for waveform onsets, detected hit rows, and representative sample rows.
34
+ - Interactive supervised state in `supervised_state.py`:
35
+ - persisted `supervision_state.json`,
36
+ - hit/cluster confidence,
37
+ - outlier-first review queue,
38
+ - constraints,
39
+ - event log,
40
+ - suggestions,
41
+ - undo stack.
42
+ - Supervision UI:
43
+ - selected-hit actions,
44
+ - move hit to cluster,
45
+ - pull hit into a new cluster,
46
+ - accept/favorite hit,
47
+ - suppress hit as bleed,
48
+ - lock/unlock cluster,
49
+ - suggestion inbox,
50
+ - cluster explanation drawer,
51
+ - constraint/event log.
52
+ - Documentation for features, progress, tasks, API, timing, hit review, realtime suitability, UI, remaining work, and interactive UX.
53
  - Legacy Gradio apps preserved in `legacy/` for reference only.
54
 
55
  Not fully complete yet:
56
 
57
+ - Semantic edits do not yet regenerate WAV/MIDI/ZIP exports.
58
+ - No force-onset/click-to-add missed onset yet.
59
+ - No restore for suppressed hits yet.
60
+ - No true cached feature-vector local reclustering yet.
61
+ - No frontend TypeScript build/test harness yet.
62
  - Demucs remains offline/batch by design.
63
 
64
  See:
 
66
  - `docs/FEATURES.md`
67
  - `docs/TASKS.md`
68
  - `docs/PROGRESS.md`
69
+ - `docs/API.md`
70
+ - `docs/interactive-ux/README.md`
71
  - `docs/REMAINING_WORK.md`
72
 
73
  ## Run locally
 
88
 
89
  That bypasses Demucs and uses the near-realtime clustering path.
90
 
91
+ ## Run checks
92
+
93
+ ```bash
94
+ python3 -m py_compile app.py pipeline_runner.py sample_extractor.py supervised_state.py scripts/*.py
95
+ node --check web/app.js
96
+ python3 scripts/test_sse_and_review_hits.py
97
+ python3 scripts/test_interactive_supervision.py
98
+ ```
99
+
100
  ## Run benchmarks
101
 
102
  ```bash
103
  python3 scripts/benchmark_subprocesses.py --runs 2 --bars 4 --output docs/benchmark-subprocesses.json
 
104
  ```
105
 
106
  The benchmark uses synthetic drum fixtures and `stem=all` so the DSP stages are measured without Demucs model download/runtime noise.
 
121
  curl http://127.0.0.1:7860/api/jobs/<job-id>
122
  ```
123
 
124
+ Read supervised state:
125
+
126
+ ```bash
127
+ curl http://127.0.0.1:7860/api/jobs/<job-id>/state
128
+ ```
129
+
130
+ Move a hit into a target cluster:
131
+
132
+ ```bash
133
+ curl -X POST http://127.0.0.1:7860/api/jobs/<job-id>/hits/hit%3A00003/move \
134
+ -H 'Content-Type: application/json' \
135
+ -d '{"target_cluster_id":"cluster:0"}'
136
+ ```
137
+
138
  List active/completed runs:
139
 
140
  ```bash
 
145
 
146
  | Path | Purpose |
147
  |---|---|
148
+ | `app.py` | FastAPI app, static UI serving, job API, run history, artifact downloads, supervised editing endpoints |
149
  | `pipeline_runner.py` | Timed extraction pipeline, disk stem/source cache, batch/online clustering routing |
150
  | `sample_extractor.py` | Core DSP/sample extraction implementation |
151
+ | `supervised_state.py` | Persistent semantic state, confidence, constraints, events, suggestions, undo |
152
+ | `web/` | Custom no-build browser frontend with waveform, hit review, sample audition, and supervision panel |
153
  | `scripts/benchmark_subprocesses.py` | Synthetic benchmark runner for stage timings |
154
+ | `scripts/test_interactive_supervision.py` | Smoke test for supervised state endpoints |
155
+ | `docs/interactive-ux/` | Supplied interactive UX docs aligned to current implementation |
156
  | `docs/` | Review, timing, API, UI, feature, task, progress, and remaining-work documentation |
157
  | `legacy/` | Previous Gradio apps retained for reference |
158
 
 
167
  - `samples/*.wav`
168
  - `review/hits/*.wav`
169
  - `manifest.json`
170
+ - `supervision_state.json`
171
 
172
  Generated runtime directories are ignored by git:
173
 
app.py CHANGED
@@ -19,20 +19,33 @@ from pathlib import Path
19
  from threading import Lock
20
  from typing import Any
21
 
22
- from fastapi import FastAPI, File, Form, HTTPException, UploadFile
23
  from fastapi.middleware.cors import CORSMiddleware
24
  from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
25
  from fastapi.staticfiles import StaticFiles
26
 
27
  from pipeline_runner import PipelineParams, clear_disk_cache, initial_stages, run_extraction_pipeline
28
  from sample_extractor import DEMUCS_MODELS, DEMUCS_STEMS, cache_clear
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
  ROOT = Path(__file__).resolve().parent
31
  WEB_DIR = ROOT / "web"
32
  RUNS_DIR = ROOT / ".runs"
33
  RUNS_DIR.mkdir(exist_ok=True)
34
 
35
- app = FastAPI(title="Drum Sample Extractor", version="11.1.0")
36
  app.add_middleware(
37
  CORSMiddleware,
38
  allow_origins=["*"],
@@ -155,6 +168,7 @@ def _run_job(job_id: str) -> None:
155
 
156
  try:
157
  result = run_extraction_pipeline(input_path, output_dir, PipelineParams.from_mapping(params), progress_cb=progress)
 
158
  _update_job(job_id, status="complete", result=asdict(result), error=None)
159
  except Exception as exc: # deliberately explicit for UI diagnostics
160
  _update_job(job_id, status="error", error=str(exc), traceback=traceback.format_exc())
@@ -248,6 +262,33 @@ def get_job(job_id: str) -> dict[str, Any]:
248
  raise HTTPException(status_code=404, detail="Job not found")
249
 
250
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
251
  @app.get("/api/jobs/{job_id}/events")
252
  def get_job_events(job_id: str) -> StreamingResponse:
253
  with jobs_lock:
@@ -286,6 +327,127 @@ def get_job_events(job_id: str) -> StreamingResponse:
286
  )
287
 
288
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
289
  @app.get("/api/jobs/{job_id}/files/{relative_path:path}")
290
  def get_job_file(job_id: str, relative_path: str) -> FileResponse:
291
  root = (RUNS_DIR / job_id / "output").resolve()
 
19
  from threading import Lock
20
  from typing import Any
21
 
22
+ from fastapi import Body, FastAPI, File, Form, HTTPException, UploadFile
23
  from fastapi.middleware.cors import CORSMiddleware
24
  from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
25
  from fastapi.staticfiles import StaticFiles
26
 
27
  from pipeline_runner import PipelineParams, clear_disk_cache, initial_stages, run_extraction_pipeline
28
  from sample_extractor import DEMUCS_MODELS, DEMUCS_STEMS, cache_clear
29
+ from supervised_state import (
30
+ accept_suggestion,
31
+ explain_cluster as build_cluster_explanation,
32
+ load_or_create_state,
33
+ lock_cluster as apply_cluster_lock,
34
+ move_hit as apply_hit_move,
35
+ public_state,
36
+ pull_hit_to_new_cluster,
37
+ reject_suggestion,
38
+ set_hit_review_status,
39
+ suppress_hit as apply_hit_suppression,
40
+ undo_last as apply_undo,
41
+ )
42
 
43
  ROOT = Path(__file__).resolve().parent
44
  WEB_DIR = ROOT / "web"
45
  RUNS_DIR = ROOT / ".runs"
46
  RUNS_DIR.mkdir(exist_ok=True)
47
 
48
+ app = FastAPI(title="Drum Sample Extractor", version="12.0.0")
49
  app.add_middleware(
50
  CORSMiddleware,
51
  allow_origins=["*"],
 
168
 
169
  try:
170
  result = run_extraction_pipeline(input_path, output_dir, PipelineParams.from_mapping(params), progress_cb=progress)
171
+ load_or_create_state(job_id, output_dir)
172
  _update_job(job_id, status="complete", result=asdict(result), error=None)
173
  except Exception as exc: # deliberately explicit for UI diagnostics
174
  _update_job(job_id, status="error", error=str(exc), traceback=traceback.format_exc())
 
262
  raise HTTPException(status_code=404, detail="Job not found")
263
 
264
 
265
+
266
+
267
+ def _job_output_dir(job_id: str) -> Path:
268
+ with jobs_lock:
269
+ job = jobs.get(job_id)
270
+ if job and job.get("output_dir"):
271
+ return Path(job["output_dir"])
272
+ manifest = _manifest_path(job_id)
273
+ if manifest.exists():
274
+ return manifest.parent
275
+ raise HTTPException(status_code=404, detail="Job not found")
276
+
277
+
278
+ def _state_payload(job_id: str) -> dict[str, Any]:
279
+ out = _job_output_dir(job_id)
280
+ try:
281
+ state = load_or_create_state(job_id, out)
282
+ except FileNotFoundError as exc:
283
+ raise HTTPException(status_code=409, detail="Job has no manifest yet; wait until extraction completes") from exc
284
+ except Exception as exc:
285
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
286
+ return public_state(state, url_for=lambda rel: _job_url(job_id, rel))
287
+
288
+
289
+ def _json_patch(payload: dict[str, Any] | None) -> dict[str, Any]:
290
+ return dict(payload or {})
291
+
292
  @app.get("/api/jobs/{job_id}/events")
293
  def get_job_events(job_id: str) -> StreamingResponse:
294
  with jobs_lock:
 
327
  )
328
 
329
 
330
+
331
+
332
+ @app.get("/api/jobs/{job_id}/state")
333
+ def get_job_state(job_id: str) -> dict[str, Any]:
334
+ return _state_payload(job_id)
335
+
336
+
337
+ @app.post("/api/jobs/{job_id}/hits/{hit_id}/move")
338
+ def post_move_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
339
+ target_cluster_id = _json_patch(payload).get("target_cluster_id")
340
+ if not target_cluster_id:
341
+ raise HTTPException(status_code=400, detail="target_cluster_id is required")
342
+ try:
343
+ apply_hit_move(_job_output_dir(job_id), job_id, hit_id, str(target_cluster_id))
344
+ except KeyError as exc:
345
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
346
+ except Exception as exc:
347
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
348
+ return _state_payload(job_id)
349
+
350
+
351
+ @app.post("/api/jobs/{job_id}/hits/{hit_id}/pull-out")
352
+ def post_pull_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
353
+ label = _json_patch(payload).get("label")
354
+ try:
355
+ pull_hit_to_new_cluster(_job_output_dir(job_id), job_id, hit_id, label=label)
356
+ except KeyError as exc:
357
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
358
+ except Exception as exc:
359
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
360
+ return _state_payload(job_id)
361
+
362
+
363
+ @app.post("/api/jobs/{job_id}/hits/{hit_id}/suppress")
364
+ def post_suppress_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
365
+ reason = str(_json_patch(payload).get("reason") or "bleed")
366
+ try:
367
+ apply_hit_suppression(_job_output_dir(job_id), job_id, hit_id, reason=reason)
368
+ except KeyError as exc:
369
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
370
+ except Exception as exc:
371
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
372
+ return _state_payload(job_id)
373
+
374
+
375
+ @app.post("/api/jobs/{job_id}/hits/{hit_id}/review")
376
+ def post_review_hit(job_id: str, hit_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
377
+ status = str(_json_patch(payload).get("status") or "accepted")
378
+ try:
379
+ set_hit_review_status(_job_output_dir(job_id), job_id, hit_id, status=status)
380
+ except KeyError as exc:
381
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
382
+ except ValueError as exc:
383
+ raise HTTPException(status_code=400, detail=str(exc)) from exc
384
+ except Exception as exc:
385
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
386
+ return _state_payload(job_id)
387
+
388
+
389
+ @app.post("/api/jobs/{job_id}/clusters/{cluster_id:path}/lock")
390
+ def post_lock_cluster(job_id: str, cluster_id: str, payload: dict[str, Any] = Body(default_factory=dict)) -> dict[str, Any]:
391
+ locked = bool(_json_patch(payload).get("locked", True))
392
+ try:
393
+ apply_cluster_lock(_job_output_dir(job_id), job_id, cluster_id, locked=locked)
394
+ except KeyError as exc:
395
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
396
+ except Exception as exc:
397
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
398
+ return _state_payload(job_id)
399
+
400
+
401
+ @app.get("/api/jobs/{job_id}/suggestions")
402
+ def get_suggestions(job_id: str) -> dict[str, Any]:
403
+ state = _state_payload(job_id)
404
+ return {"suggestions": state.get("suggestions", []), "summary": state.get("summary", {})}
405
+
406
+
407
+ @app.post("/api/jobs/{job_id}/suggestions/{suggestion_id}/accept")
408
+ def post_accept_suggestion(job_id: str, suggestion_id: str) -> dict[str, Any]:
409
+ try:
410
+ accept_suggestion(_job_output_dir(job_id), job_id, suggestion_id)
411
+ except KeyError as exc:
412
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
413
+ except ValueError as exc:
414
+ raise HTTPException(status_code=400, detail=str(exc)) from exc
415
+ except Exception as exc:
416
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
417
+ return _state_payload(job_id)
418
+
419
+
420
+ @app.post("/api/jobs/{job_id}/suggestions/{suggestion_id}/reject")
421
+ def post_reject_suggestion(job_id: str, suggestion_id: str) -> dict[str, Any]:
422
+ try:
423
+ reject_suggestion(_job_output_dir(job_id), job_id, suggestion_id)
424
+ except KeyError as exc:
425
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
426
+ except Exception as exc:
427
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
428
+ return _state_payload(job_id)
429
+
430
+
431
+ @app.get("/api/jobs/{job_id}/explain/cluster/{cluster_id:path}")
432
+ def get_cluster_explanation(job_id: str, cluster_id: str) -> dict[str, Any]:
433
+ out = _job_output_dir(job_id)
434
+ state = load_or_create_state(job_id, out)
435
+ try:
436
+ explanation = build_cluster_explanation(state, cluster_id)
437
+ except KeyError as exc:
438
+ raise HTTPException(status_code=404, detail=str(exc)) from exc
439
+ return explanation
440
+
441
+
442
+ @app.post("/api/jobs/{job_id}/undo")
443
+ def post_undo(job_id: str) -> dict[str, Any]:
444
+ try:
445
+ apply_undo(_job_output_dir(job_id), job_id)
446
+ except Exception as exc:
447
+ raise HTTPException(status_code=500, detail=str(exc)) from exc
448
+ return _state_payload(job_id)
449
+
450
+
451
  @app.get("/api/jobs/{job_id}/files/{relative_path:path}")
452
  def get_job_file(job_id: str, relative_path: str) -> FileResponse:
453
  root = (RUNS_DIR / job_id / "output").resolve()
docs/API.md CHANGED
@@ -212,3 +212,170 @@ Defined in `pipeline_runner.PipelineParams`.
212
  | `subdivision` | `16` | MIDI grid subdivision. |
213
  | `device` | `cpu` | Torch device for Demucs. |
214
  | `use_disk_cache` | `true` | Cache decoded full mix/stems by source digest and extraction settings. |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
212
  | `subdivision` | `16` | MIDI grid subdivision. |
213
  | `device` | `cpu` | Torch device for Demucs. |
214
  | `use_disk_cache` | `true` | Cache decoded full mix/stems by source digest and extraction settings. |
215
+
216
+ ## Interactive supervision API
217
+
218
+ The interactive supervision API is backed by `supervised_state.py` and persists state as:
219
+
220
+ ```text
221
+ .runs/<job_id>/output/supervision_state.json
222
+ ```
223
+
224
+ The batch `manifest.json` remains immutable. Supervised edits currently update semantic state only; they do not yet regenerate WAV/MIDI/ZIP artifacts.
225
+
226
+ ### `GET /api/jobs/{job_id}/state`
227
+
228
+ Returns the supervised state for a completed job. If the state file does not exist yet, it is created from the batch manifest.
229
+
230
+ Response keys:
231
+
232
+ | Key | Meaning |
233
+ |---|---|
234
+ | `summary` | Counts for hits, clusters, constraints, events, suggestions, suppressed hits, locked clusters, undo availability. |
235
+ | `hits` | Semantic hit rows with confidence, suppression/favorite/review flags, file URLs, and current cluster assignment. |
236
+ | `clusters` | Semantic clusters with hit IDs, representative hit, confidence, locked state, and suppressed count. |
237
+ | `review_queue` | Low-confidence/high-priority hits sorted for review. |
238
+ | `constraints` | Recent replayable constraints. |
239
+ | `events` | Recent state mutation events. |
240
+ | `suggestions` | Open move/split/suppress suggestions. |
241
+
242
+ ```bash
243
+ curl http://127.0.0.1:7860/api/jobs/<job-id>/state
244
+ ```
245
+
246
+ ### `POST /api/jobs/{job_id}/hits/{hit_id}/move`
247
+
248
+ Moves a hit into an existing target cluster.
249
+
250
+ Body:
251
+
252
+ ```json
253
+ {"target_cluster_id":"cluster:0"}
254
+ ```
255
+
256
+ Effects:
257
+
258
+ - updates hit membership in `supervision_state.json`,
259
+ - creates `force-cluster`,
260
+ - creates `must-link` to the target representative when possible,
261
+ - appends events,
262
+ - recomputes confidence/review queue,
263
+ - may create similar-hit move suggestions,
264
+ - pushes an undo snapshot.
265
+
266
+ Example:
267
+
268
+ ```bash
269
+ curl -X POST http://127.0.0.1:7860/api/jobs/<job-id>/hits/hit%3A00003/move \
270
+ -H 'Content-Type: application/json' \
271
+ -d '{"target_cluster_id":"cluster:0"}'
272
+ ```
273
+
274
+ ### `POST /api/jobs/{job_id}/hits/{hit_id}/pull-out`
275
+
276
+ Pulls a hit into a new user cluster.
277
+
278
+ Optional body:
279
+
280
+ ```json
281
+ {"label":"snare_user_1"}
282
+ ```
283
+
284
+ Effects:
285
+
286
+ - creates a new `cluster:user:*` cluster,
287
+ - creates `cannot-link` from the source representative when possible,
288
+ - creates `force-cluster`,
289
+ - may create split suggestions,
290
+ - pushes an undo snapshot.
291
+
292
+ ### `POST /api/jobs/{job_id}/hits/{hit_id}/suppress`
293
+
294
+ Marks a hit as bleed/noise/non-sample material.
295
+
296
+ Body:
297
+
298
+ ```json
299
+ {"reason":"bleed"}
300
+ ```
301
+
302
+ Effects:
303
+
304
+ - marks the hit `suppressed`,
305
+ - creates `suppress-pattern`,
306
+ - may create similar suppression suggestions,
307
+ - recomputes confidence and review priority.
308
+
309
+ ### `POST /api/jobs/{job_id}/hits/{hit_id}/review`
310
+
311
+ Stores a review decision for a hit.
312
+
313
+ Body:
314
+
315
+ ```json
316
+ {"status":"accepted"}
317
+ ```
318
+
319
+ Supported statuses:
320
+
321
+ | Status | Meaning |
322
+ |---|---|
323
+ | `unreviewed` | Clear explicit review status. |
324
+ | `accepted` | Mark the hit as reviewed/accepted. |
325
+ | `favorite` | Mark as favorite and pin as semantic representative for its cluster. |
326
+
327
+ ### `POST /api/jobs/{job_id}/clusters/{cluster_id}/lock`
328
+
329
+ Locks or unlocks a cluster.
330
+
331
+ Body:
332
+
333
+ ```json
334
+ {"locked":true}
335
+ ```
336
+
337
+ Lock state is persisted and shown in the cluster board. It does not yet alter future full pipeline reruns.
338
+
339
+ ### `GET /api/jobs/{job_id}/suggestions`
340
+
341
+ Returns open suggestions and the state summary.
342
+
343
+ ```bash
344
+ curl http://127.0.0.1:7860/api/jobs/<job-id>/suggestions
345
+ ```
346
+
347
+ ### `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/accept`
348
+
349
+ Applies a suggestion and records accepted constraints/examples.
350
+
351
+ Supported suggestion types:
352
+
353
+ - `move-hits`,
354
+ - `split-hits`,
355
+ - `suppress-hits`.
356
+
357
+ ### `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/reject`
358
+
359
+ Marks a suggestion rejected and records an event.
360
+
361
+ ### `GET /api/jobs/{job_id}/explain/cluster/{cluster_id}`
362
+
363
+ Returns explanation data for one cluster:
364
+
365
+ - label,
366
+ - locked state,
367
+ - confidence and reasons,
368
+ - representative hit,
369
+ - hit counts,
370
+ - label distribution,
371
+ - lowest-confidence outliers,
372
+ - relevant constraints,
373
+ - summary string.
374
+
375
+ ### `POST /api/jobs/{job_id}/undo`
376
+
377
+ Restores the previous semantic state snapshot if available.
378
+
379
+ ```bash
380
+ curl -X POST http://127.0.0.1:7860/api/jobs/<job-id>/undo
381
+ ```
docs/FEATURES.md CHANGED
@@ -60,3 +60,25 @@ Turn an input audio file into a practical drum sample pack: detected hits, group
60
  - Realtime Demucs. It is not realistic for this use-case and should remain offline/cached.
61
  - Perfect source separation. Stem quality depends on model choice and input material.
62
  - Full DAW/sample-editor UX. This pass creates the workstation foundation; detailed editing is next.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
  - Realtime Demucs. It is not realistic for this use-case and should remain offline/cached.
61
  - Perfect source separation. Stem quality depends on model choice and input material.
62
  - Full DAW/sample-editor UX. This pass creates the workstation foundation; detailed editing is next.
63
+
64
+ ## Interactive supervised UX features
65
+
66
+ | Area | Feature | Status | Notes |
67
+ |---|---|---|---|
68
+ | Supervision | Supplied UX docs embedded | Implemented | Added and aligned under `docs/interactive-ux/`. |
69
+ | Supervision | Persistent semantic state | Implemented | `supervision_state.json` is created beside each run manifest. |
70
+ | Supervision | Hit/cluster state model | Implemented | State tracks current cluster assignment, confidence, suppression, favorite/review flags, and representatives. |
71
+ | Supervision | Constraint store | Implemented | Stores `force-cluster`, `must-link`, `cannot-link`, `lock-cluster`, `suppress-pattern`, and `pin-representative`. |
72
+ | Supervision | Event log | Implemented | State changes append replay/audit events. |
73
+ | Supervision | Undo stack | Implemented | Last semantic edit can be undone. |
74
+ | Supervision | Confidence scoring | Partial | Heuristic and deterministic; does not yet use cached mel/transient feature margins. |
75
+ | Supervision | Outlier-first review queue | Implemented | UI prioritizes low-confidence/singleton/unstable hits. |
76
+ | Supervision | Move hit to cluster | Implemented | Creates supervision constraints and may produce suggestions. |
77
+ | Supervision | Pull hit into new cluster | Implemented | Creates a user cluster and cannot-link/force-cluster constraints. |
78
+ | Supervision | Lock cluster | Implemented | Lock state persists and updates confidence/UI. |
79
+ | Supervision | Suppress hit as bleed | Implemented | Marks hit suppressed, stores suppress-pattern, may suggest similar suppressions. |
80
+ | Supervision | Favorite representative | Partial | Pins semantic representative; supervised export does not yet honor it. |
81
+ | Supervision | Suggestion inbox | Partial | Move/split/suppress suggestions can be accepted/rejected; exact diff preview is not implemented. |
82
+ | Supervision | Cluster explanation | Implemented | Backend and UI show confidence reasons, label distribution, outliers, and constraints. |
83
+ | Supervision | Edited artifact re-export | Not implemented | Semantic edits do not yet regenerate sample WAVs, MIDI, reconstruction, or ZIP. |
84
+ | Supervision | Force-onset from waveform | Not implemented | Waveform click currently auditions nearest existing hit only. |
docs/PROGRESS.md CHANGED
@@ -81,3 +81,67 @@ Completed in this pass:
81
  Outcome:
82
 
83
  The app now supports a real review loop for inspecting what the onset detector and clustering produced. Users can audition individual detected slices, representative samples, stem audio, and reconstruction audio from one screen. Progress updates are lower-latency and less wasteful via SSE while still remaining robust in browsers that need polling fallback.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  Outcome:
82
 
83
  The app now supports a real review loop for inspecting what the onset detector and clustering produced. Users can audition individual detected slices, representative samples, stem audio, and reconstruction audio from one screen. Progress updates are lower-latency and less wasteful via SSE while still remaining robust in browsers that need polling fallback.
84
+
85
+ ## Pass 4: interactive supervised UX foundation
86
+
87
+ Completed in this pass:
88
+
89
+ 1. Added the supplied interactive UX document set under `docs/interactive-ux/`.
90
+ 2. Read and aligned the UX documents with the project as currently implemented.
91
+ 3. Added `supervised_state.py` for persistent semantic state beside each completed run manifest.
92
+ 4. Added `supervision_state.json` generation after each successful extraction.
93
+ 5. Added state schema for hits, clusters, constraints, events, suggestions, confidence, review queue, and undo snapshots.
94
+ 6. Added supervised editing endpoints:
95
+ - `GET /api/jobs/{job_id}/state`
96
+ - `POST /api/jobs/{job_id}/hits/{hit_id}/move`
97
+ - `POST /api/jobs/{job_id}/hits/{hit_id}/pull-out`
98
+ - `POST /api/jobs/{job_id}/hits/{hit_id}/suppress`
99
+ - `POST /api/jobs/{job_id}/hits/{hit_id}/review`
100
+ - `POST /api/jobs/{job_id}/clusters/{cluster_id}/lock`
101
+ - `GET /api/jobs/{job_id}/suggestions`
102
+ - `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/accept`
103
+ - `POST /api/jobs/{job_id}/suggestions/{suggestion_id}/reject`
104
+ - `GET /api/jobs/{job_id}/explain/cluster/{cluster_id}`
105
+ - `POST /api/jobs/{job_id}/undo`
106
+ 7. Added an interactive supervision UI panel with:
107
+ - state summary,
108
+ - selected-hit actions,
109
+ - target cluster picker,
110
+ - outlier-first review queue,
111
+ - cluster board,
112
+ - suggestion inbox,
113
+ - constraint/event log,
114
+ - cluster explanation drawer.
115
+ 8. Added `scripts/test_interactive_supervision.py` to verify the supervised API loop.
116
+
117
+ Outcome:
118
+
119
+ The app is now an extraction and supervised-review workstation at the semantic-state level. User corrections are persisted as constraints/events and can be inspected, suggested from, and undone. The next required step is edited-state export so these decisions affect downloadable artifacts.
120
+
121
+ ## Current assessment after Pass 4
122
+
123
+ The project now satisfies the first interactive UX milestone for replayable supervised state:
124
+
125
+ ```text
126
+ analyze audio
127
+ → inspect hits/clusters
128
+ → move/pull/suppress/favorite/lock
129
+ → persist constraints/events
130
+ → update confidence and review queue
131
+ → generate/accept/reject suggestions
132
+ → explain clusters
133
+ → undo semantic edits
134
+ → reload completed run with decisions intact
135
+ ```
136
+
137
+ It does not yet satisfy the full workstation loop because edited semantic state is not yet rendered into updated sample WAVs, MIDI, reconstruction, or ZIP output.
138
+
139
+ ## Next recommended pass after Pass 4
140
+
141
+ 1. Add supervised re-export endpoint.
142
+ 2. Exclude suppressed hits from supervised exports.
143
+ 3. Honor favorite/pinned representatives in supervised sample WAVs.
144
+ 4. Add force-onset endpoint using cached `stem.wav`.
145
+ 5. Add add-onset mode to the waveform UI.
146
+ 6. Add restore suppressed hit and batch restore.
147
+ 7. Add feature-vector cache for true local reclustering.
docs/REMAINING_WORK.md CHANGED
@@ -33,3 +33,24 @@ The project is now a usable extraction workstation, not a complete interactive s
33
  5. Add lower-level progress hooks inside expensive stages where practical.
34
  6. Convert frontend to TypeScript and add UI tests.
35
  7. Add an in-app benchmark/parameter profile panel.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  5. Add lower-level progress hooks inside expensive stages where practical.
34
  6. Convert frontend to TypeScript and add UI tests.
35
  7. Add an in-app benchmark/parameter profile panel.
36
+
37
+ ## Remaining after interactive UX foundation
38
+
39
+ Completed since the previous remaining-work snapshot:
40
+
41
+ - Supplied `docs/interactive-ux` document set embedded and aligned.
42
+ - Persistent supervised state added via `supervised_state.py` and `supervision_state.json`.
43
+ - Constraint store and event log added.
44
+ - Hit/cluster confidence and outlier-first review queue added.
45
+ - Move hit, pull-out, lock/unlock, suppress, review/favorite, suggestions, explanations, and undo endpoints added.
46
+ - Interactive supervision UI panel added.
47
+
48
+ Highest-priority remaining work now:
49
+
50
+ 1. **Supervised artifact export**: regenerate edited sample WAVs, MIDI, reconstruction, manifest, and ZIP from `supervision_state.json` without rerunning Demucs/onset detection.
51
+ 2. **Force-onset correction**: add an onset by clicking/shift-clicking the waveform, slice from cached `stem.wav`, classify, assign, and store a `force-onset` constraint.
52
+ 3. **Suppression restore**: restore suppressed hits individually and in batches.
53
+ 4. **Real constrained local reclustering**: cache hit feature vectors and recompute affected neighborhoods after edits.
54
+ 5. **Suggestion diff preview**: show exact before/after membership changes before accepting a suggestion.
55
+ 6. **Constraint violation detection**: explicitly report conflicting user constraints.
56
+ 7. **Frontend tests and TypeScript migration**: harden the increasingly stateful UI.
docs/TASKS.md CHANGED
@@ -58,3 +58,32 @@ Last updated: 2026-05-12
58
  - [ ] Add side-by-side run comparison.
59
  - [ ] Convert frontend to TypeScript with a small Vite build once UX stabilizes.
60
  - [ ] Add automated browser-level UI tests.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
  - [ ] Add side-by-side run comparison.
59
  - [ ] Convert frontend to TypeScript with a small Vite build once UX stabilizes.
60
  - [ ] Add automated browser-level UI tests.
61
+
62
+ ## Interactive UX continuation tasks
63
+
64
+ | Task | Status | Evidence |
65
+ |---|---:|---|
66
+ | Add supplied interactive UX docs under `docs/interactive-ux/` | Done | `docs/interactive-ux/*.md`. |
67
+ | Read and align UX docs with current implementation | Done | Status sections updated in every interactive UX document. |
68
+ | Add persistent semantic job state | Done | `supervised_state.py`, `supervision_state.json`. |
69
+ | Add event log and constraint store | Done | `supervised_state.py`; tested by `scripts/test_interactive_supervision.py`. |
70
+ | Add hit/cluster confidence and review queue | Done/Partial | Heuristic confidence and review queue implemented; feature-margin confidence remains open. |
71
+ | Add move hit to cluster | Done | `POST /api/jobs/{job_id}/hits/{hit_id}/move`. |
72
+ | Add pull hit into new cluster | Done | `POST /api/jobs/{job_id}/hits/{hit_id}/pull-out`. |
73
+ | Add cluster lock/unlock | Done | `POST /api/jobs/{job_id}/clusters/{cluster_id}/lock`. |
74
+ | Add suppress hit as bleed/noise | Done | `POST /api/jobs/{job_id}/hits/{hit_id}/suppress`. |
75
+ | Add accept/favorite hit action | Done/Partial | `POST /api/jobs/{job_id}/hits/{hit_id}/review`; artifact re-export still open. |
76
+ | Add suggestion inbox | Done/Partial | UI/API supports accept/reject; exact diff preview still open. |
77
+ | Add cluster explanation drawer | Done | `GET /api/jobs/{job_id}/explain/cluster/{cluster_id}` plus UI drawer. |
78
+ | Add semantic undo | Done | `POST /api/jobs/{job_id}/undo`. |
79
+ | Add supervised export from edited state | Todo | Needed so corrections affect ZIP/MIDI/WAV outputs. |
80
+ | Add click-to-add missed onset | Todo | Needed for `force-onset` constraints and direct onset correction. |
81
+ | Add suppressed-hit restore | Todo | Needed as the safety counterpart to suppression. |
82
+ | Add true local feature-neighborhood reclustering | Todo | Requires cached feature vectors and constraint-aware assignment. |
83
+
84
+ ## Latest validation tasks
85
+
86
+ - [x] `python3 -m py_compile app.py pipeline_runner.py sample_extractor.py supervised_state.py scripts/*.py`
87
+ - [x] `node --check web/app.js`
88
+ - [x] `python3 scripts/test_sse_and_review_hits.py`
89
+ - [x] `python3 scripts/test_interactive_supervision.py`
docs/interactive-ux/ARCHITECTURE_NOTES.md ADDED
@@ -0,0 +1,204 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Architecture notes for supervised interactive extraction
2
+
3
+ ## Required shift
4
+
5
+ Original extraction flow:
6
+
7
+ ```text
8
+ audio → pipeline → result artifacts
9
+ ```
10
+
11
+ Current interactive foundation:
12
+
13
+ ```text
14
+ audio/cache → immutable manifest/artifacts → supervision_state.json → reactive UI → user constraints/events/suggestions
15
+ ```
16
+
17
+ The current implementation deliberately keeps the batch extraction artifacts immutable. Interactive edits mutate `supervision_state.json`, not the original `manifest.json`, hit WAVs, representative WAVs, MIDI, reconstruction, or ZIP. This keeps edits cheap and reversible, but supervised re-export is the next architectural step.
18
+
19
+ ## Implemented modules
20
+
21
+ | Module/file | Responsibility |
22
+ |---|---|
23
+ | `pipeline_runner.py` | Batch extraction, timing, manifests, review-hit WAV exports |
24
+ | `sample_extractor.py` | Audio analysis, classification, batch/online clustering, export helpers |
25
+ | `supervised_state.py` | Persistent semantic job state, constraints, events, confidence, suggestions, undo |
26
+ | `app.py` | FastAPI endpoints for batch jobs and supervised state mutations |
27
+ | `web/app.js` | Browser state rendering, review queue, cluster board, suggestions, actions |
28
+ | `web/index.html` | Workstation layout and interactive supervision panel |
29
+ | `web/styles.css` | Visual treatment for low confidence, suppression, locks, panels |
30
+
31
+ ## Core entities as implemented
32
+
33
+ `supervised_state.py` stores JSON dictionaries equivalent to these shapes:
34
+
35
+ ```ts
36
+ type Hit = {
37
+ id: string;
38
+ index: number;
39
+ label: string;
40
+ cluster_id: string;
41
+ original_cluster_id: string;
42
+ cluster_label: string;
43
+ onset_sec: number;
44
+ duration_ms: number;
45
+ rms_energy: number;
46
+ spectral_centroid_hz: number;
47
+ file: string;
48
+ is_representative: boolean;
49
+ source: "detected" | "forced";
50
+ suppressed: boolean;
51
+ favorite: boolean;
52
+ review_status: "unreviewed" | "accepted" | "favorite" | "suppressed";
53
+ confidence: number;
54
+ confidence_reasons: string[];
55
+ explicit: boolean;
56
+ };
57
+
58
+ type Cluster = {
59
+ id: string;
60
+ label: string;
61
+ classification: string;
62
+ hit_ids: string[];
63
+ representative_hit_id: string | null;
64
+ locked: boolean;
65
+ user_named: boolean;
66
+ confidence: number;
67
+ confidence_reasons: string[];
68
+ suppressed_count: number;
69
+ original_id: string | null;
70
+ };
71
+
72
+ type Constraint =
73
+ | { id: string; type: "must-link"; a: string; b: string; source: string }
74
+ | { id: string; type: "cannot-link"; a: string; b: string; source: string }
75
+ | { id: string; type: "force-cluster"; hit_id: string; cluster_id: string; source: string }
76
+ | { id: string; type: "lock-cluster"; cluster_id: string; locked: boolean; source: string }
77
+ | { id: string; type: "suppress-pattern"; example_hit_id: string; reason: string; source: string }
78
+ | { id: string; type: "pin-representative"; hit_id: string; cluster_id: string; source: string };
79
+ ```
80
+
81
+ ## Current state file
82
+
83
+ Each completed run now gets:
84
+
85
+ ```text
86
+ .runs/<job_id>/output/manifest.json
87
+ .runs/<job_id>/output/supervision_state.json
88
+ ```
89
+
90
+ `manifest.json` is the immutable batch result. `supervision_state.json` is the mutable, replayable semantic state.
91
+
92
+ ## Implemented API additions
93
+
94
+ ```text
95
+ GET /api/jobs/{job_id}/state
96
+ POST /api/jobs/{job_id}/hits/{hit_id}/move
97
+ POST /api/jobs/{job_id}/hits/{hit_id}/pull-out
98
+ POST /api/jobs/{job_id}/hits/{hit_id}/suppress
99
+ POST /api/jobs/{job_id}/hits/{hit_id}/review
100
+ POST /api/jobs/{job_id}/clusters/{cluster_id}/lock
101
+ GET /api/jobs/{job_id}/suggestions
102
+ POST /api/jobs/{job_id}/suggestions/{suggestion_id}/accept
103
+ POST /api/jobs/{job_id}/suggestions/{suggestion_id}/reject
104
+ GET /api/jobs/{job_id}/explain/cluster/{cluster_id}
105
+ POST /api/jobs/{job_id}/undo
106
+ ```
107
+
108
+ ## Current confidence scoring
109
+
110
+ Initial confidence is heuristic and deterministic. It combines:
111
+
112
+ - cluster size,
113
+ - cluster label purity,
114
+ - representative presence,
115
+ - lock state,
116
+ - hit label agreement with cluster label,
117
+ - energy rank,
118
+ - rough duration reasonableness,
119
+ - representative/favorite/explicit assignment state,
120
+ - suppression state.
121
+
122
+ This is good enough to drive the review queue but should be replaced or supplemented by cached feature-vector margins.
123
+
124
+ ## Current suggestion engine
125
+
126
+ Implemented suggestion types:
127
+
128
+ ```ts
129
+ type Suggestion =
130
+ | { type: "move-hits"; hit_ids: string[]; target_cluster_id: string; confidence: number; reason: string }
131
+ | { type: "split-hits"; hit_ids: string[]; source_cluster_id: string; target_cluster_id: string; confidence: number; reason: string }
132
+ | { type: "suppress-hits"; hit_ids: string[]; confidence: number; reason: string };
133
+ ```
134
+
135
+ Suggestion generation currently uses label, spectral centroid, and RMS-energy similarity. Accepted suggestions become explicit constraints/examples.
136
+
137
+ ## Event log
138
+
139
+ Examples now emitted:
140
+
141
+ ```text
142
+ job.state.created
143
+ constraint.created
144
+ hit.moved
145
+ hit.pulled_out
146
+ cluster.locked
147
+ cluster.unlocked
148
+ hit.suppressed
149
+ hit.reviewed
150
+ suggestion.created
151
+ suggestion.accepted
152
+ suggestion.rejected
153
+ state.undo
154
+ ```
155
+
156
+ The UI renders recent events and constraints in the supervision panel.
157
+
158
+ ## Local recomputation boundary
159
+
160
+ Implemented now:
161
+
162
+ ```text
163
+ semantic edit
164
+ → update hit/cluster membership in supervision_state.json
165
+ → append constraints/events
166
+ → generate heuristic suggestions
167
+ → recompute hit/cluster confidence and review queue
168
+ ```
169
+
170
+ Not implemented yet:
171
+
172
+ ```text
173
+ semantic edit
174
+ → load cached feature vectors
175
+ → choose affected neighborhood
176
+ → run constrained local reclustering
177
+ → create preview diff
178
+ → optionally apply/re-export artifacts
179
+ ```
180
+
181
+ ## UI state implications
182
+
183
+ Implemented panels:
184
+
185
+ - waveform + onset audition,
186
+ - representative samples,
187
+ - detected hit review,
188
+ - outlier-first review queue,
189
+ - cluster board,
190
+ - suggestion inbox,
191
+ - constraint/history inspector,
192
+ - cluster explanation drawer,
193
+ - export/download panel for the original batch run.
194
+
195
+ Still missing:
196
+
197
+ - edited export panel,
198
+ - force-onset mode,
199
+ - suppression restore UI,
200
+ - side-by-side before/after diff preview.
201
+
202
+ ## Implementation warning
203
+
204
+ Automatic propagation must stay conservative. The current implementation follows this by creating suggestions rather than silently moving/suppressing batches. Every semantic edit is undoable.
docs/interactive-ux/FEASIBILITY_MATRIX.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Feature feasibility matrix
2
+
3
+ ## Scoring convention
4
+
5
+ All scores are between `0.0` and `1.0`.
6
+
7
+ For positive dimensions, higher is better:
8
+
9
+ - **Feasibility**: likelihood this can be implemented robustly with the current project direction.
10
+ - **Value UX**: how much nicer/faster the application feels.
11
+ - **Value quality**: expected improvement in extraction correctness.
12
+ - **Realtime fit**: how well the interaction can update from cached features without full reruns.
13
+ - **MVP fit**: whether it belongs in the first serious interactive version.
14
+
15
+ For cost dimensions, higher is harder/more expensive:
16
+
17
+ - **Complexity**: algorithmic/product complexity.
18
+ - **Effort**: implementation effort.
19
+
20
+ ## Matrix
21
+
22
+ | Interaction | Feasibility | Complexity | Effort | Value UX | Value quality | Realtime fit | MVP fit | Verdict |
23
+ |---|---:|---:|---:|---:|---:|---:|---:|---|
24
+ | Move sample to cluster → auto-tune clustering | 0.90 | 0.55 | 0.45 | 0.95 | 0.90 | 0.85 | 0.95 | Build early |
25
+ | Pull sample out → protect distinction | 0.90 | 0.55 | 0.45 | 0.90 | 0.90 | 0.85 | 0.95 | Build early |
26
+ | Lock confirmed cluster identity | 0.95 | 0.25 | 0.20 | 0.80 | 0.75 | 0.95 | 1.00 | Build immediately |
27
+ | Outlier-first review queue | 0.95 | 0.30 | 0.25 | 0.90 | 0.80 | 0.95 | 1.00 | Build immediately |
28
+ | Low-confidence visual emphasis | 0.95 | 0.25 | 0.20 | 0.85 | 0.65 | 0.95 | 0.95 | Build immediately |
29
+ | Bleed brush/suppression | 0.85 | 0.55 | 0.50 | 0.90 | 0.85 | 0.80 | 0.85 | Build early |
30
+ | Click-to-add missed onset | 0.90 | 0.45 | 0.40 | 0.90 | 0.80 | 0.90 | 0.90 | Build early |
31
+ | Cluster naming as reusable semantic hint | 0.75 | 0.65 | 0.55 | 0.75 | 0.65 | 0.70 | 0.40 | Useful, not first |
32
+ | Star/favorite sample → optimize around it | 0.95 | 0.35 | 0.30 | 0.75 | 0.70 | 0.90 | 0.75 | Build early |
33
+ | Explain this cluster | 0.85 | 0.50 | 0.45 | 0.80 | 0.65 | 0.85 | 0.80 | Build early |
34
+ | Live counterfactual parameter previews | 0.80 | 0.70 | 0.65 | 0.90 | 0.75 | 0.75 | 0.55 | High value, later |
35
+ | Temporal pattern supervision | 0.75 | 0.70 | 0.65 | 0.80 | 0.85 | 0.70 | 0.45 | Later |
36
+ | Reconstruction-error-driven correction | 0.70 | 0.80 | 0.75 | 0.85 | 0.90 | 0.55 | 0.35 | Later, powerful |
37
+ | Multi-resolution semantic clustering | 0.80 | 0.70 | 0.65 | 0.85 | 0.80 | 0.75 | 0.50 | Later |
38
+ | Auto-clean this family | 0.80 | 0.60 | 0.55 | 0.75 | 0.80 | 0.75 | 0.50 | Later |
39
+ | Drag clusters in semantic space | 0.70 | 0.75 | 0.75 | 0.90 | 0.65 | 0.65 | 0.30 | Cool, not first |
40
+ | Cluster gravity / physics metaphor | 0.60 | 0.80 | 0.80 | 0.80 | 0.50 | 0.60 | 0.20 | Risky novelty |
41
+ | Context-aware classification | 0.65 | 0.80 | 0.75 | 0.65 | 0.80 | 0.55 | 0.25 | Researchy |
42
+ | Teach mode across songs | 0.70 | 0.85 | 0.80 | 0.85 | 0.85 | 0.60 | 0.25 | Later platform feature |
43
+ | Predictive batch questions | 0.85 | 0.55 | 0.50 | 0.90 | 0.80 | 0.80 | 0.70 | Build after uncertainty scoring |
44
+
45
+ ## Highest ROI set
46
+
47
+ | Rank | Feature | Why |
48
+ |---:|---|---|
49
+ | 1 | Lock confirmed cluster identity | Easy and prevents frustrating reclustering drift |
50
+ | 2 | Outlier-first review queue | Huge UX gain from simple uncertainty ranking |
51
+ | 3 | Move sample to cluster as supervision | Core differentiator; directly improves results |
52
+ | 4 | Pull sample out / protect distinction | Required counterpart to positive supervision |
53
+ | 5 | Click-to-add missed onset | Direct correction beats indirect threshold tweaking |
54
+ | 6 | Bleed brush | Removes many false positives quickly |
55
+ | 7 | Explain cluster | Makes the system debuggable and trustworthy |
56
+ | 8 | Predictive batch questions | Multiplies the effect of each correction |
57
+ | 9 | Counterfactual previews | Makes advanced tuning understandable |
58
+ | 10 | Reconstruction-error correction | Very powerful, but architecturally heavier |
59
+
60
+ ## Technical conclusion
61
+
62
+ The most feasible "magic" is not heavy ML. It is constraint-aware clustering, cached feature vectors, uncertainty scoring, and local recomputation.
63
+
64
+ That foundation should be implemented before adding higher-risk semantic-space or personalized-model features.
65
+
66
+ ## Implementation alignment as of 2026-05-12
67
+
68
+ | Interaction | Current status | Notes |
69
+ |---|---|---|
70
+ | Move sample to cluster → auto-tune clustering | partial | Implemented as semantic state mutation with `force-cluster`/`must-link` constraints and heuristic move suggestions. True constrained local reclustering is still open. |
71
+ | Pull sample out → protect distinction | partial | Implemented as a new user cluster plus `cannot-link` and split suggestions. True constrained reclustering is still open. |
72
+ | Lock confirmed cluster identity | done | Lock/unlock persists in `supervision_state.json` and appears in the UI. Replay into future reruns is still open. |
73
+ | Outlier-first review queue | done | Implemented with heuristic confidence/priority. Feature-margin and reconstruction-impact ranking remain open. |
74
+ | Low-confidence visual emphasis | done | Low-confidence and suppressed hits are visually distinguished in the hit table and review queue. |
75
+ | Bleed brush/suppression | partial | Suppress selected hit and similar suppression suggestions are implemented. Region brush and restore are still open. |
76
+ | Click-to-add missed onset | todo | Waveform click currently auditions nearest existing hit only. |
77
+ | Cluster naming as reusable semantic hint | todo | User clusters receive generated labels; explicit rename/semantic hinting is not implemented. |
78
+ | Star/favorite sample → optimize around it | partial | Favorite pins representative in semantic state; artifact re-export does not yet honor it. |
79
+ | Explain this cluster | done | Explanation endpoint and UI drawer are implemented. |
80
+ | Predictive batch questions | partial | Suggestion inbox exists; exact diff previews and richer question phrasing are open. |
81
+ | Live counterfactual parameter previews | todo | Not implemented. |
82
+ | Reconstruction-error-driven correction | todo | Not implemented. |
83
+ | Multi-resolution semantic clustering | todo | Not implemented. |
84
+ | Auto-clean this family | todo | Not implemented. |
85
+ | Drag clusters in semantic space | backlog | Not implemented. |
86
+ | Cluster gravity / physics metaphor | backlog | Not implemented. |
87
+ | Context-aware classification | backlog | Not implemented. |
88
+ | Teach mode across songs | backlog | Not implemented. |
89
+
90
+ ## Revised highest-ROI next set
91
+
92
+ | Rank | Feature | Why now |
93
+ |---:|---|---|
94
+ | 1 | Supervised re-export | Makes current semantic edits affect the downloadable sample pack. |
95
+ | 2 | Force-onset from waveform | Adds the missing direct correction primitive for missed hits. |
96
+ | 3 | Suppression restore | Required safety counterpart to suppression. |
97
+ | 4 | Cached feature refs | Unlocks real local reclustering and better confidence. |
98
+ | 5 | Diff preview for suggestions | Makes batch suggestions safer and more trustworthy. |
99
+ | 6 | Constraint violation detection | Prevents silent conflicts once constraints become richer. |
100
+ | 7 | Browser tests | Protects the increasingly stateful UI from regressions. |
docs/interactive-ux/FEATURE_REQUIREMENTS.md ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Interactive UX feature requirements
2
+
3
+ ## Goal
4
+
5
+ Add interactions that make extraction faster and more accurate by converting user actions into reusable supervision signals.
6
+
7
+ The application should progressively converge toward the user's intended drum vocabulary with minimal manual cleanup.
8
+
9
+ ## Success criteria
10
+
11
+ - User corrections affect more than the single edited item when safe.
12
+ - Explicit user intent is preserved across reclustering and reloads.
13
+ - The system surfaces uncertain/high-leverage items before stable ones.
14
+ - The user can understand why items are grouped or separated.
15
+ - Cached stem/source analysis can be reused while downstream parameters update quickly.
16
+ - The UI supports fast audition, correction, locking, and export loops.
17
+
18
+ ## Current implementation summary
19
+
20
+ Implemented now:
21
+
22
+ - Persistent semantic state per run: `supervision_state.json`.
23
+ - Event log and constraint store.
24
+ - Confidence-weighted hit/cluster state.
25
+ - Outlier-first review queue.
26
+ - Move hit to cluster.
27
+ - Pull hit into new cluster.
28
+ - Lock/unlock cluster.
29
+ - Suppress hit as bleed/noise.
30
+ - Accept/favorite selected hit.
31
+ - Suggestion inbox with accept/reject.
32
+ - Cluster explanation endpoint and UI drawer.
33
+ - Undo for the last semantic edit.
34
+
35
+ Partially implemented:
36
+
37
+ - Local recomputation is currently semantic-state recomputation, not full feature-neighborhood reclustering.
38
+ - Suggestions are heuristic and preview-count based, not full diff previews.
39
+ - Favorite/pin changes semantic representative but does not yet regenerate the sample pack.
40
+ - Confidence scoring is heuristic, not feature-margin/stability based.
41
+
42
+ Not implemented yet:
43
+
44
+ - Click-to-add missed onset.
45
+ - Restore suppressed hit.
46
+ - Supervised re-export from edited state.
47
+ - Counterfactual parameter previews.
48
+ - Reconstruction-error correction.
49
+ - Teach mode across songs.
50
+
51
+ ## Functional requirements and status
52
+
53
+ ### FR-001: Cluster move as positive supervision
54
+
55
+ Status: **implemented as semantic-state edit**.
56
+
57
+ When a user moves a hit/sample into a cluster, the backend creates `force-cluster` and, when a representative exists, `must-link`. State confidence is recomputed and similar hit suggestions may be generated.
58
+
59
+ Remaining gap: true local feature-neighborhood reclustering and exact before/after diff preview.
60
+
61
+ ### FR-002: Pull-out as negative supervision
62
+
63
+ Status: **implemented as semantic-state edit**.
64
+
65
+ Pulling a hit out creates a new user cluster, records a `cannot-link` to the source representative when possible, and stores a `force-cluster` assignment for the new cluster.
66
+
67
+ Remaining gap: automatic split suggestions are heuristic and do not yet run constrained reclustering.
68
+
69
+ ### FR-003: Lock confirmed cluster identity
70
+
71
+ Status: **implemented**.
72
+
73
+ Clusters can be locked/unlocked through the API/UI. Lock state is persisted and influences confidence.
74
+
75
+ Remaining gap: future full reruns do not yet replay locks into batch clustering.
76
+
77
+ ### FR-004: Outlier-first review queue
78
+
79
+ Status: **implemented**.
80
+
81
+ The backend returns a `review_queue` sorted by low confidence, singleton status, review status, and suppression state. The UI renders the queue and lets the user select/audition items.
82
+
83
+ Remaining gap: expected-impact ranking should eventually use feature margin and reconstruction contribution.
84
+
85
+ ### FR-005: Confidence-weighted visual emphasis
86
+
87
+ Status: **implemented**.
88
+
89
+ Hit rows display confidence and flags. Low-confidence rows get emphasis; suppressed rows visually recede.
90
+
91
+ ### FR-006: Click-to-add missed onset
92
+
93
+ Status: **not implemented**.
94
+
95
+ Current waveform click auditions the nearest existing hit only.
96
+
97
+ Required next behavior:
98
+
99
+ - Add `force-onset` constraint at selected time.
100
+ - Slice candidate hit from cached `stem.wav`.
101
+ - Classify and assign locally.
102
+ - Store it as a forced hit in `supervision_state.json`.
103
+
104
+ ### FR-007: Bleed brush / false-positive suppression
105
+
106
+ Status: **implemented for selected hits; brush region not implemented**.
107
+
108
+ Selected hits can be suppressed as bleed/noise. The system stores `suppress-pattern` and proposes similar suppressions.
109
+
110
+ Remaining gap: region brush, restore, and supervised export exclusion.
111
+
112
+ ### FR-008: Favorite/pin sample optimization
113
+
114
+ Status: **partial**.
115
+
116
+ Favorite action records `pin-representative` and updates the semantic representative. Exported WAV/ZIP does not yet change.
117
+
118
+ ### FR-009: Explain cluster
119
+
120
+ Status: **implemented**.
121
+
122
+ Cluster explanation includes representative, hit counts, confidence reasons, label distribution, outliers, and relevant constraints.
123
+
124
+ ### FR-010: Predictive batch questions
125
+
126
+ Status: **partial**.
127
+
128
+ Suggestions exist for move/split/suppress patterns and can be accepted/rejected. They do not yet show exact diff previews.
129
+
130
+ ### FR-011: Live counterfactual parameter preview
131
+
132
+ Status: **not implemented**.
133
+
134
+ ### FR-012: Reconstruction-error correction
135
+
136
+ Status: **not implemented**.
137
+
138
+ ## Non-functional requirements and status
139
+
140
+ ### NFR-001: Reversibility
141
+
142
+ Status: **implemented for semantic edits** via undo stack.
143
+
144
+ ### NFR-002: Explainability
145
+
146
+ Status: **partial**. Events, constraints, confidence reasons, and cluster explanations are visible. Suggestion diff previews are not yet implemented.
147
+
148
+ ### NFR-003: Local recomputation first
149
+
150
+ Status: **partial**. Current recomputation is cheap semantic-state recomputation. True local feature reclustering remains open.
151
+
152
+ ### NFR-004: Cached preprocessing
153
+
154
+ Status: **partial**. Stems/source loads are cached and hit audio is exported. Feature-vector caching is still open.
155
+
156
+ ### NFR-005: Deterministic replay
157
+
158
+ Status: **partial**. Constraints/events persist and can be reloaded. A dedicated replay command/export pipeline is still open.
159
+
160
+ ### NFR-006: No silent override of explicit user intent
161
+
162
+ Status: **implemented in current semantic layer**. Explicit moves, locks, suppressions, and favorites persist unless undone or explicitly changed.
docs/interactive-ux/PROGRESS.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Interactive UX progress
2
+
3
+ ## Last updated
4
+
5
+ 2026-05-12
6
+
7
+ ## Current phase
8
+
9
+ Implementation is now in **Phase 1–3 foundation**: persistent state, constraints, events, confidence/review queue, and supervised cluster interactions are implemented at the semantic-state layer.
10
+
11
+ ## Completed in this pass
12
+
13
+ | Item | Status | Notes |
14
+ |---|---|---|
15
+ | Add supplied docs to `docs/interactive-ux/` | done | All supplied Markdown docs were copied into this directory and aligned with the implemented project. |
16
+ | Persistent job state | done | `supervision_state.json` is created beside `manifest.json` for each completed run. |
17
+ | Hit/cluster state schema | done | Implemented in `supervised_state.py` using JSON-serializable dictionaries. |
18
+ | Event log | done | State mutations append `job.state.created`, `constraint.created`, `hit.moved`, `hit.pulled_out`, `cluster.locked`, `hit.suppressed`, `suggestion.created`, etc. |
19
+ | Constraint store | done | Supports `force-cluster`, `must-link`, `cannot-link`, `lock-cluster`, `suppress-pattern`, and `pin-representative`. |
20
+ | Confidence scoring | partial | Heuristic scores based on cluster size, label agreement, energy rank, representative/favorite/explicit state, suppression, and lock state. Feature-vector margin scoring is not implemented yet. |
21
+ | Outlier-first review queue | done | Backend computes `review_queue`; UI renders it and lets the user jump to the selected hit. |
22
+ | Move hit to cluster | done | Endpoint creates constraints and updates state; it also proposes similar move suggestions. |
23
+ | Pull hit into new cluster | done | Endpoint creates `cannot-link` and `force-cluster` constraints and a user cluster. |
24
+ | Lock cluster | done | Endpoint toggles lock state and records a lock constraint. |
25
+ | Suppress hit as bleed/noise | done | Endpoint creates `suppress-pattern`, marks the hit suppressed, and proposes similar suppressions. |
26
+ | Favorite sample / pin representative | partial | Endpoint supports `review` status `favorite`, records `pin-representative`, and updates the representative hit in semantic state. Audio artifact selection is not re-exported yet. |
27
+ | Suggestion inbox | partial | Open suggestions render in the UI and can be accepted/rejected. Suggestion generation is heuristic and limited to move/split/suppress patterns. |
28
+ | Cluster explanation drawer | done | Endpoint and UI show representative, confidence reasons, outliers, relevant constraints, and label distribution. |
29
+ | Undo | done | Last semantic edit can be restored using an undo snapshot stack. |
30
+ | Validation script | done | Added `scripts/test_interactive_supervision.py`. |
31
+
32
+ ## Not yet implemented
33
+
34
+ - Real cached feature-vector store for local reclustering.
35
+ - Artifact re-export after semantic edits.
36
+ - Waveform click-to-add missed onset.
37
+ - Restore suppressed hit/batch restore.
38
+ - Real local neighborhood reclustering that changes assignments beyond explicit move/suggestion acceptance.
39
+ - Constraint violation detection and reporting.
40
+ - Predictive diff preview before accepting suggestions.
41
+ - Reconstruction-error-driven correction.
42
+ - Multi-resolution/hierarchical clusters.
43
+ - User correction profiles / teach mode across songs.
44
+ - Frontend TypeScript migration and browser automation tests.
45
+
46
+ ## Current risks
47
+
48
+ | Risk | Impact | Mitigation |
49
+ |---|---|---|
50
+ | Semantic edits do not rewrite exports yet | User may expect moved/suppressed hits to affect ZIP/MIDI immediately | Next task should be edited-state export. |
51
+ | Confidence scores are heuristic | Review queue may sometimes prioritize the wrong hits | Add cached mel/transient features and margin-to-next-cluster scoring. |
52
+ | Suggestions are simple | May over-suggest or under-suggest | Keep them previewable, explicit, and undoable; never silently apply. |
53
+ | Locks are semantic only | Batch reruns do not yet replay constraints | Add deterministic replay/local recluster using constraints. |
54
+ | No browser tests | UI regressions are easy | Add Playwright or lightweight DOM tests. |
55
+
56
+ ## Next implementation milestone
57
+
58
+ Milestone: **edited-state export and force-onset correction**.
59
+
60
+ Minimum deliverables:
61
+
62
+ 1. Add `POST /api/jobs/{job_id}/export/supervised` that creates a ZIP/MIDI/manifest from `supervision_state.json`.
63
+ 2. Exclude suppressed hits/clusters from the supervised export.
64
+ 3. Honor favorite/pinned representatives in exported samples.
65
+ 4. Add force-onset endpoint that slices a new hit from cached `stem.wav`.
66
+ 5. Add waveform shift-click or add-onset mode in the UI.
67
+ 6. Add tests proving semantic edits change the supervised export without rerunning stem extraction.
68
+
69
+ ## Definition of done for the current foundation
70
+
71
+ This loop now works:
72
+
73
+ ```text
74
+ analyze audio
75
+ → inspect clusters
76
+ → load semantic state
77
+ → move one wrong hit
78
+ → store constraints/events
79
+ → see confidence/review queue update
80
+ → lock corrected cluster
81
+ → suppress bleed
82
+ → inspect explanations and suggestions
83
+ → undo semantic edits
84
+ → reload the job and preserve explicit decisions
85
+ ```
86
+
87
+ The remaining missing piece is that edited semantic state is not yet reflected in a regenerated sample pack.
docs/interactive-ux/README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Interactive supervised extraction UX
2
+
3
+ This directory contains the supplied interactive-UX design documents, aligned with the implementation as of 2026-05-12.
4
+
5
+ The product direction is to turn drum sample extraction from a one-shot batch process into an interactive, supervised, progressively improving workflow. User edits should become constraints, preferences, and examples that improve clustering, review priority, labeling, and cleanup instead of only changing a visible table row.
6
+
7
+ ## Current implementation status
8
+
9
+ The project now has a first supervised-editing foundation layered on top of the immutable extraction manifest:
10
+
11
+ - `supervised_state.py` persists `supervision_state.json` beside each completed run manifest.
12
+ - The state contains hits, clusters, confidence scores, review queue entries, constraints, events, suggestions, and undo snapshots.
13
+ - The FastAPI backend exposes state, move, pull-out, lock, suppress, review/favorite, suggestion, explanation, and undo endpoints.
14
+ - The browser UI includes an interactive supervision panel with a review queue, cluster board, suggestion inbox, constraint/event log, and cluster explanation drawer.
15
+ - The current supervised layer updates semantic state only. It does not yet rewrite sample WAVs, MIDI, reconstruction audio, or the ZIP after edits.
16
+
17
+ ## Documents
18
+
19
+ | Document | Purpose | Alignment status |
20
+ |---|---|---|
21
+ | [`FEATURE_REQUIREMENTS.md`](./FEATURE_REQUIREMENTS.md) | Functional requirements for interaction-driven quality improvements | Updated with implemented/partial/not-started status |
22
+ | [`SCOPE.md`](./SCOPE.md) | MVP, v2, and research/backlog boundaries | Updated to reflect delivered MVP foundation |
23
+ | [`FEASIBILITY_MATRIX.md`](./FEASIBILITY_MATRIX.md) | Feasibility, complexity, effort, UX value, quality value table | Kept as design prioritization with implementation notes |
24
+ | [`TASKS.md`](./TASKS.md) | Implementation task breakdown with dependencies and acceptance criteria | Updated with current task statuses |
25
+ | [`PROGRESS.md`](./PROGRESS.md) | Current status, completed work, open work, next actions | Updated after this development pass |
26
+ | [`ARCHITECTURE_NOTES.md`](./ARCHITECTURE_NOTES.md) | Required state model and technical approach | Updated with actual module/API mapping |
27
+
28
+ ## Principle
29
+
30
+ Good interaction:
31
+
32
+ ```text
33
+ user moves one hit into another cluster
34
+ → system creates force-cluster/must-link constraints
35
+ → semantic state recomputes confidence and review priority
36
+ → similar ambiguous hits are proposed as suggestions
37
+ → the UI explains the state/event/constraint changes
38
+ ```
39
+
40
+ Bad interaction:
41
+
42
+ ```text
43
+ user moves one hit
44
+ → only that one visible DOM row changes
45
+ → future clustering ignores the correction
46
+ ```
47
+
48
+ ## Current build recommendation
49
+
50
+ Next implementation should close the gap between semantic edits and artifact output:
51
+
52
+ 1. Re-export edited sample pack from `supervision_state.json` without rerunning Demucs/onset detection.
53
+ 2. Add waveform click-to-add missed onset backed by `force-onset` constraints.
54
+ 3. Add restore for suppressed hits and batch accepted suggestions.
55
+ 4. Make move/pull-out trigger a real local reclustering pass using cached feature vectors.
56
+ 5. Add visible diff previews before accepting grouped suggestions.
57
+ 6. Add browser-level tests for the interactive supervision panel.
58
+
59
+ The strongest technical foundation remains constraint-aware clustering plus uncertainty-driven review. The current pass implements the persistent state and UI/API shell needed for that foundation.
docs/interactive-ux/SCOPE.md ADDED
@@ -0,0 +1,173 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Interactive UX scope
2
+
3
+ ## Scope model
4
+
5
+ Features are grouped by implementation risk and product leverage.
6
+
7
+ - **MVP**: should be built first; high value, technically feasible, foundational.
8
+ - **V2**: valuable after the core supervised editing loop exists.
9
+ - **Research/backlog**: plausible, but higher risk or dependent on stronger state/model infrastructure.
10
+
11
+ ## MVP scope and status
12
+
13
+ ### 1. Constraint-aware cluster editing
14
+
15
+ Status: **partial / foundation implemented**.
16
+
17
+ Implemented:
18
+
19
+ - Move hit/sample to cluster.
20
+ - Pull hit/sample out into a new cluster.
21
+ - Create `must-link`, `cannot-link`, and `force-cluster` constraints.
22
+ - Preserve explicit edits in `supervision_state.json` across reload.
23
+ - Generate basic suggestions after edits.
24
+
25
+ Remaining:
26
+
27
+ - True local neighborhood reclustering from cached feature vectors.
28
+ - Constraint violation detection.
29
+ - Edited artifact re-export.
30
+
31
+ ### 2. Lock confirmed clusters
32
+
33
+ Status: **implemented**.
34
+
35
+ Implemented:
36
+
37
+ - Lock/unlock cluster.
38
+ - Persist lock state.
39
+ - Show lock state in the cluster board and target cluster control.
40
+ - Confidence scoring accounts for locked state.
41
+
42
+ Remaining:
43
+
44
+ - Prevent future batch reruns from violating locked cluster identity through deterministic replay.
45
+
46
+ ### 3. Outlier-first review queue
47
+
48
+ Status: **implemented**.
49
+
50
+ Implemented:
51
+
52
+ - Confidence score per hit and cluster.
53
+ - Review list sorted by uncertainty priority.
54
+ - Quick actions: accept, favorite, move, pull out, suppress.
55
+
56
+ Remaining:
57
+
58
+ - Split/restore/batch actions.
59
+ - Expected-impact scoring using feature margin and reconstruction contribution.
60
+
61
+ ### 4. Confidence-weighted UI
62
+
63
+ Status: **implemented**.
64
+
65
+ Implemented:
66
+
67
+ - Confidence per hit and cluster.
68
+ - Low-confidence row emphasis.
69
+ - Suppressed rows recede.
70
+ - State summary badges.
71
+
72
+ ### 5. Click-to-add missed onset
73
+
74
+ Status: **not implemented**.
75
+
76
+ Current waveform clicks audition existing hits. Add-onset mode is the next direct correction primitive.
77
+
78
+ ### 6. Bleed suppression examples
79
+
80
+ Status: **partial**.
81
+
82
+ Implemented:
83
+
84
+ - Mark selected hit as bleed/noise.
85
+ - Store `suppress-pattern`.
86
+ - Generate similar suppression suggestions.
87
+
88
+ Remaining:
89
+
90
+ - Region brush.
91
+ - Restore suppressed hits.
92
+ - Exclude suppressed hits from supervised export.
93
+
94
+ ### 7. Explain this cluster
95
+
96
+ Status: **implemented**.
97
+
98
+ Implemented:
99
+
100
+ - Representative ID.
101
+ - Members/outliers.
102
+ - Confidence reasons.
103
+ - Label distribution.
104
+ - Relevant constraints.
105
+ - Locked/suppressed counts.
106
+
107
+ ## V2 scope
108
+
109
+ ### 1. Predictive batch questions
110
+
111
+ Status: **partial**.
112
+
113
+ Basic suggestions exist. Exact diff previews, grouped approval UX, and richer reasoning are not complete.
114
+
115
+ ### 2. Live counterfactual previews
116
+
117
+ Status: **not implemented**.
118
+
119
+ ### 3. Auto-clean this family
120
+
121
+ Status: **not implemented**.
122
+
123
+ ### 4. Multi-resolution semantic clustering
124
+
125
+ Status: **not implemented**.
126
+
127
+ ### 5. Run-to-run preference memory
128
+
129
+ Status: **not implemented**.
130
+
131
+ ## Research/backlog scope
132
+
133
+ Unchanged from design intent:
134
+
135
+ - Reconstruction-error-driven correction.
136
+ - Drag clusters in semantic space.
137
+ - Cluster gravity / physics metaphor.
138
+ - Context-aware classification.
139
+ - Teach mode across songs.
140
+
141
+ ## Explicitly out of scope for MVP
142
+
143
+ - Realtime Demucs.
144
+ - Training neural models from scratch.
145
+ - Cloud multi-user collaboration.
146
+ - DAW plugin integration.
147
+ - Full Ableton/Logic export format support.
148
+ - Automatic perfect drum transcription claims.
149
+
150
+ ## MVP acceptance criteria status
151
+
152
+ Target loop:
153
+
154
+ ```text
155
+ cached stem is analyzed
156
+ → clusters appear
157
+ → uncertain items are prioritized
158
+ → user moves one wrong hit
159
+ → system stores a constraint
160
+ → affected state recomputes
161
+ → related mistakes are suggested
162
+ → locked clusters remain stable in state
163
+ → semantic decisions reload reproducibly
164
+ ```
165
+
166
+ Status: **mostly achieved for semantic state**.
167
+
168
+ Not achieved yet:
169
+
170
+ ```text
171
+ → edited state can be exported reproducibly as updated WAV/MIDI/ZIP artifacts
172
+ → local feature reclustering updates related hits without manually accepting suggestions
173
+ ```
docs/interactive-ux/TASKS.md ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Interactive UX tasks
2
+
3
+ ## Status legend
4
+
5
+ | Status | Meaning |
6
+ |---|---|
7
+ | `todo` | Not started |
8
+ | `doing` | In progress |
9
+ | `partial` | Implemented as a useful first version but not complete |
10
+ | `done` | Completed for the current architecture |
11
+ | `blocked` | Waiting on prerequisite decision/work |
12
+
13
+ ## Phase 0: Documentation and design capture
14
+
15
+ | ID | Task | Status | Acceptance criteria |
16
+ |---|---|---|---|
17
+ | UX-000 | Capture feature requirements | done | Requirements documented in `FEATURE_REQUIREMENTS.md` |
18
+ | UX-001 | Capture scope boundaries | done | MVP/V2/backlog documented in `SCOPE.md` |
19
+ | UX-002 | Capture feasibility matrix | done | Scores documented in `FEASIBILITY_MATRIX.md` |
20
+ | UX-003 | Capture implementation tasks | done | Task backlog documented here |
21
+ | UX-004 | Capture progress | done | Progress documented in `PROGRESS.md` |
22
+ | UX-005 | Align supplied docs with implemented project | done | All documents updated to reflect current code/API/UI behavior |
23
+
24
+ ## Phase 1: State model and persistence
25
+
26
+ | ID | Task | Status | Acceptance criteria |
27
+ |---|---|---|---|
28
+ | UX-101 | Define persisted job state schema | done | `supervised_state.py` creates `supervision_state.json` with hits, clusters, constraints, events, suggestions, and undo state |
29
+ | UX-102 | Add event log | done | User/system changes append events and the event log is stored with job output |
30
+ | UX-103 | Add constraint store | done | `must-link`, `cannot-link`, `force-cluster`, `lock-cluster`, `suppress-pattern`, and `pin-representative` are saved |
31
+ | UX-104 | Add artifact/cache references | partial | State stores hit audio file refs and manifest fingerprint; feature refs and content-addressed feature cache are not implemented yet |
32
+ | UX-105 | Add deterministic replay command | todo | A job can be regenerated from source digest, params, and constraints |
33
+ | UX-106 | Add semantic undo stack | done | `POST /api/jobs/{job_id}/undo` restores the previous semantic state snapshot |
34
+
35
+ ## Phase 2: Confidence and review queue
36
+
37
+ | ID | Task | Status | Acceptance criteria |
38
+ |---|---|---|---|
39
+ | UX-201 | Compute hit assignment confidence | partial | Each hit has confidence from cluster confidence, label agreement, energy rank, representative/explicit/suppressed state; feature-vector margin is not implemented |
40
+ | UX-202 | Compute cluster confidence | partial | Each cluster has confidence from purity, size, representative, lock state; true feature cohesion/stability is not implemented |
41
+ | UX-203 | Add uncertainty ranking | done | Backend returns `review_queue` sorted by priority |
42
+ | UX-204 | Add low-confidence UI emphasis | done | Hit rows and review queue emphasize low-confidence and suppressed items |
43
+ | UX-205 | Add review quick actions | partial | Accept, favorite, move, pull-out, suppress are available; split/restore/batch actions are still incomplete |
44
+
45
+ ## Phase 3: Constraint-aware editing
46
+
47
+ | ID | Task | Status | Acceptance criteria |
48
+ |---|---|---|---|
49
+ | UX-301 | Move hit to cluster endpoint | done | Moving a hit creates `force-cluster` and usually `must-link` constraints |
50
+ | UX-302 | Pull hit into new cluster endpoint | done | Pulling a hit creates a new user cluster plus `cannot-link`/`force-cluster` constraints |
51
+ | UX-303 | Lock cluster endpoint | done | Locked cluster state persists and is shown in the UI |
52
+ | UX-304 | Local neighborhood recomputation | partial | Semantic state and confidence are recomputed; true cached feature-neighborhood reclustering is not implemented |
53
+ | UX-305 | Constraint violation detection | todo | Backend reports attempted changes that violate user constraints |
54
+ | UX-306 | Undo last interaction | done | User can reverse the previous semantic edit |
55
+ | UX-307 | Suggest similar moves from a hit move | partial | Heuristic suggestions are generated using label, centroid, and energy similarity |
56
+
57
+ ## Phase 4: Missed onset and bleed interactions
58
+
59
+ | ID | Task | Status | Acceptance criteria |
60
+ |---|---|---|---|
61
+ | UX-401 | Add force-onset endpoint | todo | User can add onset by time; hit is sliced/classified/clustered |
62
+ | UX-402 | Add waveform click interaction | partial | Existing waveform clicks audition nearest hit; add-onset mode is not implemented |
63
+ | UX-403 | Add bleed suppression endpoint | done | User can mark hit as bleed/noise and create a suppress-pattern example |
64
+ | UX-404 | Implement similar-false-positive search | partial | Heuristic suppress suggestions are generated from energy/centroid/label similarity |
65
+ | UX-405 | Add suppression restore | todo | Suppressed hits can be restored individually or in batches |
66
+
67
+ ## Phase 5: Suggestions and explanations
68
+
69
+ | ID | Task | Status | Acceptance criteria |
70
+ |---|---|---|---|
71
+ | UX-501 | Add suggestion model | done | Suggestions have type, confidence, reason, preview count, status, accept/reject state |
72
+ | UX-502 | Generate move suggestions from cluster edits | partial | Basic suggestions generated after move edits |
73
+ | UX-503 | Generate split suggestions from cannot-link edits | partial | Basic split suggestions generated after pull-out edits |
74
+ | UX-504 | Generate suppression suggestions | partial | Basic suppression suggestions generated after suppress edits |
75
+ | UX-505 | Add suggestion inbox UI | done | Suggestions are visible and can be accepted/rejected |
76
+ | UX-506 | Add explain-cluster endpoint | done | Backend returns representative, confidence reasons, outliers, constraints, label distribution |
77
+ | UX-507 | Add explanation drawer UI | done | UI renders explanation JSON for the selected cluster |
78
+ | UX-508 | Add diff previews before suggestion acceptance | todo | Suggestions show exact before/after cluster membership changes before acceptance |
79
+
80
+ ## Phase 6: Counterfactuals and advanced quality features
81
+
82
+ | ID | Task | Status | Acceptance criteria |
83
+ |---|---|---|---|
84
+ | UX-601 | Add parameter diff estimator | todo | UI can preview approximate effect of parameter changes |
85
+ | UX-602 | Add cached local parameter recompute | todo | Parameter changes reuse hit features where possible |
86
+ | UX-603 | Add reconstruction error map | todo | Backend computes original-vs-reconstruction mismatch by region |
87
+ | UX-604 | Add reconstruction correction workflow | todo | User can select bad region and get likely causes/fixes |
88
+ | UX-605 | Add multi-resolution cluster hierarchy | todo | Clusters can be browsed coarse-to-fine |
89
+
90
+ ## Phase 7: Preference memory / teach mode
91
+
92
+ | ID | Task | Status | Acceptance criteria |
93
+ |---|---|---|---|
94
+ | UX-701 | Persist user correction profiles | todo | Reusable preference profile stores accepted correction patterns |
95
+ | UX-702 | Apply profile to new jobs | todo | New jobs can opt into prior preferences |
96
+ | UX-703 | Add profile safety controls | todo | User can inspect, disable, delete, and scope learned preferences |
97
+ | UX-704 | Evaluate profile impact | todo | Benchmarks compare profile-on vs profile-off results |
98
+
99
+ ## Current recommended next task
100
+
101
+ Start with `UX-401` plus supervised export.
102
+
103
+ Reason:
104
+
105
+ The project now has a replayable state/events/constraints foundation. The largest UX gap is that semantic edits do not yet regenerate edited artifacts. Force-onset is the next direct correction primitive after move/pull/lock/suppress.
scripts/test_interactive_supervision.py ADDED
@@ -0,0 +1,112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Smoke-test manifest-backed interactive supervision endpoints."""
3
+
4
+ from __future__ import annotations
5
+
6
+ import io
7
+ import json
8
+ import sys
9
+ import time
10
+ from pathlib import Path
11
+ from urllib.parse import quote
12
+
13
+ import soundfile as sf
14
+ from fastapi.testclient import TestClient
15
+
16
+ sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
17
+
18
+ from app import app # noqa: E402
19
+ from synth_generator import generate_test_song # noqa: E402
20
+
21
+
22
+ def wait_for_job(client: TestClient, job_id: str) -> dict:
23
+ for _ in range(80):
24
+ payload = client.get(f"/api/jobs/{job_id}").json()
25
+ if payload["status"] in {"complete", "error"}:
26
+ return payload
27
+ time.sleep(0.15)
28
+ raise TimeoutError(job_id)
29
+
30
+
31
+ def post_json(client: TestClient, path: str, body: dict | None = None) -> dict:
32
+ response = client.post(path, json=body or {})
33
+ response.raise_for_status()
34
+ return response.json()
35
+
36
+
37
+ def main() -> int:
38
+ song = generate_test_song(pattern_name="funk", bars=1, bpm=124, add_bass=False)
39
+ buf = io.BytesIO()
40
+ sf.write(buf, song.drums_only, song.sr, format="WAV")
41
+ buf.seek(0)
42
+
43
+ client = TestClient(app)
44
+ response = client.post(
45
+ "/api/jobs",
46
+ files={"file": ("interactive.wav", buf, "audio/wav")},
47
+ data={"params": json.dumps({"stem": "all", "clustering_mode": "online_preview", "target_min": 3, "target_max": 10})},
48
+ )
49
+ response.raise_for_status()
50
+ job_id = response.json()["id"]
51
+ job = wait_for_job(client, job_id)
52
+ assert job["status"] == "complete", job.get("error")
53
+
54
+ state = client.get(f"/api/jobs/{job_id}/state").json()
55
+ assert state["summary"]["hit_count"] > 0
56
+ assert state["summary"]["cluster_count"] > 0
57
+ assert state["review_queue"], "expected uncertainty review queue"
58
+
59
+ hit_id = state["hits"][0]["id"]
60
+ cluster_id = state["clusters"][0]["id"]
61
+ q_hit = quote(hit_id, safe="")
62
+ q_cluster = quote(cluster_id, safe="")
63
+
64
+ state = post_json(client, f"/api/jobs/{job_id}/clusters/{q_cluster}/lock", {"locked": True})
65
+ assert state["summary"]["locked_cluster_count"] >= 1
66
+
67
+ state = post_json(client, f"/api/jobs/{job_id}/hits/{q_hit}/review", {"status": "favorite"})
68
+ assert state["summary"]["constraint_count"] >= 1
69
+
70
+ explanation = client.get(f"/api/jobs/{job_id}/explain/cluster/{q_cluster}")
71
+ explanation.raise_for_status()
72
+ assert explanation.json()["cluster_id"] == cluster_id
73
+
74
+ state = post_json(client, f"/api/jobs/{job_id}/hits/{q_hit}/pull-out", {})
75
+ assert state["summary"]["cluster_count"] >= 1
76
+ assert state["summary"]["undo_available"] is True
77
+ assert any(c["type"] in {"cannot-link", "force-cluster"} for c in state["constraints"])
78
+
79
+ state = post_json(client, f"/api/jobs/{job_id}/undo", {})
80
+ assert state["summary"]["hit_count"] > 0
81
+
82
+ if len(state["clusters"]) > 1:
83
+ target = next(c for c in state["clusters"] if c["id"] != state["hits"][0]["cluster_id"])
84
+ state = post_json(
85
+ client,
86
+ f"/api/jobs/{job_id}/hits/{q_hit}/move",
87
+ {"target_cluster_id": target["id"]},
88
+ )
89
+ assert any(c["type"] == "force-cluster" for c in state["constraints"])
90
+
91
+ if len(state["hits"]) > 1:
92
+ suppress_hit = quote(state["hits"][1]["id"], safe="")
93
+ state = post_json(client, f"/api/jobs/{job_id}/hits/{suppress_hit}/suppress", {"reason": "bleed"})
94
+ assert state["summary"]["suppressed_hit_count"] >= 1
95
+
96
+ suggestions = client.get(f"/api/jobs/{job_id}/suggestions")
97
+ suggestions.raise_for_status()
98
+
99
+ print(json.dumps({
100
+ "status": "ok",
101
+ "job_id": job_id,
102
+ "hit_count": state["summary"]["hit_count"],
103
+ "cluster_count": state["summary"]["cluster_count"],
104
+ "constraints": state["summary"]["constraint_count"],
105
+ "events": state["summary"]["event_count"],
106
+ "suggestions": state["summary"]["open_suggestion_count"],
107
+ }, indent=2))
108
+ return 0
109
+
110
+
111
+ if __name__ == "__main__":
112
+ raise SystemExit(main())
supervised_state.py ADDED
@@ -0,0 +1,675 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Persistent supervised-editing state for interactive extraction jobs.
3
+
4
+ The extraction pipeline produces immutable audio artifacts and a batch manifest.
5
+ This module layers replayable semantic state on top of that manifest: hits,
6
+ clusters, constraints, events, suggestions, confidence, and undo snapshots.
7
+
8
+ The first implementation intentionally avoids rewriting audio artifacts. It makes
9
+ supervised edits cheap, explicit, inspectable, and reproducible, then leaves
10
+ artifact re-export as a later step.
11
+ """
12
+
13
+ from __future__ import annotations
14
+
15
+ import copy
16
+ import json
17
+ import math
18
+ import time
19
+ import uuid
20
+ from pathlib import Path
21
+ from typing import Any, Callable
22
+
23
+ STATE_VERSION = "interactive-state-v1"
24
+ STATE_FILENAME = "supervision_state.json"
25
+ MAX_UNDO = 30
26
+
27
+
28
+ def now() -> float:
29
+ return round(time.time(), 6)
30
+
31
+
32
+ def state_path(output_dir: str | Path) -> Path:
33
+ return Path(output_dir) / STATE_FILENAME
34
+
35
+
36
+ def manifest_path(output_dir: str | Path) -> Path:
37
+ return Path(output_dir) / "manifest.json"
38
+
39
+
40
+ def load_manifest(output_dir: str | Path) -> dict[str, Any]:
41
+ path = manifest_path(output_dir)
42
+ if not path.exists():
43
+ raise FileNotFoundError(f"manifest.json not found in {Path(output_dir)}")
44
+ return json.loads(path.read_text(encoding="utf-8"))
45
+
46
+
47
+ def _hit_id(hit: dict[str, Any]) -> str:
48
+ return f"hit:{int(hit.get('index', 0)):05d}"
49
+
50
+
51
+ def _cluster_id(raw: Any) -> str:
52
+ return f"cluster:{raw}"
53
+
54
+
55
+ def _base_label(label: str) -> str:
56
+ text = str(label or "other")
57
+ return text.rsplit("_", 1)[0] if "_" in text else text
58
+
59
+
60
+ def _new_id(prefix: str) -> str:
61
+ return f"{prefix}:{uuid.uuid4().hex[:10]}"
62
+
63
+
64
+ def _safe_float(value: Any, default: float = 0.0) -> float:
65
+ try:
66
+ out = float(value)
67
+ if math.isfinite(out):
68
+ return out
69
+ except Exception:
70
+ pass
71
+ return default
72
+
73
+
74
+ def _snapshot(state: dict[str, Any]) -> dict[str, Any]:
75
+ snap = copy.deepcopy(state)
76
+ snap["undo_stack"] = []
77
+ return snap
78
+
79
+
80
+ def _push_undo(state: dict[str, Any]) -> None:
81
+ stack = list(state.get("undo_stack") or [])
82
+ stack.append(_snapshot(state))
83
+ del stack[:-MAX_UNDO]
84
+ state["undo_stack"] = stack
85
+
86
+
87
+ def _event(state: dict[str, Any], event_type: str, payload: dict[str, Any] | None = None, source: str = "system") -> dict[str, Any]:
88
+ event = {
89
+ "id": _new_id("event"),
90
+ "type": event_type,
91
+ "source": source,
92
+ "created_at": now(),
93
+ "payload": payload or {},
94
+ }
95
+ state.setdefault("events", []).append(event)
96
+ return event
97
+
98
+
99
+ def _constraint(state: dict[str, Any], constraint_type: str, payload: dict[str, Any], source: str = "user") -> dict[str, Any]:
100
+ constraint = {
101
+ "id": _new_id("constraint"),
102
+ "type": constraint_type,
103
+ "source": source,
104
+ "created_at": now(),
105
+ **payload,
106
+ }
107
+ state.setdefault("constraints", []).append(constraint)
108
+ _event(state, "constraint.created", {"constraint_id": constraint["id"], "type": constraint_type}, source=source)
109
+ return constraint
110
+
111
+
112
+ def _write_state(output_dir: str | Path, state: dict[str, Any]) -> dict[str, Any]:
113
+ state["updated_at"] = now()
114
+ path = state_path(output_dir)
115
+ path.write_text(json.dumps(state, indent=2, sort_keys=True), encoding="utf-8")
116
+ return state
117
+
118
+
119
+ def _cluster_label_for_hit(hit: dict[str, Any]) -> str:
120
+ return str(hit.get("cluster_label") or f"{hit.get('label', 'other')}_{hit.get('cluster_id', '0')}")
121
+
122
+
123
+ def build_initial_state(job_id: str, manifest: dict[str, Any]) -> dict[str, Any]:
124
+ hits_by_id: dict[str, dict[str, Any]] = {}
125
+ clusters: dict[str, dict[str, Any]] = {}
126
+
127
+ raw_hits = list(manifest.get("hits") or [])
128
+ if not raw_hits:
129
+ # Older manifests may only contain samples. Keep state valid even then.
130
+ raw_hits = []
131
+
132
+ for hit in raw_hits:
133
+ hid = _hit_id(hit)
134
+ cid = _cluster_id(hit.get("cluster_id", "unclustered"))
135
+ cluster_label = _cluster_label_for_hit(hit)
136
+ hits_by_id[hid] = {
137
+ "id": hid,
138
+ "index": int(hit.get("index", len(hits_by_id))),
139
+ "label": str(hit.get("label") or "other"),
140
+ "cluster_id": cid,
141
+ "original_cluster_id": cid,
142
+ "cluster_label": cluster_label,
143
+ "onset_sec": _safe_float(hit.get("onset_sec")),
144
+ "duration_ms": _safe_float(hit.get("duration_ms")),
145
+ "rms_energy": _safe_float(hit.get("rms_energy")),
146
+ "spectral_centroid_hz": _safe_float(hit.get("spectral_centroid_hz")),
147
+ "file": hit.get("file"),
148
+ "is_representative": bool(hit.get("is_representative")),
149
+ "source": "detected",
150
+ "suppressed": False,
151
+ "favorite": False,
152
+ "review_status": "unreviewed",
153
+ "confidence": 0.0,
154
+ "confidence_reasons": [],
155
+ "explicit": False,
156
+ }
157
+ clusters.setdefault(
158
+ cid,
159
+ {
160
+ "id": cid,
161
+ "label": cluster_label,
162
+ "classification": _base_label(cluster_label),
163
+ "hit_ids": [],
164
+ "representative_hit_id": None,
165
+ "locked": False,
166
+ "user_named": False,
167
+ "confidence": 0.0,
168
+ "confidence_reasons": [],
169
+ "suppressed_count": 0,
170
+ "original_id": cid,
171
+ },
172
+ )["hit_ids"].append(hid)
173
+ if bool(hit.get("is_representative")):
174
+ clusters[cid]["representative_hit_id"] = hid
175
+
176
+ for cid, cluster in clusters.items():
177
+ if cluster["representative_hit_id"] is None and cluster["hit_ids"]:
178
+ cluster["representative_hit_id"] = cluster["hit_ids"][0]
179
+
180
+ state = {
181
+ "version": STATE_VERSION,
182
+ "job_id": job_id,
183
+ "created_at": now(),
184
+ "updated_at": now(),
185
+ "manifest_fingerprint": _manifest_fingerprint(manifest),
186
+ "hits": hits_by_id,
187
+ "clusters": clusters,
188
+ "constraints": [],
189
+ "events": [],
190
+ "suggestions": [],
191
+ "undo_stack": [],
192
+ "counters": {"user_clusters": 0},
193
+ }
194
+ recompute_scores(state)
195
+ _event(
196
+ state,
197
+ "job.state.created",
198
+ {
199
+ "hit_count": len(hits_by_id),
200
+ "cluster_count": len(clusters),
201
+ "manifest_fingerprint": state["manifest_fingerprint"],
202
+ },
203
+ )
204
+ return state
205
+
206
+
207
+ def _manifest_fingerprint(manifest: dict[str, Any]) -> str:
208
+ import hashlib
209
+
210
+ payload = {
211
+ "params": manifest.get("params"),
212
+ "hit_count": manifest.get("hit_count"),
213
+ "cluster_count": manifest.get("cluster_count"),
214
+ "files": manifest.get("files"),
215
+ "hits": [
216
+ {
217
+ "index": h.get("index"),
218
+ "cluster_id": h.get("cluster_id"),
219
+ "file": h.get("file"),
220
+ "onset_sec": h.get("onset_sec"),
221
+ }
222
+ for h in manifest.get("hits", [])
223
+ ],
224
+ }
225
+ return hashlib.sha256(json.dumps(payload, sort_keys=True).encode("utf-8")).hexdigest()
226
+
227
+
228
+ def load_or_create_state(job_id: str, output_dir: str | Path) -> dict[str, Any]:
229
+ path = state_path(output_dir)
230
+ if path.exists():
231
+ state = json.loads(path.read_text(encoding="utf-8"))
232
+ if state.get("version") != STATE_VERSION:
233
+ raise ValueError(f"Unsupported supervision state version: {state.get('version')}")
234
+ return state
235
+ manifest = load_manifest(output_dir)
236
+ state = build_initial_state(job_id, manifest)
237
+ return _write_state(output_dir, state)
238
+
239
+
240
+ def _active_hits(state: dict[str, Any], cluster: dict[str, Any]) -> list[dict[str, Any]]:
241
+ hits = state.get("hits", {})
242
+ return [hits[hid] for hid in cluster.get("hit_ids", []) if hid in hits and not hits[hid].get("suppressed")]
243
+
244
+
245
+ def recompute_scores(state: dict[str, Any]) -> None:
246
+ hits = state.get("hits", {})
247
+ clusters = state.get("clusters", {})
248
+ energies = sorted(_safe_float(hit.get("rms_energy")) for hit in hits.values())
249
+
250
+ def energy_rank(value: float) -> float:
251
+ if not energies:
252
+ return 0.5
253
+ less = sum(1 for item in energies if item <= value)
254
+ return less / max(1, len(energies))
255
+
256
+ for cluster in clusters.values():
257
+ members = [hits[hid] for hid in cluster.get("hit_ids", []) if hid in hits]
258
+ active = [hit for hit in members if not hit.get("suppressed")]
259
+ if not members:
260
+ confidence = 0.15
261
+ reasons = ["empty cluster"]
262
+ else:
263
+ labels: dict[str, int] = {}
264
+ for hit in active:
265
+ labels[hit.get("label", "other")] = labels.get(hit.get("label", "other"), 0) + 1
266
+ majority = max(labels.values()) if labels else 0
267
+ purity = majority / max(1, len(active))
268
+ size_score = min(1.0, math.log2(len(active) + 1) / 4.0)
269
+ representative_bonus = 0.12 if cluster.get("representative_hit_id") in cluster.get("hit_ids", []) else 0.0
270
+ lock_bonus = 0.12 if cluster.get("locked") else 0.0
271
+ confidence = (0.42 * purity) + (0.34 * size_score) + representative_bonus + lock_bonus
272
+ reasons = []
273
+ if len(active) <= 1:
274
+ reasons.append("singleton cluster")
275
+ if purity < 0.75:
276
+ reasons.append("mixed labels")
277
+ if cluster.get("locked"):
278
+ reasons.append("user locked")
279
+ if representative_bonus:
280
+ reasons.append("has representative")
281
+ cluster["confidence"] = round(max(0.0, min(1.0, confidence)), 4)
282
+ cluster["confidence_reasons"] = reasons or ["cohesive cluster"]
283
+ cluster["suppressed_count"] = sum(1 for hit in members if hit.get("suppressed"))
284
+
285
+ for hit in hits.values():
286
+ cluster = clusters.get(hit.get("cluster_id"), {})
287
+ active_count = len(_active_hits(state, cluster)) if cluster else 0
288
+ label_match = _base_label(str(cluster.get("label", ""))) == str(hit.get("label", ""))
289
+ energy = energy_rank(_safe_float(hit.get("rms_energy")))
290
+ duration_ms = _safe_float(hit.get("duration_ms"))
291
+ duration_score = 0.65 if duration_ms <= 0 else max(0.0, min(1.0, 1.0 - abs(duration_ms - 180.0) / 700.0))
292
+ cluster_conf = _safe_float(cluster.get("confidence"), 0.2)
293
+ confidence = (0.42 * cluster_conf) + (0.18 * min(1.0, active_count / 4.0)) + (0.18 if label_match else 0.0) + (0.12 * energy) + (0.10 * duration_score)
294
+ reasons = []
295
+ if active_count <= 1:
296
+ reasons.append("singleton")
297
+ if not label_match:
298
+ reasons.append("label differs from cluster")
299
+ if energy < 0.2:
300
+ reasons.append("low energy")
301
+ if hit.get("is_representative"):
302
+ confidence += 0.08
303
+ reasons.append("representative")
304
+ if hit.get("explicit"):
305
+ confidence += 0.10
306
+ reasons.append("explicit user assignment")
307
+ if hit.get("suppressed"):
308
+ confidence = min(confidence, 0.25)
309
+ reasons.append("suppressed")
310
+ hit["confidence"] = round(max(0.0, min(1.0, confidence)), 4)
311
+ hit["confidence_reasons"] = reasons or ["consistent assignment"]
312
+ hit["cluster_label"] = cluster.get("label", hit.get("cluster_label", "unclustered"))
313
+
314
+
315
+ def review_queue(state: dict[str, Any], limit: int = 30) -> list[dict[str, Any]]:
316
+ rows = []
317
+ clusters = state.get("clusters", {})
318
+ for hit in state.get("hits", {}).values():
319
+ cluster = clusters.get(hit.get("cluster_id"), {})
320
+ score = 1.0 - _safe_float(hit.get("confidence"), 0.0)
321
+ if len(cluster.get("hit_ids", [])) <= 1:
322
+ score += 0.15
323
+ if hit.get("suppressed"):
324
+ score -= 0.35
325
+ if hit.get("review_status") == "accepted":
326
+ score -= 0.25
327
+ rows.append(
328
+ {
329
+ "hit_id": hit["id"],
330
+ "hit_index": hit.get("index"),
331
+ "label": hit.get("label"),
332
+ "cluster_id": hit.get("cluster_id"),
333
+ "cluster_label": cluster.get("label"),
334
+ "confidence": hit.get("confidence", 0.0),
335
+ "priority": round(max(0.0, score), 4),
336
+ "reasons": hit.get("confidence_reasons", []),
337
+ "suppressed": bool(hit.get("suppressed")),
338
+ "file": hit.get("file"),
339
+ }
340
+ )
341
+ rows.sort(key=lambda item: (-item["priority"], item["hit_index"] or 0))
342
+ return rows[: max(1, min(int(limit), 200))]
343
+
344
+
345
+ def _find_similar_hits(state: dict[str, Any], hit_id: str, *, exclude_cluster: str | None = None, include_suppressed: bool = False, limit: int = 12) -> list[tuple[dict[str, Any], float]]:
346
+ hits = state.get("hits", {})
347
+ src = hits[hit_id]
348
+ src_centroid = _safe_float(src.get("spectral_centroid_hz"))
349
+ src_energy = _safe_float(src.get("rms_energy"))
350
+ scored: list[tuple[dict[str, Any], float]] = []
351
+ for candidate in hits.values():
352
+ if candidate["id"] == hit_id:
353
+ continue
354
+ if exclude_cluster and candidate.get("cluster_id") == exclude_cluster:
355
+ continue
356
+ if candidate.get("suppressed") and not include_suppressed:
357
+ continue
358
+ label_score = 1.0 if candidate.get("label") == src.get("label") else 0.35
359
+ centroid_delta = abs(_safe_float(candidate.get("spectral_centroid_hz")) - src_centroid)
360
+ centroid_score = max(0.0, 1.0 - centroid_delta / 6000.0)
361
+ energy_delta = abs(_safe_float(candidate.get("rms_energy")) - src_energy)
362
+ energy_score = max(0.0, 1.0 - energy_delta / max(src_energy, 1e-4, _safe_float(candidate.get("rms_energy"))))
363
+ score = (0.48 * label_score) + (0.34 * centroid_score) + (0.18 * energy_score)
364
+ if score >= 0.62:
365
+ scored.append((candidate, round(score, 4)))
366
+ scored.sort(key=lambda item: (-item[1], item[0].get("index", 0)))
367
+ return scored[:limit]
368
+
369
+
370
+ def _add_suggestion(state: dict[str, Any], suggestion_type: str, payload: dict[str, Any], confidence: float, reason: str) -> dict[str, Any]:
371
+ suggestion = {
372
+ "id": _new_id("suggestion"),
373
+ "type": suggestion_type,
374
+ "status": "open",
375
+ "created_at": now(),
376
+ "confidence": round(max(0.0, min(1.0, confidence)), 4),
377
+ "reason": reason,
378
+ **payload,
379
+ }
380
+ state.setdefault("suggestions", []).append(suggestion)
381
+ _event(state, "suggestion.created", {"suggestion_id": suggestion["id"], "type": suggestion_type, "reason": reason})
382
+ return suggestion
383
+
384
+
385
+ def _rebuild_cluster_labels(state: dict[str, Any]) -> None:
386
+ hits = state.get("hits", {})
387
+ for cluster in state.get("clusters", {}).values():
388
+ for hid in cluster.get("hit_ids", []):
389
+ if hid in hits:
390
+ hits[hid]["cluster_label"] = cluster.get("label", "unclustered")
391
+
392
+
393
+ def move_hit(output_dir: str | Path, job_id: str, hit_id: str, target_cluster_id: str, source: str = "user") -> dict[str, Any]:
394
+ state = load_or_create_state(job_id, output_dir)
395
+ hits = state.get("hits", {})
396
+ clusters = state.get("clusters", {})
397
+ if hit_id not in hits:
398
+ raise KeyError(f"Unknown hit: {hit_id}")
399
+ if target_cluster_id not in clusters:
400
+ raise KeyError(f"Unknown cluster: {target_cluster_id}")
401
+ hit = hits[hit_id]
402
+ source_cluster_id = hit.get("cluster_id")
403
+ if source_cluster_id == target_cluster_id:
404
+ hit["review_status"] = "accepted"
405
+ recompute_scores(state)
406
+ return _write_state(output_dir, state)
407
+
408
+ _push_undo(state)
409
+ if source_cluster_id in clusters:
410
+ clusters[source_cluster_id]["hit_ids"] = [hid for hid in clusters[source_cluster_id].get("hit_ids", []) if hid != hit_id]
411
+ clusters[target_cluster_id].setdefault("hit_ids", [])
412
+ if hit_id not in clusters[target_cluster_id]["hit_ids"]:
413
+ clusters[target_cluster_id]["hit_ids"].append(hit_id)
414
+ hit["cluster_id"] = target_cluster_id
415
+ hit["cluster_label"] = clusters[target_cluster_id].get("label", target_cluster_id)
416
+ hit["explicit"] = True
417
+ hit["review_status"] = "accepted"
418
+ target_rep = clusters[target_cluster_id].get("representative_hit_id")
419
+ _constraint(state, "force-cluster", {"hit_id": hit_id, "cluster_id": target_cluster_id}, source=source)
420
+ if target_rep and target_rep != hit_id:
421
+ _constraint(state, "must-link", {"a": hit_id, "b": target_rep}, source=source)
422
+ _event(state, "hit.moved", {"hit_id": hit_id, "from_cluster_id": source_cluster_id, "to_cluster_id": target_cluster_id}, source=source)
423
+
424
+ similar = _find_similar_hits(state, hit_id, exclude_cluster=target_cluster_id, limit=10)
425
+ suggested_ids = [item[0]["id"] for item in similar if item[1] >= 0.72]
426
+ if suggested_ids:
427
+ avg = sum(score for _, score in similar if _["id"] in suggested_ids) / len(suggested_ids)
428
+ _add_suggestion(
429
+ state,
430
+ "move-hits",
431
+ {"hit_ids": suggested_ids, "target_cluster_id": target_cluster_id, "preview_count": len(suggested_ids)},
432
+ avg,
433
+ f"Similar label/spectral/energy profile to {hit_id}",
434
+ )
435
+ _rebuild_cluster_labels(state)
436
+ recompute_scores(state)
437
+ return _write_state(output_dir, state)
438
+
439
+
440
+ def pull_hit_to_new_cluster(output_dir: str | Path, job_id: str, hit_id: str, label: str | None = None, source: str = "user") -> dict[str, Any]:
441
+ state = load_or_create_state(job_id, output_dir)
442
+ hits = state.get("hits", {})
443
+ clusters = state.get("clusters", {})
444
+ if hit_id not in hits:
445
+ raise KeyError(f"Unknown hit: {hit_id}")
446
+ hit = hits[hit_id]
447
+ source_cluster_id = hit.get("cluster_id")
448
+ source_rep = clusters.get(source_cluster_id, {}).get("representative_hit_id")
449
+ _push_undo(state)
450
+ state.setdefault("counters", {})["user_clusters"] = int(state.get("counters", {}).get("user_clusters", 0)) + 1
451
+ base = label or f"{hit.get('label', 'hit')}_user_{state['counters']['user_clusters']}"
452
+ new_cluster_id = _new_id("cluster:user")
453
+ if source_cluster_id in clusters:
454
+ clusters[source_cluster_id]["hit_ids"] = [hid for hid in clusters[source_cluster_id].get("hit_ids", []) if hid != hit_id]
455
+ clusters[new_cluster_id] = {
456
+ "id": new_cluster_id,
457
+ "label": base,
458
+ "classification": _base_label(base),
459
+ "hit_ids": [hit_id],
460
+ "representative_hit_id": hit_id,
461
+ "locked": False,
462
+ "user_named": bool(label),
463
+ "confidence": 0.0,
464
+ "confidence_reasons": [],
465
+ "suppressed_count": 0,
466
+ "original_id": None,
467
+ }
468
+ hit["cluster_id"] = new_cluster_id
469
+ hit["cluster_label"] = base
470
+ hit["explicit"] = True
471
+ hit["review_status"] = "accepted"
472
+ if source_rep and source_rep != hit_id:
473
+ _constraint(state, "cannot-link", {"a": hit_id, "b": source_rep}, source=source)
474
+ _constraint(state, "force-cluster", {"hit_id": hit_id, "cluster_id": new_cluster_id}, source=source)
475
+ _event(state, "hit.pulled_out", {"hit_id": hit_id, "from_cluster_id": source_cluster_id, "to_cluster_id": new_cluster_id}, source=source)
476
+
477
+ similar = _find_similar_hits(state, hit_id, exclude_cluster=new_cluster_id, limit=8)
478
+ split_ids = [item[0]["id"] for item in similar if item[0].get("cluster_id") == source_cluster_id and item[1] >= 0.70]
479
+ if split_ids:
480
+ _add_suggestion(
481
+ state,
482
+ "split-hits",
483
+ {"hit_ids": split_ids, "source_cluster_id": source_cluster_id, "target_cluster_id": new_cluster_id, "preview_count": len(split_ids)},
484
+ 0.76,
485
+ f"Similar to pulled-out hit {hit_id}; preview split from original cluster",
486
+ )
487
+ _rebuild_cluster_labels(state)
488
+ recompute_scores(state)
489
+ return _write_state(output_dir, state)
490
+
491
+
492
+ def lock_cluster(output_dir: str | Path, job_id: str, cluster_id: str, locked: bool = True, source: str = "user") -> dict[str, Any]:
493
+ state = load_or_create_state(job_id, output_dir)
494
+ clusters = state.get("clusters", {})
495
+ if cluster_id not in clusters:
496
+ raise KeyError(f"Unknown cluster: {cluster_id}")
497
+ _push_undo(state)
498
+ clusters[cluster_id]["locked"] = bool(locked)
499
+ _constraint(state, "lock-cluster", {"cluster_id": cluster_id, "locked": bool(locked)}, source=source)
500
+ _event(state, "cluster.locked" if locked else "cluster.unlocked", {"cluster_id": cluster_id}, source=source)
501
+ recompute_scores(state)
502
+ return _write_state(output_dir, state)
503
+
504
+
505
+ def suppress_hit(output_dir: str | Path, job_id: str, hit_id: str, reason: str = "bleed", source: str = "user") -> dict[str, Any]:
506
+ state = load_or_create_state(job_id, output_dir)
507
+ hits = state.get("hits", {})
508
+ if hit_id not in hits:
509
+ raise KeyError(f"Unknown hit: {hit_id}")
510
+ _push_undo(state)
511
+ hit = hits[hit_id]
512
+ hit["suppressed"] = True
513
+ hit["review_status"] = "suppressed"
514
+ hit["explicit"] = True
515
+ _constraint(state, "suppress-pattern", {"example_hit_id": hit_id, "reason": reason}, source=source)
516
+ _event(state, "hit.suppressed", {"hit_id": hit_id, "reason": reason}, source=source)
517
+ similar = _find_similar_hits(state, hit_id, include_suppressed=False, limit=16)
518
+ suggested_ids = [item[0]["id"] for item in similar if item[1] >= 0.72 and _safe_float(item[0].get("rms_energy")) <= _safe_float(hit.get("rms_energy")) * 1.35]
519
+ if suggested_ids:
520
+ _add_suggestion(
521
+ state,
522
+ "suppress-hits",
523
+ {"hit_ids": suggested_ids, "reason_code": reason, "preview_count": len(suggested_ids)},
524
+ 0.74,
525
+ f"Similar low-energy profile to suppressed {reason} example {hit_id}",
526
+ )
527
+ recompute_scores(state)
528
+ return _write_state(output_dir, state)
529
+
530
+
531
+ def set_hit_review_status(output_dir: str | Path, job_id: str, hit_id: str, status: str = "accepted", source: str = "user") -> dict[str, Any]:
532
+ if status not in {"unreviewed", "accepted", "favorite"}:
533
+ raise ValueError("status must be unreviewed, accepted, or favorite")
534
+ state = load_or_create_state(job_id, output_dir)
535
+ if hit_id not in state.get("hits", {}):
536
+ raise KeyError(f"Unknown hit: {hit_id}")
537
+ _push_undo(state)
538
+ hit = state["hits"][hit_id]
539
+ hit["review_status"] = status
540
+ if status == "favorite":
541
+ hit["favorite"] = True
542
+ cid = hit.get("cluster_id")
543
+ if cid in state.get("clusters", {}):
544
+ state["clusters"][cid]["representative_hit_id"] = hit_id
545
+ _constraint(state, "pin-representative", {"hit_id": hit_id, "cluster_id": cid}, source=source)
546
+ _event(state, "hit.reviewed", {"hit_id": hit_id, "status": status}, source=source)
547
+ recompute_scores(state)
548
+ return _write_state(output_dir, state)
549
+
550
+
551
+ def accept_suggestion(output_dir: str | Path, job_id: str, suggestion_id: str) -> dict[str, Any]:
552
+ state = load_or_create_state(job_id, output_dir)
553
+ suggestion = next((s for s in state.get("suggestions", []) if s.get("id") == suggestion_id), None)
554
+ if not suggestion:
555
+ raise KeyError(f"Unknown suggestion: {suggestion_id}")
556
+ if suggestion.get("status") != "open":
557
+ return state
558
+ _push_undo(state)
559
+ stype = suggestion.get("type")
560
+ if stype in {"move-hits", "split-hits"}:
561
+ target = suggestion.get("target_cluster_id")
562
+ for hid in suggestion.get("hit_ids", []):
563
+ if hid in state.get("hits", {}) and target in state.get("clusters", {}):
564
+ current = state["hits"][hid].get("cluster_id")
565
+ if current in state["clusters"]:
566
+ state["clusters"][current]["hit_ids"] = [x for x in state["clusters"][current].get("hit_ids", []) if x != hid]
567
+ state["clusters"][target].setdefault("hit_ids", [])
568
+ if hid not in state["clusters"][target]["hit_ids"]:
569
+ state["clusters"][target]["hit_ids"].append(hid)
570
+ state["hits"][hid]["cluster_id"] = target
571
+ state["hits"][hid]["explicit"] = True
572
+ _constraint(state, "force-cluster", {"hit_id": hid, "cluster_id": target}, source="accepted-suggestion")
573
+ elif stype == "suppress-hits":
574
+ for hid in suggestion.get("hit_ids", []):
575
+ if hid in state.get("hits", {}):
576
+ state["hits"][hid]["suppressed"] = True
577
+ state["hits"][hid]["review_status"] = "suppressed"
578
+ _constraint(state, "suppress-pattern", {"example_hit_id": hid, "reason": suggestion.get("reason_code", "bleed")}, source="accepted-suggestion")
579
+ else:
580
+ raise ValueError(f"Unsupported suggestion type: {stype}")
581
+ suggestion["status"] = "accepted"
582
+ suggestion["resolved_at"] = now()
583
+ _event(state, "suggestion.accepted", {"suggestion_id": suggestion_id, "type": stype}, source="user")
584
+ _rebuild_cluster_labels(state)
585
+ recompute_scores(state)
586
+ return _write_state(output_dir, state)
587
+
588
+
589
+ def reject_suggestion(output_dir: str | Path, job_id: str, suggestion_id: str) -> dict[str, Any]:
590
+ state = load_or_create_state(job_id, output_dir)
591
+ suggestion = next((s for s in state.get("suggestions", []) if s.get("id") == suggestion_id), None)
592
+ if not suggestion:
593
+ raise KeyError(f"Unknown suggestion: {suggestion_id}")
594
+ _push_undo(state)
595
+ suggestion["status"] = "rejected"
596
+ suggestion["resolved_at"] = now()
597
+ _event(state, "suggestion.rejected", {"suggestion_id": suggestion_id, "type": suggestion.get("type")}, source="user")
598
+ return _write_state(output_dir, state)
599
+
600
+
601
+ def undo_last(output_dir: str | Path, job_id: str) -> dict[str, Any]:
602
+ state = load_or_create_state(job_id, output_dir)
603
+ stack = list(state.get("undo_stack") or [])
604
+ if not stack:
605
+ return state
606
+ restored = stack.pop()
607
+ restored["undo_stack"] = stack
608
+ _event(restored, "state.undo", {"restored_for_job_id": job_id}, source="user")
609
+ recompute_scores(restored)
610
+ return _write_state(output_dir, restored)
611
+
612
+
613
+ def explain_cluster(state: dict[str, Any], cluster_id: str) -> dict[str, Any]:
614
+ clusters = state.get("clusters", {})
615
+ hits = state.get("hits", {})
616
+ if cluster_id not in clusters:
617
+ raise KeyError(f"Unknown cluster: {cluster_id}")
618
+ cluster = clusters[cluster_id]
619
+ members = [hits[hid] for hid in cluster.get("hit_ids", []) if hid in hits]
620
+ active = [h for h in members if not h.get("suppressed")]
621
+ constraints = [c for c in state.get("constraints", []) if c.get("cluster_id") == cluster_id or c.get("hit_id") in cluster.get("hit_ids", []) or c.get("a") in cluster.get("hit_ids", []) or c.get("b") in cluster.get("hit_ids", [])]
622
+ outliers = sorted(active, key=lambda h: h.get("confidence", 0.0))[:8]
623
+ labels: dict[str, int] = {}
624
+ for hit in active:
625
+ labels[hit.get("label", "other")] = labels.get(hit.get("label", "other"), 0) + 1
626
+ return {
627
+ "cluster_id": cluster_id,
628
+ "label": cluster.get("label"),
629
+ "locked": bool(cluster.get("locked")),
630
+ "confidence": cluster.get("confidence"),
631
+ "confidence_reasons": cluster.get("confidence_reasons", []),
632
+ "representative_hit_id": cluster.get("representative_hit_id"),
633
+ "hit_count": len(members),
634
+ "active_hit_count": len(active),
635
+ "suppressed_count": sum(1 for hit in members if hit.get("suppressed")),
636
+ "label_distribution": labels,
637
+ "outliers": [{"hit_id": h["id"], "hit_index": h.get("index"), "confidence": h.get("confidence"), "reasons": h.get("confidence_reasons", [])} for h in outliers],
638
+ "constraints": constraints[-20:],
639
+ "summary": f"{cluster.get('label')} has {len(active)} active hits, confidence {cluster.get('confidence')}, and {len(constraints)} relevant constraints.",
640
+ }
641
+
642
+
643
+ def public_state(state: dict[str, Any], url_for: Callable[[str], str] | None = None, review_limit: int = 30) -> dict[str, Any]:
644
+ recompute_scores(state)
645
+ hits = copy.deepcopy(list(state.get("hits", {}).values()))
646
+ clusters = copy.deepcopy(list(state.get("clusters", {}).values()))
647
+ for hit in hits:
648
+ if url_for and hit.get("file"):
649
+ hit["url"] = url_for(hit["file"])
650
+ clusters.sort(key=lambda c: (-len(c.get("hit_ids", [])), c.get("label", "")))
651
+ hits.sort(key=lambda h: h.get("index", 0))
652
+ open_suggestions = [s for s in state.get("suggestions", []) if s.get("status") == "open"]
653
+ open_suggestions.sort(key=lambda s: (-_safe_float(s.get("confidence")), s.get("created_at", 0)))
654
+ return {
655
+ "version": state.get("version"),
656
+ "job_id": state.get("job_id"),
657
+ "created_at": state.get("created_at"),
658
+ "updated_at": state.get("updated_at"),
659
+ "summary": {
660
+ "hit_count": len(hits),
661
+ "cluster_count": len(clusters),
662
+ "constraint_count": len(state.get("constraints", [])),
663
+ "event_count": len(state.get("events", [])),
664
+ "open_suggestion_count": len(open_suggestions),
665
+ "suppressed_hit_count": sum(1 for h in hits if h.get("suppressed")),
666
+ "locked_cluster_count": sum(1 for c in clusters if c.get("locked")),
667
+ "undo_available": bool(state.get("undo_stack")),
668
+ },
669
+ "hits": hits,
670
+ "clusters": clusters,
671
+ "constraints": state.get("constraints", [])[-100:],
672
+ "events": state.get("events", [])[-120:],
673
+ "suggestions": open_suggestions[:50],
674
+ "review_queue": review_queue(state, review_limit),
675
+ }
web/app.js CHANGED
@@ -12,6 +12,8 @@ let selectedFile = null;
12
  let activePoll = null;
13
  let activeEvents = null;
14
  let lastResult = null;
 
 
15
  let selectedHitIndex = null;
16
 
17
  function esc(value) {
@@ -47,6 +49,60 @@ async function api(path, options = {}) {
47
  return response.json();
48
  }
49
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  function setSelectOptions(select, values, labels = null) {
51
  select.innerHTML = "";
52
  for (const value of values) {
@@ -159,15 +215,19 @@ function playAudio(el, url) {
159
 
160
  function selectHit(index, shouldPlay = true) {
161
  if (!lastResult) return;
162
- const hit = (lastResult.hits ?? []).find((item) => Number(item.index) === Number(index));
163
- if (!hit) return;
 
164
  selectedHitIndex = hit.index;
165
- $("selectedHitMeta").textContent = `#${hit.index} · ${hit.label} · ${hit.cluster_label} · ${hit.onset_sec}s · ${hit.duration_ms} ms${hit.is_representative ? " · representative" : ""}`;
 
 
166
  if (shouldPlay) playAudio($("hitAudio"), hit.url);
167
  for (const row of document.querySelectorAll("[data-hit-index]")) {
168
  row.classList.toggle("selected", Number(row.dataset.hitIndex) === Number(hit.index));
169
  }
170
  drawWaveform(lastResult.overview);
 
171
  }
172
 
173
  function auditionSample(sample) {
@@ -200,19 +260,26 @@ function renderSamples(result) {
200
 
201
  function renderHits(result) {
202
  const tbody = $("hitsTable").querySelector("tbody");
203
- const hits = result.hits ?? [];
204
- tbody.innerHTML = hits.map((hit) => `
205
- <tr data-hit-index="${esc(hit.index)}" class="${Number(hit.index) === Number(selectedHitIndex) ? "selected" : ""}">
206
- <td><button class="mini-button" type="button" data-hit-audition="${esc(hit.index)}">Audition</button></td>
207
- <td>${esc(hit.index)}</td>
208
- <td>${esc(hit.label)}${hit.is_representative ? " ★" : ""}</td>
209
- <td>${esc(hit.cluster_label)}</td>
210
- <td>${esc(hit.onset_sec)} s</td>
211
- <td>${esc(hit.duration_ms)} ms</td>
212
- <td>${esc(hit.rms_energy)}</td>
213
- <td><a href="${esc(hit.url)}" download>WAV</a></td>
214
- </tr>
215
- `).join("");
 
 
 
 
 
 
 
216
  for (const row of tbody.querySelectorAll("[data-hit-index]")) {
217
  row.addEventListener("click", () => selectHit(row.dataset.hitIndex));
218
  }
@@ -225,9 +292,143 @@ function renderHits(result) {
225
  if (hits.length && selectedHitIndex === null) selectHit(hits[0].index, false);
226
  }
227
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
228
  function renderResult(job) {
229
  const result = job.result;
230
  if (!result) return;
 
231
  lastResult = result;
232
  if (!(result.hits ?? []).some((hit) => Number(hit.index) === Number(selectedHitIndex))) {
233
  selectedHitIndex = (result.hits ?? [])[0]?.index ?? null;
@@ -245,6 +446,9 @@ function renderResult(job) {
245
  renderSamples(result);
246
  renderHits(result);
247
  drawWaveform(result.overview);
 
 
 
248
  }
249
 
250
  function renderJob(job) {
@@ -345,6 +549,8 @@ async function runExtraction() {
345
  if (!selectedFile) return;
346
  selectedHitIndex = null;
347
  lastResult = null;
 
 
348
  $("runButton").disabled = true;
349
  $("jobPill").textContent = "uploading";
350
  $("logs").textContent = "Uploading source and starting extraction…";
@@ -426,6 +632,16 @@ $("clearCacheButton").addEventListener("click", async () => {
426
  $("logs").textContent = error.message;
427
  }
428
  });
 
 
 
 
 
 
 
 
 
 
429
  $("waveform").addEventListener("click", selectNearestWaveformHit);
430
 
431
  const dropzone = $("dropzone");
 
12
  let activePoll = null;
13
  let activeEvents = null;
14
  let lastResult = null;
15
+ let lastSupervisionState = null;
16
+ let activeJobId = null;
17
  let selectedHitIndex = null;
18
 
19
  function esc(value) {
 
49
  return response.json();
50
  }
51
 
52
+ async function jsonApi(path, body = {}, method = "POST") {
53
+ return api(path, {
54
+ method,
55
+ headers: { "Content-Type": "application/json" },
56
+ body: JSON.stringify(body),
57
+ });
58
+ }
59
+
60
+ function hitIdFromIndex(index) {
61
+ if (index === null || index === undefined) return null;
62
+ return `hit:${String(Number(index)).padStart(5, "0")}`;
63
+ }
64
+
65
+ function stateHitByIndex(index) {
66
+ const id = hitIdFromIndex(index);
67
+ return (lastSupervisionState?.hits ?? []).find((hit) => hit.id === id) ?? null;
68
+ }
69
+
70
+ function decorateHit(hit) {
71
+ const stateHit = stateHitByIndex(hit.index);
72
+ return {
73
+ ...hit,
74
+ state_hit_id: stateHit?.id ?? hitIdFromIndex(hit.index),
75
+ cluster_ref: stateHit?.cluster_id ?? `cluster:${hit.cluster_id}`,
76
+ cluster_label: stateHit?.cluster_label ?? hit.cluster_label,
77
+ confidence: stateHit?.confidence,
78
+ confidence_reasons: stateHit?.confidence_reasons ?? [],
79
+ suppressed: Boolean(stateHit?.suppressed),
80
+ favorite: Boolean(stateHit?.favorite),
81
+ review_status: stateHit?.review_status ?? "unreviewed",
82
+ };
83
+ }
84
+
85
+ function currentTargetCluster() {
86
+ const id = $("targetClusterSelect")?.value;
87
+ return (lastSupervisionState?.clusters ?? []).find((cluster) => cluster.id === id) ?? null;
88
+ }
89
+
90
+ function setActionButtons() {
91
+ const hasState = Boolean(activeJobId && lastSupervisionState);
92
+ const hasHit = hasState && selectedHitIndex !== null;
93
+ for (const id of ["moveHitButton", "pullHitButton", "acceptHitButton", "favoriteHitButton", "suppressHitButton"]) {
94
+ const button = $(id);
95
+ if (button) button.disabled = !hasHit;
96
+ }
97
+ for (const id of ["refreshStateButton", "undoButton", "lockClusterButton", "explainClusterButton"]) {
98
+ const button = $(id);
99
+ if (button) button.disabled = !hasState;
100
+ }
101
+ const target = currentTargetCluster();
102
+ if ($("lockClusterButton")) $("lockClusterButton").textContent = target?.locked ? "Unlock target cluster" : "Lock target cluster";
103
+ if ($("undoButton") && lastSupervisionState) $("undoButton").disabled = !lastSupervisionState.summary?.undo_available;
104
+ }
105
+
106
  function setSelectOptions(select, values, labels = null) {
107
  select.innerHTML = "";
108
  for (const value of values) {
 
215
 
216
  function selectHit(index, shouldPlay = true) {
217
  if (!lastResult) return;
218
+ const rawHit = (lastResult.hits ?? []).find((item) => Number(item.index) === Number(index));
219
+ if (!rawHit) return;
220
+ const hit = decorateHit(rawHit);
221
  selectedHitIndex = hit.index;
222
+ const confidence = hit.confidence === undefined ? "—" : `${Math.round(Number(hit.confidence) * 100)}%`;
223
+ const flags = [hit.is_representative ? "representative" : null, hit.favorite ? "favorite" : null, hit.suppressed ? "suppressed" : null, hit.review_status !== "unreviewed" ? hit.review_status : null].filter(Boolean).join(" · ");
224
+ $("selectedHitMeta").textContent = `#${hit.index} · ${hit.label} · ${hit.cluster_label} · ${hit.onset_sec}s · ${hit.duration_ms} ms · confidence ${confidence}${flags ? ` · ${flags}` : ""}`;
225
  if (shouldPlay) playAudio($("hitAudio"), hit.url);
226
  for (const row of document.querySelectorAll("[data-hit-index]")) {
227
  row.classList.toggle("selected", Number(row.dataset.hitIndex) === Number(hit.index));
228
  }
229
  drawWaveform(lastResult.overview);
230
+ setActionButtons();
231
  }
232
 
233
  function auditionSample(sample) {
 
260
 
261
  function renderHits(result) {
262
  const tbody = $("hitsTable").querySelector("tbody");
263
+ const hits = (result.hits ?? []).map(decorateHit);
264
+ tbody.innerHTML = hits.map((hit) => {
265
+ const confidence = hit.confidence === undefined ? "" : `${Math.round(Number(hit.confidence) * 100)}%`;
266
+ const flags = [hit.is_representative ? "rep" : null, hit.favorite ? "fav" : null, hit.suppressed ? "suppressed" : null, hit.review_status !== "unreviewed" ? hit.review_status : null].filter(Boolean);
267
+ const classes = [Number(hit.index) === Number(selectedHitIndex) ? "selected" : "", hit.suppressed ? "suppressed" : "", Number(hit.confidence ?? 1) < 0.55 ? "low-confidence" : ""].filter(Boolean).join(" ");
268
+ return `
269
+ <tr data-hit-index="${esc(hit.index)}" class="${esc(classes)}">
270
+ <td><button class="mini-button" type="button" data-hit-audition="${esc(hit.index)}">Audition</button></td>
271
+ <td>${esc(hit.index)}</td>
272
+ <td>${esc(hit.label)}${hit.is_representative || hit.favorite ? " ★" : ""}</td>
273
+ <td>${esc(hit.cluster_label)}</td>
274
+ <td>${esc(confidence)}</td>
275
+ <td>${esc(flags.join(", ") || "—")}</td>
276
+ <td>${esc(hit.onset_sec)} s</td>
277
+ <td>${esc(hit.duration_ms)} ms</td>
278
+ <td>${esc(hit.rms_energy)}</td>
279
+ <td><a href="${esc(hit.url)}" download>WAV</a></td>
280
+ </tr>
281
+ `;
282
+ }).join("");
283
  for (const row of tbody.querySelectorAll("[data-hit-index]")) {
284
  row.addEventListener("click", () => selectHit(row.dataset.hitIndex));
285
  }
 
292
  if (hits.length && selectedHitIndex === null) selectHit(hits[0].index, false);
293
  }
294
 
295
+ function renderSupervisionState(state) {
296
+ lastSupervisionState = state;
297
+ const summary = state.summary ?? {};
298
+ $("supervisionSummary").innerHTML = `
299
+ <span>${esc(summary.hit_count ?? 0)} hits</span>
300
+ <span>${esc(summary.cluster_count ?? 0)} clusters</span>
301
+ <span>${esc(summary.constraint_count ?? 0)} constraints</span>
302
+ <span>${esc(summary.open_suggestion_count ?? 0)} suggestions</span>
303
+ <span>${esc(summary.suppressed_hit_count ?? 0)} suppressed</span>
304
+ <span>${esc(summary.locked_cluster_count ?? 0)} locked</span>
305
+ `;
306
+
307
+ const currentTarget = $("targetClusterSelect").value;
308
+ $("targetClusterSelect").innerHTML = (state.clusters ?? []).map((cluster) => `
309
+ <option value="${esc(cluster.id)}">${cluster.locked ? "locked · " : ""}${esc(cluster.label)} · ${esc(cluster.hit_ids?.length ?? 0)} hits · ${Math.round(Number(cluster.confidence ?? 0) * 100)}%</option>
310
+ `).join("");
311
+ if ((state.clusters ?? []).some((cluster) => cluster.id === currentTarget)) $("targetClusterSelect").value = currentTarget;
312
+
313
+ $("reviewQueue").innerHTML = (state.review_queue ?? []).slice(0, 14).map((item) => `
314
+ <button class="compact-row ${item.suppressed ? "suppressed" : ""}" type="button" data-review-hit="${esc(item.hit_index)}">
315
+ <span><strong>#${esc(item.hit_index)} · ${esc(item.label)}</strong><small>${esc(item.cluster_label)} · ${Math.round(Number(item.confidence ?? 0) * 100)}% · ${esc((item.reasons ?? []).join(", "))}</small></span>
316
+ <span>${Math.round(Number(item.priority ?? 0) * 100)}</span>
317
+ </button>
318
+ `).join("") || `<p class="empty">No review items.</p>`;
319
+ for (const button of $("reviewQueue").querySelectorAll("[data-review-hit]")) {
320
+ button.addEventListener("click", () => selectHit(button.dataset.reviewHit));
321
+ }
322
+
323
+ $("clusterBoard").innerHTML = (state.clusters ?? []).map((cluster) => `
324
+ <button class="compact-row ${cluster.locked ? "locked" : ""}" type="button" data-cluster-select="${esc(cluster.id)}">
325
+ <span><strong>${cluster.locked ? "Locked · " : ""}${esc(cluster.label)}</strong><small>${esc(cluster.hit_ids?.length ?? 0)} hits · ${Math.round(Number(cluster.confidence ?? 0) * 100)}% · ${esc((cluster.confidence_reasons ?? []).join(", "))}</small></span>
326
+ <span>${esc(cluster.suppressed_count ?? 0)} suppr.</span>
327
+ </button>
328
+ `).join("") || `<p class="empty">No clusters.</p>`;
329
+ for (const button of $("clusterBoard").querySelectorAll("[data-cluster-select]")) {
330
+ button.addEventListener("click", () => {
331
+ $("targetClusterSelect").value = button.dataset.clusterSelect;
332
+ setActionButtons();
333
+ explainTargetCluster().catch(() => {});
334
+ });
335
+ }
336
+
337
+ $("suggestionInbox").innerHTML = (state.suggestions ?? []).map((suggestion) => `
338
+ <div class="suggestion-row">
339
+ <div><strong>${esc(suggestion.type)}</strong><small>${esc(suggestion.reason)} · ${Math.round(Number(suggestion.confidence ?? 0) * 100)}% · ${esc(suggestion.preview_count ?? suggestion.hit_ids?.length ?? 0)} hits</small></div>
340
+ <div class="row-actions">
341
+ <button class="mini-button" type="button" data-accept-suggestion="${esc(suggestion.id)}">Accept</button>
342
+ <button class="mini-button" type="button" data-reject-suggestion="${esc(suggestion.id)}">Reject</button>
343
+ </div>
344
+ </div>
345
+ `).join("") || `<p class="empty">No open suggestions.</p>`;
346
+ for (const button of $("suggestionInbox").querySelectorAll("[data-accept-suggestion]")) {
347
+ button.addEventListener("click", () => acceptSuggestion(button.dataset.acceptSuggestion));
348
+ }
349
+ for (const button of $("suggestionInbox").querySelectorAll("[data-reject-suggestion]")) {
350
+ button.addEventListener("click", () => rejectSuggestion(button.dataset.rejectSuggestion));
351
+ }
352
+
353
+ const eventRows = (state.events ?? []).slice(-12).reverse().map((event) => `<div class="log-row"><strong>${esc(event.type)}</strong><small>${esc(event.source)} · ${fmtDate(event.created_at)}</small></div>`);
354
+ const constraintRows = (state.constraints ?? []).slice(-8).reverse().map((constraint) => `<div class="log-row constraint"><strong>${esc(constraint.type)}</strong><small>${esc(constraint.source)} · ${fmtDate(constraint.created_at)}</small></div>`);
355
+ $("stateLog").innerHTML = [...eventRows, ...constraintRows].join("") || `<p class="empty">No state events yet.</p>`;
356
+
357
+ setActionButtons();
358
+ if (lastResult) renderHits(lastResult);
359
+ }
360
+
361
+ async function fetchState() {
362
+ if (!activeJobId) return null;
363
+ const state = await api(`/api/jobs/${encodeURIComponent(activeJobId)}/state`);
364
+ renderSupervisionState(state);
365
+ return state;
366
+ }
367
+
368
+ async function applyStateAction(path, body = {}) {
369
+ if (!activeJobId) return;
370
+ const state = await jsonApi(path, body);
371
+ renderSupervisionState(state);
372
+ }
373
+
374
+ async function moveSelectedHit() {
375
+ const hitId = hitIdFromIndex(selectedHitIndex);
376
+ const target = $("targetClusterSelect").value;
377
+ if (!activeJobId || !hitId || !target) return;
378
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/move`, { target_cluster_id: target });
379
+ }
380
+
381
+ async function pullSelectedHit() {
382
+ const hitId = hitIdFromIndex(selectedHitIndex);
383
+ if (!activeJobId || !hitId) return;
384
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/pull-out`, {});
385
+ }
386
+
387
+ async function suppressSelectedHit() {
388
+ const hitId = hitIdFromIndex(selectedHitIndex);
389
+ if (!activeJobId || !hitId) return;
390
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/suppress`, { reason: "bleed" });
391
+ }
392
+
393
+ async function reviewSelectedHit(status) {
394
+ const hitId = hitIdFromIndex(selectedHitIndex);
395
+ if (!activeJobId || !hitId) return;
396
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/hits/${encodeURIComponent(hitId)}/review`, { status });
397
+ }
398
+
399
+ async function toggleTargetClusterLock() {
400
+ const target = currentTargetCluster();
401
+ if (!activeJobId || !target) return;
402
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/clusters/${encodeURIComponent(target.id)}/lock`, { locked: !target.locked });
403
+ }
404
+
405
+ async function explainTargetCluster() {
406
+ const target = currentTargetCluster();
407
+ if (!activeJobId || !target) return;
408
+ const explanation = await api(`/api/jobs/${encodeURIComponent(activeJobId)}/explain/cluster/${encodeURIComponent(target.id)}`);
409
+ $("clusterExplanation").classList.remove("empty");
410
+ $("clusterExplanation").textContent = JSON.stringify(explanation, null, 2);
411
+ }
412
+
413
+ async function acceptSuggestion(id) {
414
+ if (!activeJobId) return;
415
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/suggestions/${encodeURIComponent(id)}/accept`, {});
416
+ }
417
+
418
+ async function rejectSuggestion(id) {
419
+ if (!activeJobId) return;
420
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/suggestions/${encodeURIComponent(id)}/reject`, {});
421
+ }
422
+
423
+ async function undoLastEdit() {
424
+ if (!activeJobId) return;
425
+ await applyStateAction(`/api/jobs/${encodeURIComponent(activeJobId)}/undo`, {});
426
+ }
427
+
428
  function renderResult(job) {
429
  const result = job.result;
430
  if (!result) return;
431
+ activeJobId = job.id;
432
  lastResult = result;
433
  if (!(result.hits ?? []).some((hit) => Number(hit.index) === Number(selectedHitIndex))) {
434
  selectedHitIndex = (result.hits ?? [])[0]?.index ?? null;
 
446
  renderSamples(result);
447
  renderHits(result);
448
  drawWaveform(result.overview);
449
+ if (activeJobId) fetchState().catch((error) => {
450
+ $("supervisionSummary").textContent = error.message;
451
+ });
452
  }
453
 
454
  function renderJob(job) {
 
549
  if (!selectedFile) return;
550
  selectedHitIndex = null;
551
  lastResult = null;
552
+ lastSupervisionState = null;
553
+ activeJobId = null;
554
  $("runButton").disabled = true;
555
  $("jobPill").textContent = "uploading";
556
  $("logs").textContent = "Uploading source and starting extraction…";
 
632
  $("logs").textContent = error.message;
633
  }
634
  });
635
+ $("refreshStateButton").addEventListener("click", () => fetchState().catch((error) => { $("supervisionSummary").textContent = error.message; }));
636
+ $("undoButton").addEventListener("click", () => undoLastEdit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
637
+ $("moveHitButton").addEventListener("click", () => moveSelectedHit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
638
+ $("pullHitButton").addEventListener("click", () => pullSelectedHit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
639
+ $("acceptHitButton").addEventListener("click", () => reviewSelectedHit("accepted").catch((error) => { $("clusterExplanation").textContent = error.message; }));
640
+ $("favoriteHitButton").addEventListener("click", () => reviewSelectedHit("favorite").catch((error) => { $("clusterExplanation").textContent = error.message; }));
641
+ $("suppressHitButton").addEventListener("click", () => suppressSelectedHit().catch((error) => { $("clusterExplanation").textContent = error.message; }));
642
+ $("lockClusterButton").addEventListener("click", () => toggleTargetClusterLock().catch((error) => { $("clusterExplanation").textContent = error.message; }));
643
+ $("explainClusterButton").addEventListener("click", () => explainTargetCluster().catch((error) => { $("clusterExplanation").textContent = error.message; }));
644
+ $("targetClusterSelect").addEventListener("change", setActionButtons);
645
  $("waveform").addEventListener("click", selectNearestWaveformHit);
646
 
647
  const dropzone = $("dropzone");
web/index.html CHANGED
@@ -187,6 +187,51 @@
187
  <audio id="sampleAudio" controls></audio>
188
  </article>
189
  </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
190
  <div class="result-columns">
191
  <section>
192
  <h3>Representative samples</h3>
@@ -208,7 +253,7 @@
208
  <table id="hitsTable">
209
  <thead>
210
  <tr>
211
- <th>Audition</th><th>#</th><th>Label</th><th>Cluster</th><th>Onset</th><th>Duration</th><th>Energy</th><th>File</th>
212
  </tr>
213
  </thead>
214
  <tbody></tbody>
 
187
  <audio id="sampleAudio" controls></audio>
188
  </article>
189
  </div>
190
+
191
+ <section class="supervision-panel" aria-live="polite">
192
+ <div class="supervision-header">
193
+ <div>
194
+ <h3>Interactive supervision</h3>
195
+ <p class="subtle">Moves, locks, suppressions, favorites, and accepted suggestions are saved as replayable semantic state next to the run manifest.</p>
196
+ </div>
197
+ <div class="supervision-actions">
198
+ <button id="refreshStateButton" class="ghost-button" type="button">Refresh state</button>
199
+ <button id="undoButton" class="ghost-button" type="button" disabled>Undo edit</button>
200
+ </div>
201
+ </div>
202
+ <div id="supervisionSummary" class="state-summary">No interactive state loaded.</div>
203
+ <div class="supervision-tools">
204
+ <label>Target cluster
205
+ <select id="targetClusterSelect"></select>
206
+ </label>
207
+ <button id="moveHitButton" class="secondary-button" type="button" disabled>Move selected hit</button>
208
+ <button id="pullHitButton" class="secondary-button" type="button" disabled>Pull into new cluster</button>
209
+ <button id="acceptHitButton" class="secondary-button" type="button" disabled>Accept hit</button>
210
+ <button id="favoriteHitButton" class="secondary-button" type="button" disabled>Favorite as representative</button>
211
+ <button id="suppressHitButton" class="secondary-button danger-button" type="button" disabled>Suppress as bleed</button>
212
+ <button id="lockClusterButton" class="secondary-button" type="button" disabled>Lock target cluster</button>
213
+ <button id="explainClusterButton" class="secondary-button" type="button" disabled>Explain target cluster</button>
214
+ </div>
215
+ <div class="supervision-grid">
216
+ <article>
217
+ <h4>Outlier-first review queue</h4>
218
+ <div id="reviewQueue" class="compact-list"></div>
219
+ </article>
220
+ <article>
221
+ <h4>Cluster board</h4>
222
+ <div id="clusterBoard" class="compact-list"></div>
223
+ </article>
224
+ <article>
225
+ <h4>Suggestion inbox</h4>
226
+ <div id="suggestionInbox" class="compact-list"></div>
227
+ </article>
228
+ <article>
229
+ <h4>Constraint / event log</h4>
230
+ <div id="stateLog" class="compact-list"></div>
231
+ </article>
232
+ </div>
233
+ <pre id="clusterExplanation" class="explanation empty">Select a cluster and click Explain.</pre>
234
+ </section>
235
  <div class="result-columns">
236
  <section>
237
  <h3>Representative samples</h3>
 
253
  <table id="hitsTable">
254
  <thead>
255
  <tr>
256
+ <th>Audition</th><th>#</th><th>Label</th><th>Cluster</th><th>Confidence</th><th>Flags</th><th>Onset</th><th>Duration</th><th>Energy</th><th>File</th>
257
  </tr>
258
  </thead>
259
  <tbody></tbody>
web/styles.css CHANGED
@@ -99,3 +99,24 @@ tr.selected td { background: rgba(139,211,255,.12); }
99
  tr[data-hit-index] { cursor: pointer; }
100
  tr[data-hit-index]:hover td { background: rgba(255,255,255,.045); }
101
  @media (max-width: 760px) { .review-grid { grid-template-columns: 1fr; } }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
  tr[data-hit-index] { cursor: pointer; }
100
  tr[data-hit-index]:hover td { background: rgba(255,255,255,.045); }
101
  @media (max-width: 760px) { .review-grid { grid-template-columns: 1fr; } }
102
+ .supervision-panel { border: 1px solid var(--line); border-radius: 24px; background: rgba(0,0,0,.14); padding: 16px; margin: 0 0 20px; }
103
+ .supervision-header { display: flex; align-items: flex-start; justify-content: space-between; gap: 16px; margin-bottom: 14px; }
104
+ .supervision-header h3, .supervision-grid h4 { margin: 0; }
105
+ .supervision-actions, .row-actions { display: flex; gap: 8px; flex-wrap: wrap; justify-content: flex-end; }
106
+ .state-summary { display: flex; flex-wrap: wrap; gap: 8px; margin-bottom: 14px; color: #dbe5f7; }
107
+ .state-summary span { border: 1px solid var(--line); border-radius: 999px; background: rgba(255,255,255,.06); padding: 7px 10px; font-size: 12px; font-weight: 800; }
108
+ .supervision-tools { display: grid; grid-template-columns: minmax(220px, 1fr) repeat(7, auto); gap: 10px; align-items: end; margin-bottom: 16px; }
109
+ .danger-button { border-color: rgba(255,109,122,.35); color: #ffd4d8; }
110
+ .supervision-grid { display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 12px; }
111
+ .compact-list { display: grid; gap: 8px; max-height: 300px; overflow: auto; }
112
+ .compact-row, .suggestion-row, .log-row { width: 100%; display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 10px; align-items: center; border: 1px solid var(--line); border-radius: 14px; padding: 10px; background: rgba(0,0,0,.14); color: var(--text); text-align: left; }
113
+ .compact-row strong, .suggestion-row strong, .log-row strong { display: block; font-size: 13px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
114
+ .compact-row small, .suggestion-row small, .log-row small { display: block; color: var(--muted); font-size: 11px; margin-top: 3px; line-height: 1.35; }
115
+ .compact-row.locked { border-color: rgba(85,230,165,.45); background: rgba(85,230,165,.08); }
116
+ .compact-row.suppressed, tr.suppressed td { opacity: .62; text-decoration: line-through; }
117
+ .log-row.constraint { border-color: rgba(200,165,255,.26); }
118
+ .explanation { min-height: 120px; max-height: 320px; overflow: auto; border: 1px solid var(--line); border-radius: 16px; background: #05070b; color: #b9d7e9; padding: 12px; margin: 14px 0 0; font-size: 12px; line-height: 1.45; }
119
+ tr.low-confidence td { background: rgba(255,202,107,.06); }
120
+ tr.low-confidence.selected td { background: rgba(139,211,255,.15); }
121
+ @media (max-width: 1320px) { .supervision-tools { grid-template-columns: repeat(3, minmax(0, 1fr)); } .supervision-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } }
122
+ @media (max-width: 760px) { .supervision-header { display: block; } .supervision-actions { justify-content: flex-start; margin-top: 10px; } .supervision-tools, .supervision-grid { grid-template-columns: 1fr; } }