docs/interactive-ux/ARCHITECTURE_NOTES.md

# Architecture notes for supervised interactive extraction

## Required shift

Original extraction flow:

```text
audio → pipeline → result artifacts
```

Current interactive foundation:

```text
audio/cache → immutable manifest/artifacts → supervision_state.json → reactive UI → user constraints/events/suggestions
```

The current implementation keeps the batch extraction artifacts immutable. Interactive edits mutate `supervision_state.json`, then `supervised_export.py` can render those edits into a separate `supervised/` artifact tree. The original `manifest.json`, original sample WAVs, original MIDI/reconstruction, and original ZIP remain untouched.

## Implemented modules

| Module/file | Responsibility |
|---|---|
| `pipeline_runner.py` | Batch extraction, timing, manifests, review-hit WAV exports |
| `sample_extractor.py` | Audio analysis, classification, batch/online clustering, export helpers |
| `supervised_state.py` | Persistent semantic job state, constraints, events, confidence, suggestions, force-onset, restore, undo |
| `app.py` | FastAPI endpoints for batch jobs, supervised state mutations, force-onset, restore, and edited export |
| `supervised_export.py` | Converts semantic state into edited WAV/MIDI/reconstruction/ZIP artifacts under `supervised/` |
| `web/app.js` | Browser state rendering, review queue, cluster board, suggestions, actions |
| `web/index.html` | Workstation layout and interactive supervision panel |
| `web/styles.css` | Visual treatment for low confidence, suppression, locks, panels |

## Core entities as implemented

`supervised_state.py` stores JSON dictionaries equivalent to these shapes:

```ts
type Hit = {
  id: string;
  index: number;
  label: string;
  cluster_id: string;
  original_cluster_id: string;
  cluster_label: string;
  onset_sec: number;
  duration_ms: number;
  rms_energy: number;
  spectral_centroid_hz: number;
  file: string;
  is_representative: boolean;
  source: "detected" | "forced";
  suppressed: boolean;
  favorite: boolean;
  review_status: "unreviewed" | "accepted" | "favorite" | "suppressed";
  confidence: number;
  confidence_reasons: string[];
  explicit: boolean;
};

type Cluster = {
  id: string;
  label: string;
  classification: string;
  hit_ids: string[];
  representative_hit_id: string | null;
  locked: boolean;
  user_named: boolean;
  confidence: number;
  confidence_reasons: string[];
  suppressed_count: number;
  original_id: string | null;
};

type Constraint =
  | { id: string; type: "must-link"; a: string; b: string; source: string }
  | { id: string; type: "cannot-link"; a: string; b: string; source: string }
  | { id: string; type: "force-cluster"; hit_id: string; cluster_id: string; source: string }
  | { id: string; type: "lock-cluster"; cluster_id: string; locked: boolean; source: string }
  | { id: string; type: "suppress-pattern"; example_hit_id: string; reason: string; source: string }
  | { id: string; type: "pin-representative"; hit_id: string; cluster_id: string; source: string };
```

## Current state file

Each completed run now gets:

```text
.runs/<job_id>/output/manifest.json
.runs/<job_id>/output/supervision_state.json
.runs/<job_id>/output/supervised/manifest.json        # after edited export
.runs/<job_id>/output/supervised/sample-pack.zip      # after edited export
```

`manifest.json` is the immutable batch result. `supervision_state.json` is the mutable, replayable semantic state.

## Implemented API additions

```text
GET    /api/jobs/{job_id}/state
POST   /api/jobs/{job_id}/hits/{hit_id}/move
POST   /api/jobs/{job_id}/hits/{hit_id}/pull-out
POST   /api/jobs/{job_id}/hits/{hit_id}/suppress
POST   /api/jobs/{job_id}/hits/{hit_id}/review
POST   /api/jobs/{job_id}/hits/{hit_id}/restore
POST   /api/jobs/{job_id}/hits/force-onset
POST   /api/jobs/{job_id}/export
POST   /api/jobs/{job_id}/clusters/{cluster_id}/lock
GET    /api/jobs/{job_id}/suggestions
POST   /api/jobs/{job_id}/suggestions/{suggestion_id}/accept
POST   /api/jobs/{job_id}/suggestions/{suggestion_id}/reject
GET    /api/jobs/{job_id}/explain/cluster/{cluster_id}
POST   /api/jobs/{job_id}/undo
```

## Current confidence scoring

Initial confidence is heuristic and deterministic. It combines:

- cluster size,
- cluster label purity,
- representative presence,
- lock state,
- hit label agreement with cluster label,
- energy rank,
- rough duration reasonableness,
- representative/favorite/explicit assignment state,
- suppression state.

This is good enough to drive the review queue but should be replaced or supplemented by cached feature-vector margins.

## Current suggestion engine

Implemented suggestion types:

```ts
type Suggestion =
  | { type: "move-hits"; hit_ids: string[]; target_cluster_id: string; confidence: number; reason: string }
  | { type: "split-hits"; hit_ids: string[]; source_cluster_id: string; target_cluster_id: string; confidence: number; reason: string }
  | { type: "suppress-hits"; hit_ids: string[]; confidence: number; reason: string };
```

Suggestion generation currently uses label, spectral centroid, and RMS-energy similarity. Open suggestions include exact `diff` previews showing affected hits and cluster counts before/after. Accepted suggestions become explicit constraints/examples.

## Event log

Examples now emitted:

```text
job.state.created
constraint.created
hit.moved
hit.pulled_out
cluster.locked
cluster.unlocked
hit.suppressed
hit.reviewed
suggestion.created
suggestion.accepted
suggestion.rejected
state.undo
hit.force_onset
hit.restored
supervised.exported
```

The UI renders recent events and constraints in the supervision panel.

## Local recomputation boundary

Implemented now:

```text
semantic edit
→ update hit/cluster membership in supervision_state.json
→ append constraints/events
→ generate heuristic suggestions
→ recompute hit/cluster confidence and review queue
```

Still not implemented:

```text
semantic edit
→ load cached feature vectors
→ choose affected neighborhood
→ run constrained local reclustering
→ update suggestion/recluster preview from real feature margins
```

Implemented now:

```text
semantic edit
→ export supervised state
→ write edited WAV/MIDI/reconstruction/ZIP under supervised/
```

## UI state implications

Implemented panels:

- waveform + onset audition,
- representative samples,
- detected hit review,
- outlier-first review queue,
- cluster board,
- suggestion inbox,
- constraint/history inspector,
- cluster explanation drawer,
- export/download panel for the original batch run.

Still missing:

- cluster merge/relabel/split controls,
- edited-vs-original comparison view,
- cached feature-neighborhood local reclustering.

## Implementation warning

Automatic propagation must stay conservative. The current implementation follows this by creating suggestions rather than silently moving/suppressing batches. Every semantic edit is undoable.