# Backend scripts

Utility / probe scripts for tagging accuracy, throughput, and model audits.

## Tag recall benchmark (ML vs WD)

UI: **Debug → Tag recall benchmark**.

Pinned suite: `backend/app/data/tag_recall_suite.json`  
Cache (gitignored): `sample_data/tag_recall_suite/`

```text
.venv\Scripts\python.exe backend\scripts\fetch_tag_recall_suite.py --rebuild
.venv\Scripts\python.exe backend\scripts\run_tag_recall_eval.py
```

Latest report: `backend/scripts/out/tag_recall_eval_latest.json`.

False-positive rates on the same suite (selected Settings folders → taxonomy
evidence tags):

```text
.venv\Scripts\python.exe backend\scripts\run_tag_fp_eval.py --also-wd-threshold
```

UI: **Debug → Tag false-positive benchmark**.  
Latest report: `backend/scripts/out/tag_fp_eval_latest.json`.

## THR3SHR debug: real-life vs anime eval

UI: **Debug → Real-life vs anime**.

API: `POST /api/debug/realism-eval` with
`{"count_per_class": 20, "compare_models": true}`.

CLI (thorough, multi-model + local video probe):

```text
.venv\Scripts\python.exe backend\scripts\run_realism_debug_eval.py 20
```

Latest report: `backend/scripts/out/realism_debug_eval_latest.json`.

Photos: RandomUser people portraits (remote). Anime: Safebooru typical +
`realistic` / `photorealistic` / `3d` edges. Routing uses production
`real_life` taxonomy.

## Real-life vs anime (`real_life` bucket)

Production `backend/app/data/taxonomy.json` includes a priority-`0` `real_life`
folder (aliases: `photo`, `Real Life`, …). Select that folder in the UI to
route accidental camera photos away from content buckets.

### Probe

```text
.venv\Scripts\python.exe backend\scripts\debug_realism_probe.py
```

Report: `backend/scripts/out/realism_probe_report.json`.

Findings (12 photos + 12 anime):

- **GO** with WD, best model **`wd_eva02_large`**
- WD v3 has **`realistic` / `photorealistic`** only (no `photo_(medium)` / `3d`)
- Anime max `realistic` ≈ 0.004 on EVA02; photos often 0.11–0.99
- Inference always includes `realistic`/`photorealistic` when score ≥ **0.10**
  even if below `wd_general_threshold` (so moon/sky shots are not dropped)

### Evidence policy

| Tag | Weight | Notes |
|-----|--------|--------|
| `photorealistic` | 1.0 | Hard (WD + ML) |
| `realistic` | 1.0 | Hard on WD (probe-safe vs anime) |
| `photo_(medium)` | 1.0 | Hard, **ML only** |
| `3d` | 0.45 | Soft, **ML only** |

Prop/framing tags (`photo_(object)`, `selfie`, `photo_background`, …) are
ignored so they never win the folder alone.