Backend scripts

Utility / probe scripts for tagging accuracy, throughput, and model audits.

Tag recall benchmark (ML vs WD)

UI: Debug → Tag recall benchmark.

Pinned suite: backend/app/data/tag_recall_suite.json
Cache (gitignored): sample_data/tag_recall_suite/

.venv\Scripts\python.exe backend\scripts\fetch_tag_recall_suite.py --rebuild
.venv\Scripts\python.exe backend\scripts\run_tag_recall_eval.py

Latest report: backend/scripts/out/tag_recall_eval_latest.json.

False-positive rates on the same suite (selected Settings folders → taxonomy evidence tags):

.venv\Scripts\python.exe backend\scripts\run_tag_fp_eval.py --also-wd-threshold

UI: Debug → Tag false-positive benchmark.
Latest report: backend/scripts/out/tag_fp_eval_latest.json.

THR3SHR debug: real-life vs anime eval

UI: Debug → Real-life vs anime.

API: POST /api/debug/realism-eval with {"count_per_class": 20, "compare_models": true}.

CLI (thorough, multi-model + local video probe):

.venv\Scripts\python.exe backend\scripts\run_realism_debug_eval.py 20

Latest report: backend/scripts/out/realism_debug_eval_latest.json.

Photos: RandomUser people portraits (remote). Anime: Safebooru typical + realistic / photorealistic / 3d edges. Routing uses production real_life taxonomy.

Real-life vs anime (`real_life` bucket)

Production backend/app/data/taxonomy.json includes a priority-0 real_life folder (aliases: photo, Real Life, …). Select that folder in the UI to route accidental camera photos away from content buckets.

Probe

.venv\Scripts\python.exe backend\scripts\debug_realism_probe.py

Report: backend/scripts/out/realism_probe_report.json.

Findings (12 photos + 12 anime):

GO with WD, best model wd_eva02_large
WD v3 has realistic / photorealistic only (no photo_(medium) / 3d)
Anime max realistic ≈ 0.004 on EVA02; photos often 0.11–0.99
Inference always includes realistic/photorealistic when score ≥ 0.10 even if below wd_general_threshold (so moon/sky shots are not dropped)

Evidence policy

Tag	Weight	Notes
`photorealistic`	1.0	Hard (WD + ML)
`realistic`	1.0	Hard on WD (probe-safe vs anime)
`photo_(medium)`	1.0	Hard, ML only
`3d`	0.45	Soft, ML only

Prop/framing tags (photo_(object), selfie, photo_background, …) are ignored so they never win the folder alone.