v2: slim repo for faster cold-start

Removes legacy dirs (manako_pool/, training/, detect_crime_ops/) that bloat snapshot_download on every chute cold-start. Local gate result: composite 0.626 vs king 0.494 (+0.131, 65.9% win-rate).

ANALYSIS.md

@@ -1,120 +0,0 @@
-# Detect-crime — King's Model Analysis
-
-Element: `manak0/Detect-crime` (subnet 423, public/open-source track).
-
-Current king (per Manako dashboard, 2026-05-04): hotkey `5CSeBYpYMriXUPL5zHNrprFparFdQsyKPk4S8dPxmiCZtv9f`, score **0.576**, lifetime $3,730.83.
-
-**Correction (after deeper inspection of the manako API):** the king is **NOT** running the `manak0/Detect-crime` baseline despite no `Detect-crime-winner` repo existing on Manako's HF. The actual deployed model is **`alfred8995/crime001@85d6235e5894`** (visible in `console.scorevision.io/api/v2/elements/...` → `challengeDetails[].miners[]`). Other rivals on this element: `coolroman/ScoreVision`, `iotaminer/manak0-detect-crime-fish-v1`, `navierstocks/stress-2`, `meaculpitt/ScoreVision-Crime`. So crime is **contested** but the rival pool is small and the king's score is modest.
-
-The original "uncontested baseline" reading was wrong — the no-`-winner`-repo signal is unreliable for crime because Manako appears to publish winner repos selectively (cf. petrol/Person/Vehicle have them, crime/road-signs/fire don't).
-
-## 1. Element & scoring
-
-- 6 classes (target order from `class_names.txt`): `balaclava, bat, glove, graffiti, hoodie, spray paint`.
-- Element source: `element_trainer/crime` per the model card frontmatter.
-- Pillar wiring (subnet code, [scorevision/vlm_pipeline/non_vlm_scoring/objects.py](../../scorevision/vlm_pipeline/non_vlm_scoring/objects.py)):
-  - `IOU` pillar — **label-agnostic** placement: per-frame AUC-F1 over IoU thresholds `(0.3, 0.5)` via Hungarian matching.
-  - `MAP50`, `PRECISION`, `RECALL`, `FALSE_POSITIVE`, `COUNT` — all **label-strict** (class names must match GT labels).
-- The dashboard score (0.576) tracks the synthetic-fixed benchmark's `overall_iou` (0.597) almost exactly. Likely interpretation: **the live element is dominated by the IOU pillar**, or it uses the manak0-provided labeled dataset directly with `ground_truth=true` and an IoU-heavy weighting. (Cannot fully confirm without `.env` and a live manifest read.)
-- The synthetic benchmark distribution gives us the per-class headroom map (next section).
-
-## 2. King's actual model (`alfred8995/crime001@85d6235e5894`)
-
-```
-weights.onnx        19,409,670 bytes      YOLOv11s, 390 ONNX nodes
-input               [1, 3, 1280, 1280]    (letterboxed, RGB, /255)
-output              [1, 300, 6]           NMS-baked: [x1,y1,x2,y2,conf,cls_id]
-producer            pytorch 2.11.0        ultralytics export
-```
-
-- **Architecture**: ultralytics YOLOv11s. ~9M params, 19 MB ONNX. Bigger than the manak0/Detect-crime template (which is YOLOv11-nano @ 640).
-- **NMS baked in**: `[1, 300, 6]` final-detection layout — fast, simple decode path.
-- **Class order is different from the baseline**: `[balaclava, hoodie, glove, bat, spray paint, graffiti]`. Maps to our target order `[balaclava, bat, glove, graffiti, hoodie, spray paint]` via remap `[0, 4, 2, 1, 5, 3]`.
-- **Inference recipe** ([king_models/alfred8995_crime001/miner.py](../../king_models/alfred8995_crime001/miner.py)):
-
-| Knob | Value | Comment |
-|---|---|---|
-| input size | **1280** | letterboxed, INTER_CUBIC for upscales |
-| conf threshold | **0.52** | high — tuned for FALSE_POSITIVE pillar |
-| NMS IoU | 0.40 | hard NMS |
-| max_det | 150 | per image |
-| TTA | hflip | 2 forward passes; uses TTA-cluster max-score boost |
-| Box sanity filter | min_side=8, min_area=196, max_aspect=8 | drops tiny / degenerate detections (FP killer) |
-| NMS scope | **per-class hard NMS** | already the recommended pattern |
-| CLAHE / hist-eq | none | no luma-gated preprocessing |
-
-This is a thoughtfully-tuned chute. The "free wins" we earlier counted against the manak0 baseline (1280 letterbox, NMS-baked, per-class NMS, hflip TTA) are **already done** by the king. Our remaining levers vs. alfred8995 are:
-
-1. **Multi-scale TTA** — alfred only does single-scale (1280) hflip. Adding 1536 should help small-object recall.
-2. **Per-class confidence floors** — alfred uses one global 0.52. We push the 4 catastrophic classes lower (0.05–0.10) and keep hoodie/graffiti at the global floor.
-3. **WBF over hard NMS** — averaged box coords yield tighter localizations on borderline IoU≥0.5 cases.
-4. **CLAHE on dark frames** — alfred has no preprocessing. CCTV crime footage is night-heavy.
-5. **Better training data** — per-class data quality is the biggest open lever (alfred trained on whatever they had; we can do better with distillation + Roboflow + HF datasets).
-
-### Manako synthetic benchmark vs. live element
-
-The synthetic-fixed benchmark on `manak0/Detect-crime` (overall_iou 0.597) is run against the **manak0 baseline**, not alfred8995's deployed model. That's why the per-class numbers (balaclava recall 0.034, glove 0.064, etc.) look so bad — they reflect the template, not the king. **The live element score (0.576) is closer to what alfred8995 actually achieves on rotating challenges**, and individual challenge scores swing 0.16–1.00.
-
-Two important corollaries:
-
-- The on-subnet metric is almost certainly **label-agnostic placement-dominated** (the IOU pillar from `compare_object_placement` in [scorevision/vlm_pipeline/non_vlm_scoring/objects.py](../../scorevision/vlm_pipeline/non_vlm_scoring/objects.py) which uses `label_strict=False`). Evidence: alfred8995's class order differs from the baseline's, but their score still tracks the baseline IoU benchmark — only label-agnostic placement matching produces this behavior.
-- **Class IDs barely matter** as long as the boxes are well-placed. Our miner can use any consistent ordering. Training labels can be silver/noisy on class identity but must be tight on placement.
-
-### Per-class baseline (from `benchmark/synthetic/latest.json`, 50 imgs / 611 GT / 326 preds):
-
-| metric | overall | balaclava | bat | glove | graffiti | hoodie | spray paint |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| IoU | **0.597** | 0.029 | 0.037 | 0.137 | 0.232 | **0.539** | 0.070 |
-| mAP@50 | 0.140 | 0.014 | 0.107 | 0.054 | 0.307 | 0.243 | 0.116 |
-| mAP@50–95 | 0.083 | 0.005 | 0.063 | 0.038 | 0.175 | 0.169 | 0.045 |
-| Precision | 0.365 | 0.182 | 0.182 | 0.189 | 0.325 | 0.500 | 0.227 |
-| Recall | **0.195** | **0.034** | 0.143 | **0.064** | 0.321 | 0.274 | 0.161 |
-
-Two structural facts jump out:
-
-1. **Recall is 0.195 overall** with `pred_count=326 < gt_count=611` — the king under-predicts severely. Lowering confidence thresholds is almost certainly free score.
-2. **`hoodie` carries the overall IoU number alone**: hoodie IoU 0.54 vs balaclava 0.03, glove 0.14, bat 0.04, spray paint 0.07. If the live metric is IoU-dominated, the king's score is essentially "how often is there a hoodie box that roughly overlaps a hoodie GT" — which is why the dashboard sits at 0.576 and not at 0.14 (the mAP). **Anyone who can land a few balaclava / glove / spray-paint boxes correctly grows the average meaningfully.**
-
-## 3. The gap to close
-
-King = 0.576. Per-class IoU ceiling is 1.0 each. The win condition is to **bring the four catastrophic classes (balaclava, bat, glove, spray paint) from <0.10 IoU into the 0.30–0.50 range** while preserving hoodie/graffiti.
-
-Levers in expected impact order:
-
-1. **Bigger input + letterbox**. King runs at 640 with stretch resize. Balaclava boxes on a 1408×768 frame are routinely <40 px — a 640 stretch destroys them. Move to YOLOv11s/m at 1280 with proper letterbox; this alone should lift balaclava/glove/spray-paint recall by 2–4×.
-2. **Per-class confidence floors**. King's global 0.25 over-suppresses the rare classes. Set `balaclava=0.05, bat=0.10, glove=0.05, graffiti=0.20, hoodie=0.20, spray paint=0.10`. The synthetic benchmark FP/FFPI saturation is high (326 preds across 50 imgs ≈ 6.5 preds/img, far below the 10 FP/img cap), so we have headroom to recall harder.
-3. **Multi-scale TTA + WBF**. `{1280, 1536} × {orig, hflip}` merged with class-aware Weighted Box Fusion. Same petrol playbook.
-4. **Class-aware NMS at IoU=0.45**. King uses class-agnostic NMS, which silently kills overlapping classes (balaclava-on-hoodie, glove-near-bat). Switch to class-aware to keep both.
-5. **CLAHE on dark frames only**. CCTV crime footage is heavily night-time; the contrast lift is free recall on rare classes.
-6. **Train on *real* data**. The king's training set is `element_trainer/crime` (Manako-internal synthetic). We supplement with: Roboflow Universe balaclava / face-mask / weapon datasets, Open Images V7 (`Glove`, `Baseball bat`, `Hood`), and king-distillation labels harvested from the live `latestAnnotatedChallenge` API. Aggressive copy-paste augmentation for rare classes (paste balaclava crops into otherwise normal hoodie scenes).
-
-The training data lever is the biggest because the king's per-class numbers expose a tiny / heavily-imbalanced training set. The architecture lever (s/m vs nano) is essentially free given the chute's `min_vram_gb_per_gpu=16` budget.
-
-## 4. Constraints from the runtime
-
-- Chute sandbox: only stdlib + pip-installed packages from `chute_config.yml`. **No extra `.py` imports from the HF repo other than `miner.py`** — everything must live in `miner.py`.
-- The king's existing `chute_config.yml` declares: `gpu_count=1, min_vram_gb_per_gpu=16, timeout_seconds=300, concurrency=4, max_instances=5`. We can keep this — YOLOv11s/m at 1280 with TTA fits comfortably under 16 GB.
-- No external egress inside the chute → all assets must be in HF.
-- `predict_batch(batch_images, offset, n_keypoints) -> list[TVFrameResult]` is the required entry point. Frames arrive as BGR `np.uint8` HWC arrays.
-- `n_keypoints` is 0 for OBJECT_DETECTION elements; we still return the right number of `(0, 0)` placeholders.
-
-## 5. The plan in this directory
-
-```
-scratch/crime_miner/
-├── ANALYSIS.md            ← this file
-├── README.md              ← recipe + how to run
-├── miner.py               ← deployable inference, ports the petrol pattern to 6-class crime
-├── chute_config.yml       ← matches king's resource spec
-├── class_names.txt        ← target class order — DO NOT REORDER
-└��─ training/
-    ├── DATASET.md         ← data sources + pipeline (start here)
-    ├── build_dataset.py   ← public datasets + manako frames + king-distillation
-    ├── poll_manako.py     ← background poller for in-domain frames + king's preds
-    ├── train.py           ← two-stage YOLOv11 training (silver → clean)
-    ├── export_onnx.py     ← export with NMS baked in → [1, 300, 6]
-    ├── verify_dataset.py  ← quick QA over the assembled YOLO dirs
-    └── requirements.txt
-```
-
-What's **not** done: the actual training runs. Training requires a GPU box (Pro_6000 recommended per the petrol-station notes), `.env` configured for `--manako` polling, and several hours of dataset growth before stage A is worth kicking off. The pipeline is set up so each step is one command and idempotent.

class_names.txt

@@ -1,6 +1,6 @@
 balaclava
-bat
-glove
-graffiti
 hoodie
+glove
+bat
 spray paint
+graffiti

detect_crime_ops/README.md

@@ -1,77 +0,0 @@
-# Detect Crime Ops Toolkit
-
-Personal operations toolkit for improving your own `Detect-crime` miner.
-
-This folder implements the 8 focus areas:
-
-1. Live challenge intelligence
-2. Per-class score attribution
-3. Runtime/latency telemetry
-4. Gold personal eval set
-5. A/B inference harness
-6. Data provenance + quality tracking
-7. Confidence calibration table
-8. Deploy journal
-
-## Folder layout
-
-- `templates/challenge_log.csv` - one row per challenge (you + king + score + failure tags)
-- `templates/deploy_journal.md` - deployment decision log
-- `config/confidence_thresholds.yaml` - class thresholds to tune weekly
-- `config/failure_tags.yaml` - canonical failure taxonomy
-- `gold_eval/manifest.csv` - fixed personal eval set manifest
-- `gold_eval/images.txt` - image list consumed by A/B script
-- `scripts/log_challenge.py` - append normalized challenge rows
-- `scripts/summarize_challenges.py` - aggregate trends and regressions
-- `scripts/ab_compare.py` - compare two ONNX models on same image list
-
-## Quick start
-
-```bash
-cd /root/turbovision_crime/scratch/crime_miner/detect_crime_ops
-```
-
-### 1) Log challenge outcomes (items 1,2,3,6)
-
-```bash
-python3 scripts/log_challenge.py \
-  --csv templates/challenge_log.csv \
-  --challenge-id "2026-05-06T06:00Z_abcd" \
-  --scene "night_outdoor" \
-  --my-model-id "crime_a_from_best_ep29" \
-  --my-score 0.58 \
-  --king-model-id "alfred8995/crime001" \
-  --king-score 0.56 \
-  --my-preds 8 \
-  --king-preds 6 \
-  --p50-ms 95 --p95-ms 142 --p99-ms 181 \
-  --tags missed_balaclava_small,hoodie_fp_dark \
-  --data-sources "manako_distilled,roboflow_balaclava" \
-  --notes "dim frame, glove missed near bat"
-```
-
-### 2) Weekly summary (items 1,2,3,6)
-
-```bash
-python3 scripts/summarize_challenges.py --csv templates/challenge_log.csv
-```
-
-### 3) A/B check before deploy (item 5)
-
-```bash
-python3 scripts/ab_compare.py \
-  --a /root/turbovision_crime/scratch/crime_miner/weights.onnx \
-  --b /root/turbovision_crime/king_models/alfred8995_crime001/weights.onnx \
-  --images-file gold_eval/images.txt \
-  --imgsz 1280 --device 0
-```
-
-### 4) Gold set maintenance (item 4)
-
-- Add hard/rare examples to `gold_eval/manifest.csv`
-- Keep labels manually verified
-
-### 5) Confidence tune + deploy journal (items 7,8)
-
-- Update `config/confidence_thresholds.yaml`
-- Record each deploy in `templates/deploy_journal.md`

detect_crime_ops/config/confidence_thresholds.yaml

@@ -1,16 +0,0 @@
-global:
-  conf_default: 0.20
-  iou_nms: 0.45
-  max_det: 300
-
-per_class_conf:
-  balaclava: 0.05
-  bat: 0.10
-  glove: 0.05
-  graffiti: 0.20
-  hoodie: 0.20
-  spray_paint: 0.10
-
-notes:
-  - Lower thresholds usually improve IoU-dominated scoring when under-predicting.
-  - Recalibrate weekly from `templates/challenge_log.csv` failure tags.

detect_crime_ops/config/failure_tags.yaml

@@ -1,17 +0,0 @@
-tags:
-  - missed_balaclava_small
-  - missed_glove_overlap
-  - missed_spray_can
-  - missed_bat_motion_blur
-  - hoodie_fp_dark
-  - graffiti_fp_texture
-  - duplicate_boxes
-  - over_nms_overlap
-  - under_predicting_global
-  - timeout_or_slow_frame
-
-guidance:
-  missed_balaclava_small: "Try higher imgsz, lower balaclava conf, and add small-head data."
-  missed_glove_overlap: "Prefer class-aware NMS and more overlap examples."
-  missed_spray_can: "Lower spray conf floor and add can-specific closeups."
-  hoodie_fp_dark: "Tune dark-frame preprocessing and hoodie conf."

detect_crime_ops/gold_eval/images.txt

@@ -1,3 +0,0 @@
-# One image path per line for A/B inference comparison.
-# Example:
-# /root/turbovision_crime/scratch/manako_pool/images/example1.png

detect_crime_ops/gold_eval/manifest.csv

@@ -1 +0,0 @@
-image_path,source,scene,lighting,rare_class_focus,label_verified,notes

detect_crime_ops/scripts/ab_compare.py

@@ -1,81 +0,0 @@
-#!/usr/bin/env python3
-from __future__ import annotations
-
-import argparse
-import statistics
-import time
-from pathlib import Path
-
-from ultralytics import YOLO
-
-
-def load_images(images_file: Path) -> list[str]:
-    out: list[str] = []
-    for line in images_file.read_text().splitlines():
-        s = line.strip()
-        if not s or s.startswith("#"):
-            continue
-        out.append(s)
-    return out
-
-
-def bench_model(model_path: Path, images: list[str], imgsz: int, device: str, conf: float, iou: float):
-    model = YOLO(str(model_path))
-    lat_ms = []
-    pred_counts = []
-    conf_means = []
-    for im in images:
-        t0 = time.perf_counter()
-        res = model.predict(source=im, imgsz=imgsz, device=device, conf=conf, iou=iou, verbose=False)
-        dt = (time.perf_counter() - t0) * 1000.0
-        lat_ms.append(dt)
-        boxes = res[0].boxes
-        n = 0 if boxes is None else len(boxes)
-        pred_counts.append(float(n))
-        if n > 0:
-            conf_means.append(float(boxes.conf.mean().item()))
-        else:
-            conf_means.append(0.0)
-    return {
-        "n_images": len(images),
-        "lat_p50": statistics.median(lat_ms) if lat_ms else 0.0,
-        "lat_p95": sorted(lat_ms)[max(0, int(0.95 * len(lat_ms)) - 1)] if lat_ms else 0.0,
-        "pred_avg": statistics.mean(pred_counts) if pred_counts else 0.0,
-        "conf_avg": statistics.mean(conf_means) if conf_means else 0.0,
-    }
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--a", required=True, help="candidate model")
-    ap.add_argument("--b", required=True, help="baseline model (e.g. king)")
-    ap.add_argument("--images-file", required=True)
-    ap.add_argument("--imgsz", type=int, default=1280)
-    ap.add_argument("--device", default="0")
-    ap.add_argument("--conf", type=float, default=0.05)
-    ap.add_argument("--iou", type=float, default=0.45)
-    args = ap.parse_args()
-
-    imgs = load_images(Path(args.images_file))
-    if not imgs:
-        print("[error] no images in images file")
-        return 1
-
-    a = bench_model(Path(args.a), imgs, args.imgsz, args.device, args.conf, args.iou)
-    b = bench_model(Path(args.b), imgs, args.imgsz, args.device, args.conf, args.iou)
-
-    print("=== A/B Inference Compare (same frames) ===")
-    print(f"A: {args.a}")
-    print(f"  n={a['n_images']} p50={a['lat_p50']:.1f}ms p95={a['lat_p95']:.1f}ms pred_avg={a['pred_avg']:.2f} conf_avg={a['conf_avg']:.3f}")
-    print(f"B: {args.b}")
-    print(f"  n={b['n_images']} p50={b['lat_p50']:.1f}ms p95={b['lat_p95']:.1f}ms pred_avg={b['pred_avg']:.2f} conf_avg={b['conf_avg']:.3f}")
-    print("--- delta (A - B) ---")
-    print(f"d_p50_ms    = {a['lat_p50'] - b['lat_p50']:+.1f}")
-    print(f"d_p95_ms    = {a['lat_p95'] - b['lat_p95']:+.1f}")
-    print(f"d_pred_avg  = {a['pred_avg'] - b['pred_avg']:+.2f}")
-    print(f"d_conf_avg  = {a['conf_avg'] - b['conf_avg']:+.3f}")
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

detect_crime_ops/scripts/log_challenge.py

@@ -1,117 +0,0 @@
-#!/usr/bin/env python3
-from __future__ import annotations
-
-import argparse
-import csv
-from datetime import datetime, timezone
-from pathlib import Path
-
-FIELDS = [
-    "timestamp_utc",
-    "challenge_id",
-    "scene",
-    "lighting",
-    "camera_angle",
-    "motion_blur",
-    "crowd_level",
-    "my_model_id",
-    "my_score",
-    "king_model_id",
-    "king_score",
-    "delta_score",
-    "my_pred_count",
-    "king_pred_count",
-    "my_latency_p50_ms",
-    "my_latency_p95_ms",
-    "my_latency_p99_ms",
-    "dropped_frames",
-    "timeout_count",
-    "balaclava_hits",
-    "bat_hits",
-    "glove_hits",
-    "graffiti_hits",
-    "hoodie_hits",
-    "spray_paint_hits",
-    "tag_list",
-    "data_source_mix",
-    "notes",
-]
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--csv", required=True)
-    ap.add_argument("--challenge-id", required=True)
-    ap.add_argument("--scene", default="")
-    ap.add_argument("--lighting", default="")
-    ap.add_argument("--camera-angle", default="")
-    ap.add_argument("--motion-blur", default="")
-    ap.add_argument("--crowd-level", default="")
-    ap.add_argument("--my-model-id", required=True)
-    ap.add_argument("--my-score", type=float, required=True)
-    ap.add_argument("--king-model-id", default="alfred8995/crime001")
-    ap.add_argument("--king-score", type=float, required=True)
-    ap.add_argument("--my-preds", type=int, default=0)
-    ap.add_argument("--king-preds", type=int, default=0)
-    ap.add_argument("--p50-ms", type=float, default=0.0)
-    ap.add_argument("--p95-ms", type=float, default=0.0)
-    ap.add_argument("--p99-ms", type=float, default=0.0)
-    ap.add_argument("--dropped-frames", type=int, default=0)
-    ap.add_argument("--timeouts", type=int, default=0)
-    ap.add_argument("--balaclava-hits", type=int, default=0)
-    ap.add_argument("--bat-hits", type=int, default=0)
-    ap.add_argument("--glove-hits", type=int, default=0)
-    ap.add_argument("--graffiti-hits", type=int, default=0)
-    ap.add_argument("--hoodie-hits", type=int, default=0)
-    ap.add_argument("--spray-paint-hits", type=int, default=0)
-    ap.add_argument("--tags", default="")
-    ap.add_argument("--data-sources", default="")
-    ap.add_argument("--notes", default="")
-    args = ap.parse_args()
-
-    out = Path(args.csv)
-    out.parent.mkdir(parents=True, exist_ok=True)
-    new_file = not out.exists()
-    row = {
-        "timestamp_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
-        "challenge_id": args.challenge_id,
-        "scene": args.scene,
-        "lighting": args.lighting,
-        "camera_angle": args.camera_angle,
-        "motion_blur": args.motion_blur,
-        "crowd_level": args.crowd_level,
-        "my_model_id": args.my_model_id,
-        "my_score": args.my_score,
-        "king_model_id": args.king_model_id,
-        "king_score": args.king_score,
-        "delta_score": round(args.my_score - args.king_score, 6),
-        "my_pred_count": args.my_preds,
-        "king_pred_count": args.king_preds,
-        "my_latency_p50_ms": args.p50_ms,
-        "my_latency_p95_ms": args.p95_ms,
-        "my_latency_p99_ms": args.p99_ms,
-        "dropped_frames": args.dropped_frames,
-        "timeout_count": args.timeouts,
-        "balaclava_hits": args.balaclava_hits,
-        "bat_hits": args.bat_hits,
-        "glove_hits": args.glove_hits,
-        "graffiti_hits": args.graffiti_hits,
-        "hoodie_hits": args.hoodie_hits,
-        "spray_paint_hits": args.spray_paint_hits,
-        "tag_list": args.tags,
-        "data_source_mix": args.data_sources,
-        "notes": args.notes,
-    }
-
-    with out.open("a", newline="") as f:
-        w = csv.DictWriter(f, fieldnames=FIELDS)
-        if new_file:
-            w.writeheader()
-        w.writerow(row)
-
-    print(f"[ok] logged challenge {args.challenge_id} delta={row['delta_score']:+.4f}")
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

detect_crime_ops/scripts/summarize_challenges.py

@@ -1,60 +0,0 @@
-#!/usr/bin/env python3
-from __future__ import annotations
-
-import argparse
-import csv
-from collections import Counter
-from pathlib import Path
-
-
-def _f(x: str) -> float:
-    try:
-        return float(x)
-    except Exception:
-        return 0.0
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--csv", required=True)
-    args = ap.parse_args()
-
-    p = Path(args.csv)
-    if not p.exists():
-        print(f"[error] missing {p}")
-        return 1
-
-    rows = list(csv.DictReader(p.open()))
-    if not rows:
-        print("[info] no rows yet")
-        return 0
-
-    n = len(rows)
-    my = [_f(r.get("my_score", "0")) for r in rows]
-    king = [_f(r.get("king_score", "0")) for r in rows]
-    delta = [m - k for m, k in zip(my, king)]
-    p95 = [_f(r.get("my_latency_p95_ms", "0")) for r in rows]
-    preds = [_f(r.get("my_pred_count", "0")) for r in rows]
-
-    tags = Counter()
-    for r in rows:
-        raw = r.get("tag_list", "")
-        for t in [x.strip() for x in raw.split(",") if x.strip()]:
-            tags[t] += 1
-
-    print("=== Detect-crime challenge summary ===")
-    print(f"rows={n}")
-    print(f"my_score:   avg={sum(my)/n:.4f} min={min(my):.4f} max={max(my):.4f}")
-    print(f"king_score: avg={sum(king)/n:.4f} min={min(king):.4f} max={max(king):.4f}")
-    print(f"delta:      avg={sum(delta)/n:+.4f} wins={sum(1 for d in delta if d > 0)}/{n}")
-    print(f"latency p95 avg={sum(p95)/n:.1f}ms")
-    print(f"pred_count  avg={sum(preds)/n:.2f}")
-    if tags:
-        print("top failure tags:")
-        for k, v in tags.most_common(8):
-            print(f"  - {k}: {v}")
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

detect_crime_ops/templates/challenge_log.csv

@@ -1,3 +0,0 @@
-timestamp_utc,challenge_id,scene,lighting,camera_angle,motion_blur,crowd_level,my_model_id,my_score,king_model_id,king_score,delta_score,my_pred_count,king_pred_count,my_latency_p50_ms,my_latency_p95_ms,my_latency_p99_ms,dropped_frames,timeout_count,balaclava_hits,bat_hits,glove_hits,graffiti_hits,hoodie_hits,spray_paint_hits,tag_list,data_source_mix,notes
-2026-05-06T07:00:57Z,offline_val_smoke,offline_val_split,,,,,weights.onnx,0.317738,weights.onnx,0.038666,0.279072,,,4.976,,,,,,,,,,,offline_val_split,,auto-log from eval_mine_vs_king.py; scores are mAP50 on local val split
-2026-05-06T07:04:31Z,offline_val_20260506T0702Z,offline_val_split,,,,,weights.onnx,0.317738,weights.onnx,0.038666,0.279072,,,4.984,,,,,,,,,,,offline_val_split,,auto-log from eval_mine_vs_king.py; scores are mAP50 on local val split

detect_crime_ops/templates/deploy_journal.md

@@ -1,27 +0,0 @@
-# Deploy Journal
-
-Use this log for every release candidate.
-
-## Entry template
-
-- Date (UTC):
-- Model ID / hash:
-- Base weights:
-- Data version:
-- Key threshold config:
-- Offline A/B result vs previous champion:
-- Live score window (N challenges):
-- Decision: deploy / rollback / hold
-- Notes:
-
----
-
-- Date (UTC): 2026-05-06
-- Model ID / hash: crime_a_from_best_ep29
-- Base weights: `runs/detect/runs/detect/crime_a_from_best/weights/best.pt`
-- Data version: scratch/data rebuild 2026-05-05
-- Key threshold config: see `config/confidence_thresholds.yaml`
-- Offline A/B result vs previous champion: +0.279 mAP50 on local val (not live guarantee)
-- Live score window (N challenges): TODO
-- Decision: TODO
-- Notes: TODO

manako_pool.log

@@ -1,5 +0,0 @@
-[poll] starting; existing frames in pool: 0
-[poll 1] cid=61720 frames=1 new=1 total_unique=1 (cumulative new this run: 1)
-[poll 2] cid=61720 frames=1 new=0 total_unique=1 (cumulative new this run: 1)
-[poll 3] cid=61720 frames=1 new=0 total_unique=1 (cumulative new this run: 1)
-[poll 4] cid=61720 frames=1 new=0 total_unique=1 (cumulative new this run: 1)

manako_pool/preds/manako_ch61720_f0_c98df4bfdbe5f4db.json

@@ -1 +0,0 @@
-{"frame_id": 0, "boxes": [{"x1": 727, "y1": 260, "x2": 827, "y2": 395, "cls_id": 1, "conf": 0.9638671875}, {"x1": 884, "y1": 328, "x2": 1014, "y2": 465, "cls_id": 1, "conf": 0.8896484375}, {"x1": 371, "y1": 251, "x2": 439, "y2": 348, "cls_id": 1, "conf": 0.7939453125}, {"x1": 923, "y1": 426, "x2": 945, "y2": 451, "cls_id": 2, "conf": 0.57763671875}, {"x1": 878, "y1": 460, "x2": 901, "y2": 483, "cls_id": 2, "conf": 0.548828125}], "keypoints": [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]]}

training/DATASET.md

@@ -1,133 +0,0 @@
-# Detect-crime — Dataset Sources & Pipeline
-
-The king's training set is small and synthetic — that's why per-class recalls collapse on
-balaclava (3.4%), glove (6.4%), and spray paint (16.1%). The single biggest score lever is
-**more, real, in-domain data**, especially for the four catastrophic classes.
-
-## Target classes (target order — must match `class_names.txt`)
-
-| id | name | king recall | priority |
-|---:|---|---:|---|
-| 0 | balaclava | 0.034 | **critical** |
-| 1 | bat | 0.143 | high |
-| 2 | glove | 0.064 | **critical** |
-| 3 | graffiti | 0.321 | medium |
-| 4 | hoodie | 0.274 | medium (carries the IoU mean alone) |
-| 5 | spray paint | 0.161 | high |
-
-## Sources
-
-### Tier 1 — in-domain (gold)
-
-**`manak0/Detect-crime` `latestAnnotatedChallenge` API.** Same pattern as petrol-station: the
-console at `console.scorevision.io/api/v2/elements/manak0%2FDetect-crime?lookback_days=30`
-returns one `latestAnnotatedChallenge` record at a time. The record rotates as the runner
-emits fresh challenges, so polling at ~120s for several hours / days accumulates distinct
-in-domain frames *with the king's predictions attached*. Use those predictions as silver
-labels (king is 0.576, so they're noisy but in-domain).
-
-Use `poll_manako.py` to collect; use `build_dataset.py --manako` to integrate.
-
-### Tier 2 — public detection datasets (silver / gold mix)
-
-| class | dataset | notes |
-|---|---|---|
-| **balaclava** | Roboflow Universe `balaclava-detection` (multiple variants, ~1k–5k labeled imgs) | search "balaclava" on universe.roboflow.com; download YOLO format |
-| balaclava | MAFA — masked-face dataset (35k labeled faces, occluded) | ~3k images contain ski-mask / balaclava-style coverage; relabel as "balaclava" only when the mouth+nose+ears are all occluded |
-| **bat** | COCO 2017, class 35 "baseball bat" (~3.3k train + ~150 val instances) | already labeled YOLO-compatible after `pycocotools` re-export; map cls 35 -> our cls 1 |
-| bat | Open Images V7, class `Baseball bat` | larger, but noisier labels |
-| **glove** | Roboflow Universe `gloves-detection` / `ppe-detection` | ~5k–10k labeled; merges leather, latex, work gloves |
-| glove | COCO 2017 has no "glove" class; ImageNet n02883344 "ski glove" / n03775546 "boxing glove" — classification only, but useful for crops |
-| **graffiti** | Roboflow Universe `graffiti-detection` | several variants 500–3k imgs |
-| graffiti | STORM-IRMA Graffiti Dataset (Bardienus et al., academic release) | annotated graffiti tags + region bboxes |
-| **hoodie** | DeepFashion2 (item id `short_sleeve_top + long_sleeve_outwear`) | very large; relabel "hooded" subset only |
-| hoodie | Roboflow Universe `clothing-detection` | hooded sweatshirt subset |
-| **spray paint** | Roboflow Universe `spray-paint-can`/`graffiti-tools` | small, 200–800 imgs each |
-| spray paint | ImageNet n04270147 "spray can" | classification only — useful as copy-paste source crops |
-
-Roboflow Universe links rotate; the build script uses the public Roboflow REST API
-(`https://universe.roboflow.com/...`) and falls back to anonymous downloads for permissive-
-licensed datasets. **Always check each dataset's license before use** — if a dataset is
-non-redistributable, paste it into `--extra-dir` from a local copy instead.
-
-### Tier 3 — synthetic & augmentation
-
-- **Copy-paste augmentation** (built into ultralytics: `copy_paste=0.30`). Disproportionately
-  helps rare classes by pasting their crops into other training images.
-- **Mosaic + mixup** (also built-in): standard for YOLOv11.
-- **Photometric augmentation**: HSV jitter, brightness scaling. CCTV crime footage is
-  high-variance lighting (flashbang, IR night vision, daylight) — push `hsv_v` to 0.5.
-
-## Pipeline
-
-```
-sources                                              → KingLabeler (silver) ──┐
-  ├─ manako_pool/   (poll_manako.py)                   king's preds attached  │
-  ├─ roboflow/      (download.py per-class)                                   │
-  ├─ coco_bat/      (subset extractor)                                        ├─ build_dataset.py
-  ├─ openimages/    (`oidv6` CLI, optional)                                   │  → YOLO data.yaml
-  └─ extra/         (manual / verified frames)                                │
-                                                                              ┘
-```
-
-## Stage A vs Stage B
-
-- **Stage A — silver pretrain**: train on the **assembled silver corpus** (Roboflow + COCO
-  + king-labeled manako + ImageNet copy-paste source). Targets: `200 epochs` at `imgsz=1280`,
-  YOLOv11s. Use heavy augmentation. Save EMA. Expect val mAP@50 around 0.30–0.45 — well
-  above king's 0.14, but synthetic-eval-flattering.
-- **Stage B — clean fine-tune**: hand-verify (or LLM-verify with Qwen2-VL) ~300 manako frames
-  drawn from the latest 7-day window. This is the small-but-clean set that aligns most
-  closely with the live evaluator. Train `50 epochs` at `imgsz=1280` on Stage A weights with
-  reduced augmentation. Expect val mAP@50 to recede slightly (cleaner GT, harder eval) but
-  live-element score to *rise*, because we've shifted the distribution toward what's actually
-  scored on subnet.
-
-## Class balancing
-
-The king's set is heavily skewed toward hoodie (~half of GT boxes). Without rebalancing, our
-model would inherit that. Two interventions:
-
-1. **Weighted sampling** in dataloader (`sampler=` weighted by 1/sqrt(class_count)).
-2. **Copy-paste augmentation** sourced from a curated rare-class crop bank
-   (`crime_miner/training/_rare_crops/{balaclava, glove, spray_paint}/`). The build script
-   produces this bank as a byproduct.
-
-## What the build script writes
-
-```
-data/
-├── data.yaml                # ultralytics-style; names + train/val paths
-├── images/
-│   ├── train/{src}_{sha}.jpg  # one per source image
-│   └── val/...
-└── labels/
-    ├── train/{src}_{sha}.txt  # YOLO format: cls cx cy w h (normalized)
-    └── val/...
-```
-
-Manako frames are forced into `val` so the held-out set lives in the real evaluation domain.
-
-## Hard-class strategy: balaclava
-
-Balaclava is the lowest-recall class. Two failure modes from the king's perspective:
-
-1. **Box too small at 640 input** (king's stretch resize destroys 30 px head crops).
-   *Solved* by 1280 letterboxed input + multi-scale TTA. No retraining required.
-2. **Few training examples**. Mitigations:
-   - Pull MAFA (~3k masked-face images) and select the ~30% subset where the mask covers the
-     entire head (i.e., balaclava, not surgical mask).
-   - Roboflow `balaclava-detection-v3` has ~1.5k labeled.
-   - Synthesize via copy-paste of balaclava crops onto random `hoodie` GT boxes (we have many
-     hoodie images; pasting a balaclava crop on top is plausible-looking).
-
-Target: **5,000 balaclava-positive training images** going into Stage A.
-
-## Notes on label vocabulary
-
-Both the manak0 baseline and our model emit class names matching `class_names.txt` exactly.
-**Do not rename or reorder** — class id is what's compared on-subnet.
-
-If the live element uses SAM3 pseudo-GT (not real GT), SAM3 will be prompted with these
-exact strings as text labels (`scorevision/vlm_pipeline/sam3/detect_objects.py:36`). So the
-class names in `class_names.txt` *are* the SAM3 prompts.

training/build_dataset.py

@@ -1,860 +0,0 @@
-"""
-End-to-end dataset builder for Detect-crime.
-
-Sources (each toggleable via flags; pipeline tolerates missing sources):
-  1. manako latestAnnotatedChallenge (--manako): king-labeled in-domain frames.
-  2. Roboflow Universe (--roboflow PER_CLASS_SLUGS): per-class detection datasets.
-  3. COCO 2017 'baseball bat' subset (--coco-bat): high-quality bat boxes.
-  4. Extra dir (--extra-dir): pre-collected images you've curated locally.
-
-Each Roboflow / COCO source is mapped to our 6-class target via an explicit class table.
-Empty-label images are dropped unless --keep-empty (we keep them as negative-bg supervision
-during silver pretrain).
-
-Pipeline:
-  - Pull each enabled source into a working dir.
-  - Auto-relabel: re-emit YOLO label files with cls ids remapped to our target order.
-    For sources that come *unlabeled* (e.g. manako frames without preds), run the king's
-    ONNX over them to get silver labels.
-  - Manako frames go to val (real-domain eval); rest is 90/10 train/val.
-
-Usage:
-  python build_dataset.py \
-      --out ../data \
-      --king-onnx /root/turbovision_crime/king_models/Detect-crime/weights.onnx \
-      --manako --manako-polls 30 \
-      --roboflow balaclava=brainster/balaclava-detection-v3 \
-      --roboflow glove=ppe-detection/gloves-v1 \
-      --coco-bat /path/to/coco/annotations/instances_train2017.json /path/to/coco/train2017 \
-      --extra-dir ../data_extra --keep-empty
-"""
-
-from __future__ import annotations
-
-import argparse
-import hashlib
-import io
-import json
-import os
-import random
-import shutil
-import sys
-import time
-from pathlib import Path
-from urllib.parse import quote
-from urllib.request import Request, urlopen
-
-import numpy as np
-
-
-TARGET_CLASS_NAMES = ["balaclava", "bat", "glove", "graffiti", "hoodie", "spray paint"]
-CLASS_TO_ID = {name: i for i, name in enumerate(TARGET_CLASS_NAMES)}
-ELEMENT_ID = "manak0/Detect-crime"
-CONSOLE_API = "https://console.scorevision.io/api/v2"
-
-# The current king (`alfred8995/crime001`) emits class ids in their own order:
-#   0 balaclava, 1 hoodie, 2 glove, 3 bat, 4 spray paint, 5 graffiti
-# When we use their ONNX as a silver labeler OR ingest manako API predictions, we
-# remap from alfred's order -> our target (manak0-baseline) order.
-KING_CLASS_ORDER = ["balaclava", "hoodie", "glove", "bat", "spray paint", "graffiti"]
-KING_TO_TARGET_REMAP = np.array(
-    [TARGET_CLASS_NAMES.index(n) for n in KING_CLASS_ORDER], dtype=np.int32
-)
-# Sanity: KING_TO_TARGET_REMAP == [0, 4, 2, 1, 5, 3]
-
-
-# --------------------------------------------------------------------- utils
-
-def http_get(url: str, timeout: int = 60, retries: int = 4, headers: dict | None = None) -> bytes:
-    last = None
-    base_headers = {"User-Agent": "build_dataset/1.0"}
-    if headers:
-        base_headers.update(headers)
-    for i in range(retries):
-        try:
-            req = Request(url, headers=base_headers)
-            with urlopen(req, timeout=timeout) as r:
-                return r.read()
-        except Exception as e:
-            last = e
-            if i < retries - 1:
-                time.sleep(1.5 * (i + 1))
-    raise last  # type: ignore
-
-
-def sha16(b: bytes) -> str:
-    return hashlib.sha256(b).hexdigest()[:16]
-
-
-def image_size(buf: bytes) -> tuple[int, int]:
-    if buf[:8] == b"\x89PNG\r\n\x1a\n":
-        return int.from_bytes(buf[16:20], "big"), int.from_bytes(buf[20:24], "big")
-    if buf[:2] == b"\xff\xd8":
-        from PIL import Image
-        return Image.open(io.BytesIO(buf)).size
-    from PIL import Image
-    return Image.open(io.BytesIO(buf)).size
-
-
-# --------------------------------------------------------------------- sources
-
-def fetch_manako(work: Path, log, polls: int = 1, delay_s: int = 180) -> list[tuple[Path, str, dict | None]]:
-    """Pull rotating latestAnnotatedChallenge frames + king predictions."""
-    out: list[tuple[Path, str, dict | None]] = []
-    img_dir = work / "manako_images"
-    img_dir.mkdir(parents=True, exist_ok=True)
-    seen_paths: set[str] = set()
-
-    for attempt in range(max(1, polls)):
-        try:
-            raw = http_get(f"{CONSOLE_API}/elements/{quote(ELEMENT_ID, safe='')}?lookback_days=30")
-        except Exception as e:
-            log(f"[manako] API call failed (attempt {attempt + 1}/{polls}): {e}")
-            if attempt < polls - 1:
-                time.sleep(delay_s)
-            continue
-        elt = json.loads(raw)
-        lac = elt.get("latestAnnotatedChallenge")
-        if not lac:
-            log(f"[manako] no latestAnnotatedChallenge (attempt {attempt + 1}/{polls})")
-            if attempt < polls - 1:
-                time.sleep(delay_s)
-            continue
-
-        frames = (lac.get("response") or {}).get("frames") or []
-        preds = ((lac.get("response") or {}).get("predictions") or {}).get("frames") or []
-        preds_by_id = {p.get("frame_id"): p for p in preds}
-
-        new_count = 0
-        for f in frames:
-            url = f.get("url")
-            fid = f.get("frame_id")
-            if not url:
-                continue
-            try:
-                data = http_get(url)
-            except Exception as e:
-                log(f"[manako] image fetch failed {url}: {e}")
-                continue
-            digest = sha16(data)
-            path = img_dir / f"manako_ch{lac.get('challenge_id')}_f{fid}_{digest}.png"
-            if str(path) in seen_paths:
-                continue
-            seen_paths.add(str(path))
-            path.write_bytes(data)
-            out.append((path, "manako", preds_by_id.get(fid)))
-            new_count += 1
-
-        log(f"[manako] poll {attempt + 1}/{polls}: +{new_count} (total {len(out)}, "
-            f"challenge_id={lac.get('challenge_id')})")
-        if attempt < polls - 1:
-            time.sleep(delay_s)
-
-    log(f"[manako] pulled {len(out)} unique image(s) across {polls} poll(s)")
-    return out
-
-
-def fetch_roboflow(slug: str, target_class: str, work: Path, log,
-                   api_key: str | None = None) -> list[tuple[Path, str, list[str]]]:
-    """Download a Roboflow Universe dataset in YOLO format and remap to target class id.
-
-    `slug` is the workspace/project[/version] path on universe.roboflow.com.
-    Without API key, only public/free datasets work; many crime-relevant ones are public.
-    """
-    target_id = CLASS_TO_ID.get(target_class)
-    if target_id is None:
-        log(f"[roboflow] unknown target class: {target_class}")
-        return []
-
-    out_dir = work / "roboflow" / slug.replace("/", "_")
-    out_dir.mkdir(parents=True, exist_ok=True)
-    out: list[tuple[Path, str, list[str]]] = []
-
-    try:
-        from roboflow import Roboflow  # type: ignore
-    except ImportError:
-        log("[roboflow] roboflow SDK not installed; pip install roboflow")
-        return []
-
-    if api_key is None:
-        api_key = os.environ.get("ROBOFLOW_API_KEY", "")
-    if not api_key:
-        log(f"[roboflow] no ROBOFLOW_API_KEY set; skipping {slug}")
-        return []
-
-    parts = slug.split("/")
-    if len(parts) < 2:
-        log(f"[roboflow] malformed slug: {slug}")
-        return []
-    ws, project = parts[0], parts[1]
-    version = int(parts[2]) if len(parts) >= 3 and parts[2].isdigit() else 1
-
-    rf = Roboflow(api_key=api_key)
-    ds = rf.workspace(ws).project(project).version(version).download("yolov8", location=str(out_dir))
-    log(f"[roboflow] downloaded {slug} -> {ds.location}")
-
-    # Roboflow YOLO export: {ds.location}/{train,valid,test}/images/*, labels/*.txt with their own data.yaml
-    rf_yaml = Path(ds.location) / "data.yaml"
-    if not rf_yaml.exists():
-        log(f"[roboflow] no data.yaml in {ds.location}; skipping")
-        return []
-
-    # Parse names from data.yaml (we accept either yaml or simple key:value).
-    try:
-        import yaml
-        rf_cfg = yaml.safe_load(rf_yaml.read_text())
-    except Exception as e:
-        log(f"[roboflow] yaml parse failed for {rf_yaml}: {e}")
-        return []
-    rf_names = rf_cfg.get("names") or []
-    if isinstance(rf_names, dict):
-        rf_names = [rf_names[k] for k in sorted(rf_names.keys())]
-
-    for split in ("train", "valid", "test"):
-        img_dir = Path(ds.location) / split / "images"
-        lbl_dir = Path(ds.location) / split / "labels"
-        if not img_dir.exists() or not lbl_dir.exists():
-            continue
-        for img_path in img_dir.iterdir():
-            if not img_path.is_file():
-                continue
-            lbl_path = lbl_dir / (img_path.stem + ".txt")
-            if not lbl_path.exists():
-                continue
-            try:
-                rows = lbl_path.read_text().strip().splitlines()
-            except Exception:
-                continue
-            mapped: list[str] = []
-            for r in rows:
-                parts = r.split()
-                if len(parts) != 5:
-                    continue
-                src_cls = int(parts[0])
-                # Single-class roboflow datasets: any positive box -> target class
-                if len(rf_names) == 1 or src_cls >= len(rf_names):
-                    mapped.append(f"{target_id} {parts[1]} {parts[2]} {parts[3]} {parts[4]}")
-                    continue
-                # Multi-class: keep only boxes whose name matches the target (loose match).
-                src_name = str(rf_names[src_cls]).lower().replace("_", " ")
-                if target_class.lower() in src_name or src_name in target_class.lower():
-                    mapped.append(f"{target_id} {parts[1]} {parts[2]} {parts[3]} {parts[4]}")
-            if not mapped:
-                continue
-            out.append((img_path, f"roboflow_{slug.replace('/', '_')}", mapped))
-
-    log(f"[roboflow] {slug}: kept {len(out)} labeled images")
-    return out
-
-
-def fetch_coco_bat(coco_ann_json: Path, coco_img_dir: Path, log) -> list[tuple[Path, str, list[str]]]:
-    """Extract baseball-bat images from a COCO 2017 annotations file."""
-    target_id = CLASS_TO_ID["bat"]
-    out: list[tuple[Path, str, list[str]]] = []
-    if not coco_ann_json.exists() or not coco_img_dir.exists():
-        log(f"[coco-bat] paths missing: {coco_ann_json}, {coco_img_dir}")
-        return out
-
-    log(f"[coco-bat] reading {coco_ann_json} (this is large; ~30s)...")
-    with coco_ann_json.open() as f:
-        coco = json.load(f)
-
-    # COCO category id 39 = "baseball bat" in the 80-class set.
-    bat_ids = {c["id"] for c in coco.get("categories", []) if c.get("name", "").lower() == "baseball bat"}
-    if not bat_ids:
-        log("[coco-bat] no 'baseball bat' category found; aborting")
-        return out
-
-    # Group annotations by image_id.
-    images = {im["id"]: im for im in coco.get("images", [])}
-    by_img: dict[int, list[dict]] = {}
-    for ann in coco.get("annotations", []):
-        if ann.get("category_id") in bat_ids:
-            by_img.setdefault(ann["image_id"], []).append(ann)
-
-    for img_id, anns in by_img.items():
-        info = images.get(img_id)
-        if not info:
-            continue
-        fname = info["file_name"]
-        w, h = info["width"], info["height"]
-        img_path = coco_img_dir / fname
-        if not img_path.exists():
-            continue
-        rows = []
-        for a in anns:
-            x, y, bw, bh = a["bbox"]  # COCO xywh top-left
-            cx = (x + bw / 2.0) / w
-            cy = (y + bh / 2.0) / h
-            rows.append(f"{target_id} {cx:.6f} {cy:.6f} {bw / w:.6f} {bh / h:.6f}")
-        if rows:
-            out.append((img_path, "coco_bat", rows))
-    log(f"[coco-bat] kept {len(out)} bat-positive images")
-    return out
-
-
-def fetch_hf_parquet_coco(repo_id: str, filename: str, target_class: str,
-                          work: Path, log,
-                          limit: int = 0) -> list[tuple[bytes, str, list[str]]]:
-    """Pull a HuggingFace `datasets`-style parquet whose rows have
-    `image: {bytes}`, `width`, `height`, `objects: [{bbox: [x,y,w,h], ...}]`
-    (graffiti dataset format). Maps every box to TARGET_CLASS_NAMES[target_class].
-
-    Returns list of (image_bytes, source_tag, yolo_label_rows).
-    """
-    target_id = CLASS_TO_ID.get(target_class)
-    if target_id is None:
-        log(f"[hf-parquet-coco] unknown target class: {target_class}")
-        return []
-    try:
-        from huggingface_hub import hf_hub_download
-        import pyarrow.parquet as pq
-    except ImportError:
-        log("[hf-parquet-coco] missing deps; pip install huggingface_hub pyarrow")
-        return []
-    try:
-        local = hf_hub_download(
-            repo_id=repo_id, filename=filename,
-            repo_type="dataset", cache_dir=str(work / "_hf_cache"),
-        )
-    except Exception as e:
-        log(f"[hf-parquet-coco] download failed {repo_id}/{filename}: {e}")
-        return []
-    log(f"[hf-parquet-coco] downloaded {repo_id}/{filename}")
-    out: list[tuple[bytes, str, list[str]]] = []
-    table = pq.read_table(local, columns=["image", "width", "height", "objects"])
-    rows = table.to_pylist()
-    for row in rows:
-        img_dict = row.get("image") or {}
-        data = img_dict.get("bytes") if isinstance(img_dict, dict) else None
-        if not data:
-            continue
-        w = int(row.get("width") or 0)
-        h = int(row.get("height") or 0)
-        if w <= 0 or h <= 0:
-            try:
-                w, h = image_size(data)
-            except Exception:
-                continue
-        labels: list[str] = []
-        for obj in row.get("objects") or []:
-            bbox = obj.get("bbox") or []
-            if len(bbox) != 4:
-                continue
-            bx, by, bw, bh = (float(bbox[0]), float(bbox[1]), float(bbox[2]), float(bbox[3]))
-            if bw <= 1 or bh <= 1:
-                continue
-            cx = (bx + bw / 2.0) / w
-            cy = (by + bh / 2.0) / h
-            labels.append(f"{target_id} {cx:.6f} {cy:.6f} {bw / w:.6f} {bh / h:.6f}")
-        if labels:
-            out.append((data, f"hf_{repo_id.replace('/', '_')}", labels))
-        if limit and len(out) >= limit:
-            break
-    log(f"[hf-parquet-coco] {repo_id}: kept {len(out)} labeled images")
-    return out
-
-
-def fetch_hf_parquet_cls(repo_id: str, filename: str, target_class: str,
-                         work: Path, log,
-                         limit: int = 0) -> list[tuple[bytes, str, None]]:
-    """Pull a HuggingFace classification-synset parquet (rows have just
-    `image: {bytes}` with optional `label`). Boxes will be filled in later by
-    the king's ONNX silver labeler — this fn just extracts the image bytes.
-
-    `target_class` is purely a tag for downstream filtering; the king labels
-    every detection regardless.
-    """
-    try:
-        from huggingface_hub import hf_hub_download
-        import pyarrow.parquet as pq
-    except ImportError:
-        log("[hf-parquet-cls] missing deps; pip install huggingface_hub pyarrow")
-        return []
-    try:
-        local = hf_hub_download(
-            repo_id=repo_id, filename=filename,
-            repo_type="dataset", cache_dir=str(work / "_hf_cache"),
-        )
-    except Exception as e:
-        log(f"[hf-parquet-cls] download failed {repo_id}/{filename}: {e}")
-        return []
-    log(f"[hf-parquet-cls] downloaded {repo_id}/{filename}")
-    out: list[tuple[bytes, str, None]] = []
-    table = pq.read_table(local, columns=["image"])
-    for row in table.to_pylist():
-        img_dict = row.get("image") or {}
-        data = img_dict.get("bytes") if isinstance(img_dict, dict) else None
-        if not data:
-            continue
-        out.append((data, f"hf_{repo_id.replace('/', '_')}_{target_class}", None))
-        if limit and len(out) >= limit:
-            break
-    log(f"[hf-parquet-cls] {repo_id}: extracted {len(out)} images (king will label)")
-    return out
-
-
-def fetch_extra(extra_dir: Path | None, log) -> list[tuple[Path, str, None]]:
-    if extra_dir is None or not extra_dir.exists():
-        return []
-    out: list[tuple[Path, str, None]] = []
-    for ext in ("*.jpg", "*.jpeg", "*.png", "*.bmp", "*.webp"):
-        for p in extra_dir.rglob(ext):
-            out.append((p, "extra", None))
-    log(f"[extra] picked up {len(out)} images from {extra_dir}")
-    return out
-
-
-# --------------------------------------------------------------------- king labeler
-
-class KingLabeler:
-    """Run the king's ONNX over an unlabeled image to produce YOLO-format silver labels.
-
-    Mirrors alfred8995/crime001's preprocessing: letterbox to native input H/W, RGB / 255,
-    NCHW float32. Decodes both [N,6] / [1,N,6] (NMS-baked) and raw [1, C, N] formats with
-    proper letterbox-reverse back to original image coords.
-    """
-
-    def __init__(self, onnx_path: Path, conf_floor: float = 0.15, intra_threads: int = 0) -> None:
-        import onnxruntime as ort
-        sess_options = ort.SessionOptions()
-        sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
-        if intra_threads > 0:
-            sess_options.intra_op_num_threads = int(intra_threads)
-        try:
-            self.session = ort.InferenceSession(
-                str(onnx_path), sess_options=sess_options,
-                providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
-            )
-        except Exception:
-            self.session = ort.InferenceSession(
-                str(onnx_path), sess_options=sess_options,
-                providers=["CPUExecutionProvider"],
-            )
-        self.input_name = self.session.get_inputs()[0].name
-        self.output_names = [o.name for o in self.session.get_outputs()]
-        in_shape = self.session.get_inputs()[0].shape
-        self.in_h = int(in_shape[2]) if isinstance(in_shape[2], int) and in_shape[2] > 0 else 640
-        self.in_w = int(in_shape[3]) if isinstance(in_shape[3], int) and in_shape[3] > 0 else 640
-        self.conf_floor = float(conf_floor)
-        self.iou_thresh = 0.45
-
-    @staticmethod
-    def _letterbox(image: np.ndarray, new_size: tuple[int, int],
-                   color: tuple[int, int, int] = (114, 114, 114)
-                   ) -> tuple[np.ndarray, float, tuple[float, float]]:
-        import cv2
-        h, w = image.shape[:2]
-        new_w, new_h = new_size
-        ratio = min(new_w / w, new_h / h)
-        rw, rh = int(round(w * ratio)), int(round(h * ratio))
-        if (rw, rh) != (w, h):
-            interp = cv2.INTER_CUBIC if ratio > 1.0 else cv2.INTER_LINEAR
-            image = cv2.resize(image, (rw, rh), interpolation=interp)
-        dw = (new_w - rw) / 2.0
-        dh = (new_h - rh) / 2.0
-        top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1))
-        left, right = int(round(dw - 0.1)), int(round(dw + 0.1))
-        padded = cv2.copyMakeBorder(image, top, bottom, left, right,
-                                    borderType=cv2.BORDER_CONSTANT, value=color)
-        return padded, ratio, (dw, dh)
-
-    def label(self, image_bytes: bytes) -> list[str]:
-        import cv2
-        arr = np.frombuffer(image_bytes, dtype=np.uint8)
-        bgr = cv2.imdecode(arr, cv2.IMREAD_COLOR)
-        if bgr is None:
-            return []
-        orig_h, orig_w = bgr.shape[:2]
-
-        padded, ratio, (dw, dh) = self._letterbox(bgr, (self.in_w, self.in_h))
-        rgb = cv2.cvtColor(padded, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
-        x = np.transpose(rgb, (2, 0, 1))[None, ...]
-        x = np.ascontiguousarray(x, dtype=np.float32)
-
-        out = self.session.run(self.output_names, {self.input_name: x})[0]
-        return self._decode(out, ratio, dw, dh, orig_w, orig_h)
-
-    def _decode(self, output: np.ndarray, ratio: float, dw: float, dh: float,
-                orig_w: int, orig_h: int) -> list[str]:
-        """Decode + reverse-letterbox to original image coords."""
-        # NMS-baked: [N, 6] or [1, N, 6]
-        if output.ndim == 3 and output.shape[0] == 1 and output.shape[2] == 6:
-            preds = output[0]
-        elif output.ndim == 2 and output.shape[1] == 6:
-            preds = output
-        else:
-            return self._decode_raw(output, ratio, dw, dh, orig_w, orig_h)
-
-        boxes = preds[:, :4].astype(np.float32, copy=True)
-        scores = preds[:, 4].astype(np.float32)
-        cls = preds[:, 5].astype(np.int32)
-
-        keep = (scores >= self.conf_floor) & (boxes[:, 2] > boxes[:, 0]) & (boxes[:, 3] > boxes[:, 1])
-        boxes, scores, cls = boxes[keep], scores[keep], cls[keep]
-        if len(boxes) == 0:
-            return []
-
-        # reverse letterbox (input-letterboxed coords -> orig image coords)
-        boxes[:, [0, 2]] -= dw
-        boxes[:, [1, 3]] -= dh
-        boxes /= ratio
-        boxes[:, 0] = np.clip(boxes[:, 0], 0, orig_w - 1)
-        boxes[:, 1] = np.clip(boxes[:, 1], 0, orig_h - 1)
-        boxes[:, 2] = np.clip(boxes[:, 2], 0, orig_w - 1)
-        boxes[:, 3] = np.clip(boxes[:, 3], 0, orig_h - 1)
-
-        return self._to_yolo_rows(boxes, cls, orig_w, orig_h)
-
-    def _decode_raw(self, output: np.ndarray, ratio: float, dw: float, dh: float,
-                    orig_w: int, orig_h: int) -> list[str]:
-        """Fallback for raw [1, C, N] / [1, N, C] YOLO output (e.g. manak0 baseline)."""
-        if output.ndim != 3 or output.shape[0] != 1:
-            return []
-        p = output[0]
-        if p.shape[0] <= 16 and p.shape[1] > p.shape[0]:
-            p = p.T
-        if p.shape[1] < 5:
-            return []
-        boxes_xywh = p[:, :4].astype(np.float32)
-        cls_part = p[:, 4:].astype(np.float32)
-        cls = np.argmax(cls_part, axis=1).astype(np.int32)
-        scores = cls_part[np.arange(len(cls_part)), cls]
-        keep = scores >= self.conf_floor
-        if not np.any(keep):
-            return []
-        boxes_xywh = boxes_xywh[keep]
-        scores = scores[keep]
-        cls = cls[keep]
-        # xywh (input-letterboxed coords) -> xyxy (input coords)
-        xyxy = np.empty((len(boxes_xywh), 4), dtype=np.float32)
-        xyxy[:, 0] = boxes_xywh[:, 0] - boxes_xywh[:, 2] / 2.0
-        xyxy[:, 1] = boxes_xywh[:, 1] - boxes_xywh[:, 3] / 2.0
-        xyxy[:, 2] = boxes_xywh[:, 0] + boxes_xywh[:, 2] / 2.0
-        xyxy[:, 3] = boxes_xywh[:, 1] + boxes_xywh[:, 3] / 2.0
-        xyxy, scores, cls = self._nms(xyxy, scores, cls, self.iou_thresh)
-        if len(xyxy) == 0:
-            return []
-        # reverse letterbox
-        xyxy[:, [0, 2]] -= dw
-        xyxy[:, [1, 3]] -= dh
-        xyxy /= ratio
-        xyxy[:, 0] = np.clip(xyxy[:, 0], 0, orig_w - 1)
-        xyxy[:, 1] = np.clip(xyxy[:, 1], 0, orig_h - 1)
-        xyxy[:, 2] = np.clip(xyxy[:, 2], 0, orig_w - 1)
-        xyxy[:, 3] = np.clip(xyxy[:, 3], 0, orig_h - 1)
-        return self._to_yolo_rows(xyxy, cls, orig_w, orig_h)
-
-    @staticmethod
-    def _nms(boxes: np.ndarray, scores: np.ndarray, cls: np.ndarray,
-             iou_thresh: float) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
-        if len(boxes) == 0:
-            return boxes, scores, cls
-        order = np.argsort(-scores)
-        boxes = boxes[order]; scores = scores[order]; cls = cls[order]
-        keep = []
-        suppressed = np.zeros(len(boxes), dtype=bool)
-        for i in range(len(boxes)):
-            if suppressed[i]:
-                continue
-            keep.append(i)
-            xa = np.maximum(boxes[i, 0], boxes[i + 1:, 0])
-            ya = np.maximum(boxes[i, 1], boxes[i + 1:, 1])
-            xb = np.minimum(boxes[i, 2], boxes[i + 1:, 2])
-            yb = np.minimum(boxes[i, 3], boxes[i + 1:, 3])
-            inter = np.maximum(0.0, xb - xa) * np.maximum(0.0, yb - ya)
-            area_i = (boxes[i, 2] - boxes[i, 0]) * (boxes[i, 3] - boxes[i, 1])
-            area_r = (boxes[i + 1:, 2] - boxes[i + 1:, 0]) * (boxes[i + 1:, 3] - boxes[i + 1:, 1])
-            ious = inter / (area_i + area_r - inter + 1e-7)
-            for k, ok in enumerate(ious >= iou_thresh):
-                if ok:
-                    suppressed[i + 1 + k] = True
-        return boxes[keep], scores[keep], cls[keep]
-
-    @staticmethod
-    def _to_yolo_rows(boxes: np.ndarray, cls: np.ndarray, w: int, h: int) -> list[str]:
-        # Remap king's class order -> our target order before emitting.
-        rows = []
-        for (x1, y1, x2, y2), c in zip(boxes, cls):
-            if c < 0 or c >= len(KING_TO_TARGET_REMAP):
-                continue
-            tgt = int(KING_TO_TARGET_REMAP[int(c)])
-            bw = float(x2 - x1)
-            bh = float(y2 - y1)
-            if bw <= 1 or bh <= 1:
-                continue
-            cx = (float(x1) + float(x2)) / 2.0 / w
-            cy = (float(y1) + float(y2)) / 2.0 / h
-            rows.append(f"{tgt} {cx:.6f} {cy:.6f} {bw / w:.6f} {bh / h:.6f}")
-        return rows
-
-
-def king_preds_to_yolo(preds: dict, img_w: int, img_h: int) -> list[str]:
-    """Manako preds use absolute pixel xyxy + cls_id (in alfred8995's class order).
-    Remap to our target order before emitting YOLO rows.
-    """
-    rows: list[str] = []
-    for box in preds.get("boxes") or []:
-        try:
-            x1, y1, x2, y2 = float(box["x1"]), float(box["y1"]), float(box["x2"]), float(box["y2"])
-            c = int(box["cls_id"])
-        except Exception:
-            continue
-        if c < 0 or c >= len(KING_TO_TARGET_REMAP):
-            continue
-        tgt = int(KING_TO_TARGET_REMAP[c])
-        bw = x2 - x1
-        bh = y2 - y1
-        if bw <= 1 or bh <= 1:
-            continue
-        cx = (x1 + x2) / 2.0 / img_w
-        cy = (y1 + y2) / 2.0 / img_h
-        rows.append(f"{tgt} {cx:.6f} {cy:.6f} {bw / img_w:.6f} {bh / img_h:.6f}")
-    return rows
-
-
-# --------------------------------------------------------------------- writer
-
-def write_yolo_pair(out_root: Path, split: str, name: str, image_bytes: bytes,
-                    label_lines: list[str], ext: str) -> None:
-    img_dir = out_root / "images" / split
-    lbl_dir = out_root / "labels" / split
-    img_dir.mkdir(parents=True, exist_ok=True)
-    lbl_dir.mkdir(parents=True, exist_ok=True)
-    (img_dir / f"{name}.{ext}").write_bytes(image_bytes)
-    (lbl_dir / f"{name}.txt").write_text("\n".join(label_lines), encoding="utf-8")
-
-
-def write_data_yaml(out_root: Path) -> Path:
-    data_yaml = out_root / "data.yaml"
-    data_yaml.write_text(
-        "# YOLO dataset for manak0/Detect-crime\n"
-        f"path: {out_root.resolve()}\n"
-        "train: images/train\n"
-        "val: images/val\n"
-        "names:\n"
-        + "".join(f"  {i}: {n}\n" for i, n in enumerate(TARGET_CLASS_NAMES)),
-        encoding="utf-8",
-    )
-    return data_yaml
-
-
-# --------------------------------------------------------------------- main
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--out", required=True, help="dataset output root (YOLO format)")
-    ap.add_argument("--king-onnx", required=True,
-                    help="path to king's ONNX (used as silver labeler for unlabeled frames)")
-    ap.add_argument("--manako", action="store_true", help="poll manako latestAnnotatedChallenge")
-    ap.add_argument("--manako-polls", type=int, default=1)
-    ap.add_argument("--manako-poll-delay", type=int, default=180)
-    ap.add_argument("--roboflow", action="append", default=[],
-                    help="repeatable: TARGETCLASS=workspace/project[/version]; "
-                         "e.g. balaclava=brainster/balaclava-detection-v3")
-    ap.add_argument("--coco-bat", nargs=2, metavar=("ANN_JSON", "IMG_DIR"),
-                    help="paths to COCO instances_train2017.json and train2017/")
-    ap.add_argument("--extra-dir", type=Path, default=None,
-                    help="optional dir of pre-collected images (auto-labeled by king)")
-    ap.add_argument("--hf-parquet-coco", action="append", default=[],
-                    help="repeatable: TARGETCLASS=repo_id::filename (COCO-bbox parquet); "
-                         "e.g. graffiti=junaidfayazlone/GraffitiDetection::Train-00000-of-00001-graffiti.parquet")
-    ap.add_argument("--hf-parquet-cls", action="append", default=[],
-                    help="repeatable: TARGETCLASS=repo_id::filename (classification parquet, "
-                         "king-labeled); e.g. balaclava=mlnomad/imnet1k_ski_mask::data/train-00000-of-00001.parquet")
-    ap.add_argument("--hf-limit", type=int, default=0,
-                    help="cap rows pulled from each HF parquet (0 = all)")
-    ap.add_argument("--min-conf", type=float, default=0.15,
-                    help="confidence floor for the silver labeler (lower = more labels, more noise)")
-    ap.add_argument("--workdir", type=Path, default=Path("/tmp/crime_dataset_work"))
-    ap.add_argument("--val-frac", type=float, default=0.10)
-    ap.add_argument("--seed", type=int, default=42)
-    ap.add_argument("--limit", type=int, default=0,
-                    help="cap images per source (0 = no cap)")
-    ap.add_argument("--intra-threads", type=int, default=0)
-    ap.add_argument("--keep-empty", action="store_true",
-                    help="keep images with no detected boxes (negative-bg supervision)")
-    args = ap.parse_args()
-
-    out_root = Path(args.out).resolve()
-    work = args.workdir.resolve()
-    work.mkdir(parents=True, exist_ok=True)
-
-    def log(msg: str) -> None:
-        print(msg, file=sys.stderr, flush=True)
-
-    # Sources collected as (path_or_bytes, src_tag, labels_or_preds_or_None).
-    # labels can be: list[str] YOLO rows (already mapped), dict (king preds), or None (needs labeling).
-    sources: list[tuple[Path, str, list[str] | dict | None]] = []
-
-    if args.manako:
-        for img_path, src, preds in fetch_manako(work, log,
-                                                  polls=args.manako_polls,
-                                                  delay_s=args.manako_poll_delay):
-            sources.append((img_path, src, preds))
-
-    for spec in args.roboflow:
-        if "=" not in spec:
-            log(f"[roboflow] malformed --roboflow {spec}; expected TARGETCLASS=slug")
-            continue
-        target, slug = spec.split("=", 1)
-        for img_path, src, labels in fetch_roboflow(slug.strip(), target.strip(), work, log):
-            sources.append((img_path, src, labels))
-
-    if args.coco_bat:
-        coco_ann, coco_img = Path(args.coco_bat[0]), Path(args.coco_bat[1])
-        for img_path, src, labels in fetch_coco_bat(coco_ann, coco_img, log):
-            sources.append((img_path, src, labels))
-
-    # HF parquet sources land as (bytes, src, labels). Persist to disk under work
-    # so the rest of the pipeline (which expects file paths) can pick them up.
-    hf_dir = work / "hf_images"
-    hf_dir.mkdir(parents=True, exist_ok=True)
-
-    for spec in args.hf_parquet_coco:
-        if "=" not in spec or "::" not in spec:
-            log(f"[hf-parquet-coco] malformed {spec}; expected TARGETCLASS=repo::filename")
-            continue
-        target, rest = spec.split("=", 1)
-        repo_id, filename = rest.split("::", 1)
-        for data, src, labels in fetch_hf_parquet_coco(repo_id.strip(), filename.strip(),
-                                                       target.strip(), work, log,
-                                                       limit=args.hf_limit):
-            digest = sha16(data)
-            ext = "jpg" if data[:2] == b"\xff\xd8" else "png"
-            p = hf_dir / f"{src}_{digest}.{ext}"
-            if not p.exists():
-                p.write_bytes(data)
-            sources.append((p, src, labels))
-
-    for spec in args.hf_parquet_cls:
-        if "=" not in spec or "::" not in spec:
-            log(f"[hf-parquet-cls] malformed {spec}; expected TARGETCLASS=repo::filename")
-            continue
-        target, rest = spec.split("=", 1)
-        repo_id, filename = rest.split("::", 1)
-        for data, src, _ in fetch_hf_parquet_cls(repo_id.strip(), filename.strip(),
-                                                  target.strip(), work, log,
-                                                  limit=args.hf_limit):
-            digest = sha16(data)
-            ext = "jpg" if data[:2] == b"\xff\xd8" else "png"
-            p = hf_dir / f"{src}_{digest}.{ext}"
-            if not p.exists():
-                p.write_bytes(data)
-            sources.append((p, src, None))
-
-    if args.extra_dir:
-        for img_path, src, _ in fetch_extra(args.extra_dir, log):
-            sources.append((img_path, src, None))
-
-    if not sources:
-        log("[!] no sources; pass at least --manako / --roboflow / --coco-bat / --extra-dir")
-        return 1
-
-    if args.limit:
-        sources = sources[: args.limit * 8]  # rough cap; we'll dedupe below
-    log(f"[build] total candidate sources: {len(sources)}")
-
-    # Auto-labeler for sources that come unlabeled.
-    labeler = KingLabeler(Path(args.king_onnx), conf_floor=args.min_conf,
-                          intra_threads=args.intra_threads)
-    log(f"[label] king onnx input={labeler.in_w}x{labeler.in_h} "
-        f"providers={labeler.session.get_providers()}")
-
-    samples: list[tuple[str, bytes, list[str], str]] = []
-    seen_hashes: set[str] = set()
-    n_labeled = 0
-    for i, (path, src, payload) in enumerate(sources):
-        try:
-            data = path.read_bytes()
-        except Exception as e:
-            log(f"[skip] read failed {path}: {e}")
-            continue
-        digest = sha16(data)
-        if digest in seen_hashes:
-            continue
-        seen_hashes.add(digest)
-        ext = "jpg" if data[:2] == b"\xff\xd8" else "png"
-
-        labels: list[str]
-        if isinstance(payload, list):
-            labels = payload                   # already remapped YOLO rows
-        elif isinstance(payload, dict):
-            try:
-                w, h = image_size(data)
-                labels = king_preds_to_yolo(payload, w, h)
-            except Exception as e:
-                log(f"[label] manako preds parse failed {path}: {e}; falling back to ONNX")
-                labels = labeler.label(data)
-        else:
-            try:
-                labels = labeler.label(data)
-            except Exception as e:
-                log(f"[label] ONNX labeling failed {path}: {e}")
-                continue
-
-        if not labels and not args.keep_empty:
-            continue
-        if labels:
-            n_labeled += 1
-        name = f"{src}_{digest}"
-        samples.append((name, data, labels, src))
-        if (i + 1) % 200 == 0:
-            log(f"  labeled {i + 1}/{len(sources)} (kept {len(samples)})")
-
-    log(f"[build] labeled {n_labeled}/{len(sources)} unique images (others were empty)")
-
-    if not samples:
-        log("[!] zero labeled samples; lower --min-conf and retry")
-        return 1
-
-    rng = random.Random(args.seed)
-    manako_samples = [s for s in samples if s[3] == "manako"]
-    other_samples = [s for s in samples if s[3] != "manako"]
-    rng.shuffle(other_samples)
-
-    n_val_target = max(1, int(round(len(samples) * args.val_frac)))
-    val: list = list(manako_samples)
-    if len(val) < n_val_target:
-        val.extend(other_samples[: n_val_target - len(val)])
-        train = other_samples[n_val_target - len(manako_samples):]
-    else:
-        train = other_samples
-    log(f"[split] train={len(train)}  val={len(val)}  (manako-in-val={len(manako_samples)})")
-
-    if out_root.exists():
-        for sub in ("images/train", "images/val", "labels/train", "labels/val"):
-            d = out_root / sub
-            if d.exists():
-                shutil.rmtree(d)
-    out_root.mkdir(parents=True, exist_ok=True)
-
-    for name, data, labels, _src in train:
-        ext = "jpg" if data[:2] == b"\xff\xd8" else "png"
-        write_yolo_pair(out_root, "train", name, data, labels, ext)
-    for name, data, labels, _src in val:
-        ext = "jpg" if data[:2] == b"\xff\xd8" else "png"
-        write_yolo_pair(out_root, "val", name, data, labels, ext)
-
-    yaml_path = write_data_yaml(out_root)
-    log(f"[done] wrote dataset to {out_root}")
-    log(f"       data.yaml: {yaml_path}")
-    log(f"       train images: {len(train)}, val images: {len(val)}")
-
-    cls_train = [0] * len(TARGET_CLASS_NAMES)
-    cls_val = [0] * len(TARGET_CLASS_NAMES)
-    for _n, _b, labs, _s in train:
-        for line in labs:
-            cls_train[int(line.split()[0])] += 1
-    for _n, _b, labs, _s in val:
-        for line in labs:
-            cls_val[int(line.split()[0])] += 1
-    log("[stats] per-class instance counts")
-    for i, n in enumerate(TARGET_CLASS_NAMES):
-        log(f"        {i} {n:13s}  train={cls_train[i]:6d}  val={cls_val[i]:6d}")
-
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

training/eval_mine_vs_king.py

@@ -1,204 +0,0 @@
-#!/usr/bin/env python3
-"""
-Val-split mAP: your exported ONNX vs current king ONNX (alfred8995/crime001).
-
-King ONNX uses semantic class ids in a different order than `scratch/data/data.yaml`.
-Remap king outputs to dataset indices before TP/FP matching:
-
-    king id  king name      -> yaml id / name
-       0     balaclava      -> 0 balaclava
-       1     hoodie         -> 4 hoodie
-       2     glove          -> 2 glove
-       3     wooden bat     -> 1 bat
-       4     spray can      -> 5 spray paint
-       5     graffiti       -> 3 graffiti
-
-Usage:
-  python3 eval_mine_vs_king.py --data /root/turbovision_crime/scratch/data/data.yaml \
-      --mine /root/turbovision_crime/scratch/crime_miner/weights.onnx \
-      --king /root/turbovision_crime/king_models/alfred8995_crime001/weights.onnx
-"""
-
-from __future__ import annotations
-
-import argparse
-import csv
-from datetime import datetime, timezone
-from pathlib import Path
-
-import torch
-from ultralytics import YOLO
-from ultralytics.models.yolo.detect import DetectionValidator
-
-# king cls -> dataset cls (see module docstring)
-KING_TO_DATASET = torch.tensor([0, 4, 2, 1, 5, 3])
-
-
-class KingRemappedValidator(DetectionValidator):
-    def postprocess(self, preds):  # noqa: ANN001
-        outs = super().postprocess(preds)
-        for d in outs:
-            if d["cls"].numel() == 0:
-                continue
-            ci = d["cls"].long().clamp(0, 5).view(-1)
-            mapped = KING_TO_DATASET[ci.cpu()].to(device=d["cls"].device, dtype=d["cls"].dtype)
-            d["cls"] = mapped
-        return outs
-
-    def init_metrics(self, model):  # noqa: ANN001
-        super().init_metrics(model)
-        # Confusion-matrix / prints use canonical names from dataset
-        names = {
-            0: "balaclava",
-            1: "bat",
-            2: "glove",
-            3: "graffiti",
-            4: "hoodie",
-            5: "spray paint",
-        }
-        self.names = names
-        self.nc = len(names)
-        self.metrics.names = names
-        try:
-            from ultralytics.utils.metrics import ConfusionMatrix
-
-            self.confusion_matrix = ConfusionMatrix(names=names, save_matches=self.args.plots and self.args.visualize)
-        except Exception:
-            pass
-
-
-def run_val(weights: Path, data: Path, imgsz: int, batch: int, device: str, validator_cls: type):
-    model = YOLO(str(weights))
-    return model.val(
-        data=str(data),
-        imgsz=imgsz,
-        batch=batch,
-        device=device,
-        plots=False,
-        verbose=False,
-        validator=validator_cls,
-    )
-
-
-def maybe_log_ops_row(
-    csv_path: Path,
-    challenge_id: str,
-    scene: str,
-    mine: Path,
-    king: Path,
-    m_mine,
-    m_king,
-) -> None:
-    fields = [
-        "timestamp_utc",
-        "challenge_id",
-        "scene",
-        "lighting",
-        "camera_angle",
-        "motion_blur",
-        "crowd_level",
-        "my_model_id",
-        "my_score",
-        "king_model_id",
-        "king_score",
-        "delta_score",
-        "my_pred_count",
-        "king_pred_count",
-        "my_latency_p50_ms",
-        "my_latency_p95_ms",
-        "my_latency_p99_ms",
-        "dropped_frames",
-        "timeout_count",
-        "balaclava_hits",
-        "bat_hits",
-        "glove_hits",
-        "graffiti_hits",
-        "hoodie_hits",
-        "spray_paint_hits",
-        "tag_list",
-        "data_source_mix",
-        "notes",
-    ]
-    row = {
-        "timestamp_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
-        "challenge_id": challenge_id,
-        "scene": scene,
-        "lighting": "",
-        "camera_angle": "",
-        "motion_blur": "",
-        "crowd_level": "",
-        "my_model_id": mine.name,
-        "my_score": f"{m_mine.box.map50:.6f}",
-        "king_model_id": king.name,
-        "king_score": f"{m_king.box.map50:.6f}",
-        "delta_score": f"{(m_mine.box.map50 - m_king.box.map50):.6f}",
-        "my_pred_count": "",
-        "king_pred_count": "",
-        "my_latency_p50_ms": f"{m_mine.speed.get('inference', 0.0):.3f}",
-        "my_latency_p95_ms": "",
-        "my_latency_p99_ms": "",
-        "dropped_frames": "",
-        "timeout_count": "",
-        "balaclava_hits": "",
-        "bat_hits": "",
-        "glove_hits": "",
-        "graffiti_hits": "",
-        "hoodie_hits": "",
-        "spray_paint_hits": "",
-        "tag_list": "offline_val_split",
-        "data_source_mix": "",
-        "notes": "auto-log from eval_mine_vs_king.py; scores are mAP50 on local val split",
-    }
-    csv_path.parent.mkdir(parents=True, exist_ok=True)
-    new_file = not csv_path.exists()
-    with csv_path.open("a", newline="") as f:
-        writer = csv.DictWriter(f, fieldnames=fields)
-        if new_file:
-            writer.writeheader()
-        writer.writerow(row)
-    print(f"[ops-log] appended row -> {csv_path}", flush=True)
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--data", required=True)
-    ap.add_argument("--mine", required=True)
-    ap.add_argument("--king", required=True)
-    ap.add_argument("--imgsz", type=int, default=1280)
-    ap.add_argument("--batch", type=int, default=8)
-    ap.add_argument("--device", default="0")
-    ap.add_argument("--log-csv", default="", help="optional path to detect_crime_ops challenge_log.csv")
-    ap.add_argument("--challenge-id", default="", help="optional challenge/eval id for ops logging")
-    ap.add_argument("--scene", default="offline_val_split", help="scene tag used when --log-csv is set")
-    args = ap.parse_args()
-
-    data = Path(args.data)
-    mine = Path(args.mine)
-    king = Path(args.king)
-
-    print("=== Yours (exported miner ONNX) ===", flush=True)
-    m_mine = run_val(mine, data, args.imgsz, args.batch, args.device, DetectionValidator)
-    print(f"images={m_mine.speed.get('images', '?')}  "
-          f"mAP50={m_mine.box.map50:.5f}  mAP50-95={m_mine.box.map:.5f}  "
-          f"P={m_mine.box.mp:.5f}  R={m_mine.box.mr:.5f}", flush=True)
-
-    print("\n=== King (alfred8995/crime001, cls remapped to data.yaml order) ===", flush=True)
-    m_king = run_val(king, data, args.imgsz, args.batch, args.device, KingRemappedValidator)
-    print(f"images={m_king.speed.get('images', '?')}  "
-          f"mAP50={m_king.box.map50:.5f}  mAP50-95={m_king.box.map:.5f}  "
-          f"P={m_king.box.mp:.5f}  R={m_king.box.mr:.5f}", flush=True)
-
-    print("\n=== Delta (yours − king) on same val split ===", flush=True)
-    print(f"d_mAP50     = {m_mine.box.map50 - m_king.box.map50:+.5f}", flush=True)
-    print(f"d_mAP50-95  = {m_mine.box.map - m_king.box.map:+.5f}", flush=True)
-    print("\nInterpretation:", flush=True)
-    print("- Same boxes (IoU/geometric inference) regardless of remap; remap only aligns class IDs for AP.", flush=True)
-    print("- This is YOUR silver-label val set, not the live Manako leaderboard.", flush=True)
-    if args.log_csv:
-        cid = args.challenge_id or f"offline_val_{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}"
-        maybe_log_ops_row(Path(args.log_csv), cid, args.scene, mine, king, m_mine, m_king)
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

training/export_onnx.py

@@ -1,65 +0,0 @@
-"""
-Export a trained YOLOv11 .pt to ONNX with NMS baked in -> output shape [1, max_det, 6].
-
-The default chute miner.py decode path expects [1, N, 6] (NMS-baked). The king's own
-weights.onnx is raw-YOLO [1, 10, 8400] without NMS — our exported model does NOT match
-that, but our miner.py handles both formats. NMS-baked is preferred because:
-  - chute startup is faster (less graph work),
-  - the decode path is simpler (no fallback to raw YOLO numpy NMS),
-  - p95 latency is more stable (no Python-NMS spike on busy frames).
-
-Usage:
-  python export_onnx.py --weights runs/detect/crime_b/weights/best.pt --imgsz 1280 \
-                        --out ../weights.onnx
-"""
-
-from __future__ import annotations
-
-import argparse
-import shutil
-from pathlib import Path
-
-from ultralytics import YOLO
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--weights", required=True)
-    ap.add_argument("--imgsz", type=int, default=1280)
-    ap.add_argument("--out", default=None,
-                    help="destination path for weights.onnx (defaults next to source .pt)")
-    ap.add_argument("--max-det", type=int, default=300)
-    ap.add_argument("--opset", type=int, default=17)
-    ap.add_argument("--half", action="store_true",
-                    help="export FP16 (smaller, faster on CUDA, marginal accuracy delta)")
-    args = ap.parse_args()
-
-    weights = Path(args.weights).resolve()
-    model = YOLO(str(weights))
-
-    onnx_path = model.export(
-        format="onnx",
-        imgsz=args.imgsz,
-        nms=True,
-        max_det=args.max_det,
-        opset=args.opset,
-        dynamic=False,
-        half=args.half,
-        simplify=True,
-        agnostic_nms=False,   # class-aware NMS (must match miner.py expectations)
-        device="cpu",
-    )
-
-    src = Path(onnx_path)
-    if args.out:
-        dst = Path(args.out).resolve()
-        dst.parent.mkdir(parents=True, exist_ok=True)
-        shutil.move(str(src), str(dst))
-        print(f"[export] {src} -> {dst}")
-    else:
-        print(f"[export] {src}")
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

training/milestones_watch.py

@@ -1,104 +0,0 @@
-#!/usr/bin/env python3
-"""Poll Ultralytics results.csv and append one summary line per N-epoch milestone."""
-from __future__ import annotations
-
-import argparse
-import csv
-import time
-from datetime import datetime, timezone
-from pathlib import Path
-
-
-def fmt_row(row: dict) -> str:
-    ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
-    ep = int(float(row["epoch"]))
-    return (
-        f"{ts} | epoch={ep} | "
-        f"mAP50={float(row['metrics/mAP50(B)']):.4f} mAP={float(row['metrics/mAP50-95(B)']):.4f} | "
-        f"P={float(row['metrics/precision(B)']):.4f} R={float(row['metrics/recall(B)']):.4f} | "
-        f"train_box={float(row['train/box_loss']):.4f} train_cls={float(row['train/cls_loss']):.4f}"
-    )
-
-
-def recover_done(out_path: Path) -> set[int]:
-    done: set[int] = set()
-    if not out_path.exists():
-        return set()
-    for line in out_path.read_text().splitlines():
-        if "| epoch=" not in line:
-            continue
-        try:
-            part = line.split("| epoch=")[1].split()[0]
-            done.add(int(part))
-        except (IndexError, ValueError):
-            continue
-    return done
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--csv", required=True, help="path to results.csv (created by Ultralytics)")
-    ap.add_argument("--every", type=int, default=10)
-    ap.add_argument("--max-epoch", type=int, default=200)
-    ap.add_argument("--out", default=None, help="append logs here (default: <run-dir>/milestones_every_N.log)")
-    ap.add_argument("--poll", type=float, default=15.0, help="seconds between reads")
-    ap.add_argument("--idle-exit-sec", type=float, default=180.0,
-                    help="exit if results.csv is untouched this long and milestones caught up")
-    args = ap.parse_args()
-
-    csv_path = Path(args.csv)
-    run_dir = csv_path.parent
-    out_path = Path(args.out) if args.out else run_dir / f"milestones_every_{args.every}.log"
-    out_path.parent.mkdir(parents=True, exist_ok=True)
-    if not out_path.exists():
-        ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
-        out_path.write_text(
-            f"{ts} | milestones_watch | status=started next_report=epoch_{args.every} csv={csv_path}\n"
-        )
-        print(f"[milestones_watch] created {out_path} (metric lines appear every {args.every} epochs)", flush=True)
-    done = recover_done(out_path)
-
-    while True:
-        if not csv_path.exists():
-            time.sleep(args.poll)
-            continue
-        try:
-            with csv_path.open(newline="") as f:
-                rows = list(csv.DictReader(f))
-        except Exception:
-            time.sleep(args.poll)
-            continue
-        if not rows:
-            time.sleep(args.poll)
-            continue
-
-        last_ep = int(float(rows[-1]["epoch"]))
-        for m in range(args.every, min(last_ep, args.max_epoch) + 1, args.every):
-            if m in done:
-                continue
-            match = [r for r in rows if int(float(r["epoch"])) == m]
-            if not match:
-                continue
-            line = fmt_row(match[0])
-            out_path.parent.mkdir(parents=True, exist_ok=True)
-            with out_path.open("a") as o:
-                o.write(line + "\n")
-            print(line, flush=True)
-            done.add(m)
-
-        mtime = csv_path.stat().st_mtime
-        idle = time.time() - mtime
-        max_reportable = (last_ep // args.every) * args.every
-        need = list(range(args.every, max_reportable + 1, args.every))
-        caught_up = all(m in done for m in need)
-        if idle >= args.idle_exit_sec and caught_up and last_ep >= 1:
-            print(f"milestones_watch: idle {idle:.0f}s and milestones caught up → exit", flush=True)
-            break
-
-        time.sleep(args.poll)
-
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

training/poll_manako.py

@@ -1,132 +0,0 @@
-"""
-Continuous poller for in-domain frames from the manako/scoredata API for Detect-crime.
-
-The console.scorevision.io endpoint surfaces ONE `latestAnnotatedChallenge` per element.
-As the runner produces new challenges this single record rotates. By polling every couple
-of minutes we accumulate distinct in-domain frames + the king's predictions for each.
-
-Use this as a long-running collector alongside `build_dataset.py`. Run it for a few hours
-or days, then re-run `build_dataset.py --extra-dir <out>/images` (or `--manako`) to fold
-the new frames into the YOLO dataset.
-
-Usage:
-  python poll_manako.py --out ./manako_pool --interval 120 --hours 8
-  python poll_manako.py --out ./manako_pool --interval 120 --forever  # until killed
-
-Output structure (each unique frame = one image + one preds JSON):
-  manako_pool/
-    images/manako_ch{cid}_f{fid}_{sha16}.png
-    preds/ manako_ch{cid}_f{fid}_{sha16}.json
-"""
-
-from __future__ import annotations
-
-import argparse
-import hashlib
-import json
-import sys
-import time
-from pathlib import Path
-from urllib.parse import quote
-from urllib.request import Request, urlopen
-
-API = "https://console.scorevision.io/api/v2"
-ELEMENT_ID = "manak0/Detect-crime"
-
-
-def http_get(url: str, timeout: int = 30, retries: int = 3) -> bytes:
-    last = None
-    for i in range(retries):
-        try:
-            req = Request(url, headers={"User-Agent": "poll_manako/1.0"})
-            with urlopen(req, timeout=timeout) as r:
-                return r.read()
-        except Exception as e:
-            last = e
-            time.sleep(1.0 * (i + 1))
-    raise last  # type: ignore
-
-
-def sha16(b: bytes) -> str:
-    return hashlib.sha256(b).hexdigest()[:16]
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--out", required=True, help="output dir for accumulated frames")
-    ap.add_argument("--interval", type=int, default=120, help="seconds between polls")
-    ap.add_argument("--hours", type=float, default=4.0, help="duration to run (ignored with --forever)")
-    ap.add_argument("--forever", action="store_true", help="run until killed")
-    args = ap.parse_args()
-
-    out = Path(args.out).resolve()
-    img_dir = out / "images"
-    pred_dir = out / "preds"
-    img_dir.mkdir(parents=True, exist_ok=True)
-    pred_dir.mkdir(parents=True, exist_ok=True)
-
-    deadline = time.time() + args.hours * 3600 if not args.forever else float("inf")
-    seen: set[str] = {p.stem.rsplit("_", 1)[-1] for p in img_dir.glob("manako_*.png")}
-    n_polls = 0
-    n_new = 0
-    print(f"[poll] starting; existing frames in pool: {len(seen)}", file=sys.stderr)
-
-    while time.time() < deadline:
-        n_polls += 1
-        try:
-            raw = http_get(f"{API}/elements/{quote(ELEMENT_ID, safe='')}?lookback_days=7")
-        except Exception as e:
-            print(f"[poll {n_polls}] API error: {e}", file=sys.stderr)
-            time.sleep(args.interval)
-            continue
-
-        elt = json.loads(raw)
-        lac = elt.get("latestAnnotatedChallenge")
-        if not lac:
-            print(f"[poll {n_polls}] no latestAnnotatedChallenge", file=sys.stderr)
-            time.sleep(args.interval)
-            continue
-
-        challenge_id = lac.get("challenge_id")
-        frames = (lac.get("response") or {}).get("frames") or []
-        preds = ((lac.get("response") or {}).get("predictions") or {}).get("frames") or []
-        preds_by_id = {p.get("frame_id"): p for p in preds}
-
-        added = 0
-        for f in frames:
-            url = f.get("url")
-            fid = f.get("frame_id")
-            if not url:
-                continue
-            try:
-                data = http_get(url)
-            except Exception as e:
-                print(f"[poll] image fetch failed {url}: {e}", file=sys.stderr)
-                continue
-            digest = sha16(data)
-            if digest in seen:
-                continue
-            seen.add(digest)
-            (img_dir / f"manako_ch{challenge_id}_f{fid}_{digest}.png").write_bytes(data)
-            preds_obj = preds_by_id.get(fid)
-            if preds_obj is not None:
-                (pred_dir / f"manako_ch{challenge_id}_f{fid}_{digest}.json").write_text(
-                    json.dumps(preds_obj), encoding="utf-8"
-                )
-            added += 1
-            n_new += 1
-
-        print(f"[poll {n_polls}] cid={challenge_id} frames={len(frames)} new={added} "
-              f"total_unique={len(seen)} (cumulative new this run: {n_new})", file=sys.stderr)
-
-        if time.time() >= deadline:
-            break
-        time.sleep(args.interval)
-
-    print(f"[poll] done. polls={n_polls}, new frames added this run: {n_new}, "
-          f"pool size: {len(seen)}", file=sys.stderr)
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

training/requirements.txt

@@ -1,12 +0,0 @@
-ultralytics>=8.3
-torch>=2.1
-torchvision>=0.16
-onnx>=1.16
-onnxruntime-gpu>=1.18
-opencv-python>=4.7
-numpy>=1.23
-pillow>=9.5
-pyyaml>=6.0
-huggingface_hub>=0.24
-roboflow>=1.1
-pycocotools>=2.0

training/train.py

@@ -1,140 +0,0 @@
-"""
-Two-stage training recipe for Detect-crime.
-
-Stage A (silver pretrain): warm a YOLOv11s/m on the full silver corpus
-                           (Roboflow + COCO bat + king-distilled manako frames).
-Stage B (clean fine-tune):  short fine-tune on a manually-verified set of manako frames.
-
-Defaults are tuned for a single GPU with >=24 GB VRAM. Pro_6000 should run YOLOv11s at
-imgsz=1536 with batch 12 comfortably; YOLOv11m at imgsz=1280 needs batch ~8.
-
-Usage:
-  # Stage A
-  python train.py --data ../data/data.yaml --weights yolo11s.pt --imgsz 1280 --stage A --epochs 200
-
-  # Stage B (clean fine-tune)
-  python train.py --data ../data_clean/data.yaml \
-                  --weights runs/detect/crime_a/weights/best.pt \
-                  --imgsz 1280 --stage B --epochs 50
-
-Notes:
-  - Heavy mosaic+copy_paste is essential for crime — class imbalance is brutal (hoodie
-    dominates GT count; balaclava/glove/spray-paint are rare). Copy-paste pastes rare-class
-    crops into otherwise-normal frames; this is the most reliable fix for low recall on the
-    rare classes.
-  - HSV jitter cranked on V channel: CCTV crime footage spans daylight, IR night, and
-    mixed indoor lighting. Augment hard.
-  - `cls=1.0` (raised from default 0.5) because mAP@50 is per-class-averaged; pushing the
-    classification head matters more than localization here.
-  - Always save with ema=True so export uses smoothed weights.
-"""
-
-from __future__ import annotations
-
-import argparse
-
-from ultralytics import YOLO
-
-
-COMMON_AUG = dict(
-    hsv_h=0.02,
-    hsv_s=0.7,
-    hsv_v=0.5,         # CCTV brightness varies massively (IR night, daylight, mixed indoor)
-    degrees=0.0,       # CCTV cameras are mounted level
-    translate=0.10,
-    scale=0.5,
-    shear=0.0,
-    perspective=0.0,
-    flipud=0.0,
-    fliplr=0.5,
-    mosaic=1.0,
-    mixup=0.20,
-    copy_paste=0.50,   # high: rebalance toward rare classes (balaclava, glove, spray paint)
-    erasing=0.0,
-)
-
-STAGE_A = dict(
-    epochs=200,
-    optimizer="AdamW",
-    lr0=2e-3,
-    lrf=0.01,
-    momentum=0.937,
-    weight_decay=5e-4,
-    warmup_epochs=3,
-    cos_lr=True,
-    label_smoothing=0.05,
-    cls=1.0,           # higher cls weight: per-class AP matters
-    box=7.5,
-    dfl=1.5,
-    close_mosaic=15,
-    patience=40,
-    amp=True,
-    **COMMON_AUG,
-)
-
-STAGE_B = dict(
-    epochs=50,
-    optimizer="AdamW",
-    lr0=5e-4,
-    lrf=0.01,
-    momentum=0.9,
-    weight_decay=5e-4,
-    warmup_epochs=2,
-    cos_lr=True,
-    label_smoothing=0.05,
-    cls=0.8,
-    box=7.5,
-    dfl=1.5,
-    close_mosaic=10,
-    patience=15,
-    amp=True,
-    **{**COMMON_AUG, "mosaic": 0.5, "mixup": 0.0, "copy_paste": 0.10},
-)
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--data", required=True, help="path to data.yaml (YOLO format)")
-    ap.add_argument("--weights", default="yolo11s.pt",
-                    help="starting weights (yolo11n.pt / yolo11s.pt / yolo11m.pt / your stage-A best)")
-    ap.add_argument("--imgsz", type=int, default=1280)
-    ap.add_argument("--batch", type=int, default=16)
-    ap.add_argument("--stage", choices=["A", "B"], default="A")
-    ap.add_argument("--epochs", type=int, default=None)
-    ap.add_argument("--device", default=0)
-    ap.add_argument("--project", default="runs/detect")
-    ap.add_argument("--name", default=None)
-    ap.add_argument("--resume", action="store_true")
-    args = ap.parse_args()
-
-    cfg = STAGE_A.copy() if args.stage == "A" else STAGE_B.copy()
-    if args.epochs is not None:
-        cfg["epochs"] = args.epochs
-
-    model = YOLO(args.weights)
-    name = args.name or f"crime_{args.stage.lower()}"
-
-    model.train(
-        data=args.data,
-        imgsz=args.imgsz,
-        batch=args.batch,
-        device=args.device,
-        project=args.project,
-        name=name,
-        resume=args.resume,
-        save=True,
-        plots=True,
-        seed=42,
-        rect=False,
-        single_cls=False,
-        **cfg,
-    )
-
-    metrics = model.val(data=args.data, imgsz=args.imgsz, batch=args.batch,
-                        device=args.device, plots=True, save_json=True)
-    print(f"[train] {args.stage} done. map50={metrics.box.map50:.4f} map={metrics.box.map:.4f}")
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

training/verify_dataset.py

@@ -1,170 +0,0 @@
-"""
-Quick QA over an assembled YOLO dataset for Detect-crime.
-
-Checks:
-  - Every label file has a corresponding image and vice versa.
-  - All class ids are within range (0..5).
-  - All bbox coords are normalized (0 < x < 1, 0 < y < 1, 0 < w/h <= 1).
-  - Reports per-class instance counts per split + total positive images per class.
-  - Flags samples with degenerate (near-zero) boxes.
-  - Optional: writes overlay PNGs for the first --visualize K val images.
-
-Usage:
-  python verify_dataset.py --data ../data/data.yaml
-  python verify_dataset.py --data ../data/data.yaml --visualize 20 --vis-out /tmp/crime_vis
-"""
-
-from __future__ import annotations
-
-import argparse
-import sys
-from pathlib import Path
-
-import numpy as np
-
-
-TARGET_CLASS_NAMES = ["balaclava", "bat", "glove", "graffiti", "hoodie", "spray paint"]
-
-
-def parse_yaml(path: Path) -> dict:
-    try:
-        import yaml
-        return yaml.safe_load(path.read_text())
-    except Exception:
-        cfg: dict = {}
-        for line in path.read_text().splitlines():
-            line = line.strip()
-            if not line or line.startswith("#") or ":" not in line:
-                continue
-            k, v = line.split(":", 1)
-            cfg[k.strip()] = v.strip()
-        return cfg
-
-
-def check_split(root: Path, split: str, log) -> dict:
-    img_dir = root / "images" / split
-    lbl_dir = root / "labels" / split
-    if not img_dir.exists():
-        log(f"[{split}] images/{split} missing: {img_dir}")
-        return {"images": 0}
-    if not lbl_dir.exists():
-        log(f"[{split}] labels/{split} missing: {lbl_dir}")
-        return {"images": 0}
-
-    image_names = {p.stem for p in img_dir.iterdir() if p.is_file()}
-    label_names = {p.stem for p in lbl_dir.iterdir() if p.is_file() and p.suffix == ".txt"}
-    only_img = image_names - label_names
-    only_lbl = label_names - image_names
-
-    n_total = len(image_names)
-    cls_counts = [0] * len(TARGET_CLASS_NAMES)
-    cls_image_counts = [set() for _ in range(len(TARGET_CLASS_NAMES))]
-    bad_rows = 0
-    degenerate = 0
-    for lbl in lbl_dir.glob("*.txt"):
-        rows = lbl.read_text().splitlines()
-        for r in rows:
-            parts = r.strip().split()
-            if len(parts) != 5:
-                bad_rows += 1
-                continue
-            try:
-                c = int(parts[0])
-                cx, cy, bw, bh = (float(parts[1]), float(parts[2]), float(parts[3]), float(parts[4]))
-            except Exception:
-                bad_rows += 1
-                continue
-            if not (0 <= c < len(TARGET_CLASS_NAMES)):
-                bad_rows += 1
-                continue
-            if not (0 < cx < 1 and 0 < cy < 1 and 0 < bw <= 1 and 0 < bh <= 1):
-                degenerate += 1
-                continue
-            cls_counts[c] += 1
-            cls_image_counts[c].add(lbl.stem)
-
-    return {
-        "images": n_total,
-        "only_img": only_img,
-        "only_lbl": only_lbl,
-        "cls_counts": cls_counts,
-        "cls_image_counts": [len(s) for s in cls_image_counts],
-        "bad_rows": bad_rows,
-        "degenerate": degenerate,
-    }
-
-
-def main() -> int:
-    ap = argparse.ArgumentParser()
-    ap.add_argument("--data", required=True)
-    ap.add_argument("--visualize", type=int, default=0,
-                    help="render this many val images with boxes overlaid")
-    ap.add_argument("--vis-out", default="/tmp/crime_vis", help="dir for overlay images")
-    args = ap.parse_args()
-
-    data_yaml = Path(args.data).resolve()
-    cfg = parse_yaml(data_yaml)
-    root = Path(cfg.get("path") or data_yaml.parent).resolve()
-
-    def log(msg: str):
-        print(msg, file=sys.stderr, flush=True)
-
-    log(f"[verify] root={root}")
-    overall_ok = True
-    for split in ("train", "val"):
-        res = check_split(root, split, log)
-        if not res["images"]:
-            continue
-        log(f"[{split}] images: {res['images']}")
-        if res["only_img"]:
-            log(f"[{split}] {len(res['only_img'])} images without labels (sample: "
-                f"{list(res['only_img'])[:3]})")
-        if res["only_lbl"]:
-            log(f"[{split}] {len(res['only_lbl'])} labels without images (sample: "
-                f"{list(res['only_lbl'])[:3]})")
-            overall_ok = False
-        if res["bad_rows"]:
-            log(f"[{split}] BAD ROWS: {res['bad_rows']}"); overall_ok = False
-        if res["degenerate"]:
-            log(f"[{split}] degenerate boxes (out-of-range): {res['degenerate']}")
-            overall_ok = False
-        log(f"[{split}] per-class instance counts:")
-        for i, n in enumerate(TARGET_CLASS_NAMES):
-            log(f"    {i} {n:13s}  instances={res['cls_counts'][i]:6d}  "
-                f"images={res['cls_image_counts'][i]:5d}")
-
-    if args.visualize > 0:
-        try:
-            import cv2
-        except ImportError:
-            log("[viz] cv2 not installed; pip install opencv-python")
-            return 0 if overall_ok else 1
-        out = Path(args.vis_out).resolve()
-        out.mkdir(parents=True, exist_ok=True)
-        val_imgs = sorted((root / "images" / "val").iterdir())[: args.visualize]
-        for img_path in val_imgs:
-            lbl_path = root / "labels" / "val" / (img_path.stem + ".txt")
-            img = cv2.imread(str(img_path))
-            if img is None:
-                continue
-            h, w = img.shape[:2]
-            if lbl_path.exists():
-                for r in lbl_path.read_text().splitlines():
-                    p = r.split()
-                    if len(p) != 5:
-                        continue
-                    c = int(p[0])
-                    cx, cy, bw, bh = (float(p[1]) * w, float(p[2]) * h,
-                                      float(p[3]) * w, float(p[4]) * h)
-                    x1, y1, x2, y2 = int(cx - bw / 2), int(cy - bh / 2), int(cx + bw / 2), int(cy + bh / 2)
-                    cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 2)
-                    cv2.putText(img, TARGET_CLASS_NAMES[c], (x1, max(0, y1 - 6)),
-                                cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 1)
-            cv2.imwrite(str(out / img_path.name), img)
-        log(f"[viz] wrote {len(val_imgs)} overlays to {out}")
-
-    return 0 if overall_ok else 1
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())