README.md

# Detect-crime Miner — Recipe to Beat the King

Target element: `manak0/Detect-crime` on subnet 423 (open-source / public track).

Read [ANALYSIS.md](ANALYSIS.md) first — it documents the king's model (the manak0 baseline)
and where the gap lives.

Current king of record (2026-05-04 leaderboard): hotkey `5CSeBY…tv9f`, score **0.576**.
Crime is **uncontested**: there is no `Detect-crime-winner` HF repo, and the king's score
is within rounding of the published baseline's `overall_iou` (0.597). Anybody who lands a
modest improvement takes the throne.

## Layout

```
crime_miner/
├── ANALYSIS.md            ← analysis of the king + scoring + constraints
├── README.md              ← this file
├── miner.py               ← deployable inference (multi-scale TTA + WBF + CLAHE)
├── chute_config.yml       ← chute resource spec (16 GB GPU, matches king's)
├── class_names.txt        ← target class order — DO NOT REORDER
└── training/
    ├── DATASET.md         ← dataset sources + pipeline (start here)
    ├── build_dataset.py   ← end-to-end builder: manako + Roboflow + COCO bat
    ├── poll_manako.py     ← background poller for in-domain frames + king's preds
    ├── train.py           ← two-stage YOLOv11 training (silver → clean fine-tune)
    ├── verify_dataset.py  ← QA over assembled YOLO dirs
    ├── export_onnx.py     ← export with NMS baked in -> [1, 300, 6]
    └── requirements.txt
```

## What the miner does differently

`miner.py` keeps the king's I/O contract (single `weights.onnx` → `TVFrameResult`) but adds
six concrete improvements over the auto-generated `subnet_bridge` template the king ships:

1. **Letterboxed input at 1280** instead of stretch-resized 640. Small objects (balaclava
   ~30 px, glove ~25 px, spray paint can ~20 px) survive — the king's stretch resize
   destroys them. This alone lifts recall on the four catastrophic classes.
2. **Per-class confidence floors**. King uses one global 0.25 across all six classes; we
   set `balaclava=0.05, bat=0.10, glove=0.05, graffiti=0.20, hoodie=0.20, spray paint=0.10`.
   Synthetic-benchmark recalls were 0.034 / 0.143 / 0.064 / 0.321 / 0.274 / 0.161 — the
   bottleneck is recall, and the FFPI cap has plenty of headroom (~6.5 preds/img today).
3. **Multi-scale TTA** at `{1280, 1536} × {orig, hflip}` = 4 forward passes, collapsed to 2
   when the ONNX export is static-shape. Pro_6000 has the budget (latency p95 = 10 s).
4. **Weighted Box Fusion** across TTA streams. WBF averages cluster boxes weighted by
   score, which yields tighter localizations than always picking the highest-confidence
   proposal — and tighter boxes mean more cases cross the IoU≥0.5 bar that the scorer uses.
5. **CLAHE on dark frames only** (luma gate). Crime CCTV is night-heavy. King applies no
   preprocessing.
6. **Class-aware NMS at IoU=0.45**. King uses class-agnostic NMS, which suppresses
   balaclava-on-hoodie or glove-near-bat overlaps. Class-aware keeps both.

Total ONNX inference cost on Pro_6000 with YOLOv11s + 2-scale TTA is well under 1 s/frame.

## How to deploy

You need: a `weights.onnx` exported in `[1, 300, 6]` layout (NMS baked in) — produced by
`training/export_onnx.py` after training, OR you can ship the king's raw ONNX directly to
test the inference improvements alone.

### Option 1 — drop-in test with the king's weights

Sanity-check that the inference improvements alone help, before training:

```bash
cp /root/turbovision_crime/king_models/Detect-crime/weights.onnx ./weights.onnx
python miner.py   # smoke test on /tmp/crime_proof.png
```

Expected: with the king's weights but our miner.py, you should already see a noticeable lift
on the rare classes (recall driven up by the lower per-class conf floors and the 1280 input
that the dynamic-shape ONNX accepts). The king's published ONNX is **static** at 640×640,
so the dynamic letterbox path won't help unless you re-export — see below.

### Option 2 — train a real beating model

See [training/DATASET.md](training/DATASET.md) for full data-pipeline notes. Quick path:

```bash
cd training
pip install -r requirements.txt

# 1) Start the manako poller in the background to accumulate in-domain frames
#    (each rotation surfaces a fresh challenge ~every few minutes during active scoring).
python poll_manako.py --out ../manako_pool --interval 120 --forever &

# 2) Build the silver dataset. Combine manako frames (king-labeled), Roboflow
#    per-class detection sets, and (optional) COCO baseball bat. Roboflow needs
#    ROBOFLOW_API_KEY in env.
python build_dataset.py \
    --out ../data \
    --king-onnx /root/turbovision_crime/king_models/Detect-crime/weights.onnx \
    --manako --manako-polls 30 --manako-poll-delay 120 \
    --roboflow balaclava=brainster/balaclava-detection-v3 \
    --roboflow glove=ppe-detection/gloves-v1 \
    --roboflow graffiti=graffiti-detection/graffiti-v3 \
    --roboflow "spray paint=tools/spray-paint-can-v1" \
    --coco-bat /path/to/coco/instances_train2017.json /path/to/coco/train2017 \
    --extra-dir ../manako_pool/images \
    --min-conf 0.10 --keep-empty --intra-threads 16

# 3) Verify the assembled dataset
python verify_dataset.py --data ../data/data.yaml --visualize 20

# 4) Stage A: silver pretrain
python train.py --data ../data/data.yaml --weights yolo11s.pt \
                --imgsz 1280 --batch 16 --stage A --epochs 200 --name crime_a

# 5) Build a clean set: hand-verify (or LLM-verify) ~300 manako frames into
#    ../data_clean/data.yaml with the same YOLO layout.

# 6) Stage B: clean fine-tune
python train.py --data ../data_clean/data.yaml \
                --weights ../runs/detect/crime_a/weights/best.pt \
                --imgsz 1280 --batch 16 --stage B --epochs 50 --name crime_b

# 7) Export with NMS baked in -> [1, 300, 6]
python export_onnx.py --weights ../runs/detect/crime_b/weights/best.pt \
                      --imgsz 1280 --out ../weights.onnx
```

### Option 3 — deploy via the turbovision CLI

```bash
cd /root/turbovision_crime
sv -vv deploy-os-miner --model-path scratch/crime_miner --element-id manak0/Detect-crime
```

The CLI uploads `miner.py`, `weights.onnx`, `class_names.txt`, `chute_config.yml` to your
HF repo, builds the chute, and commits the on-chain pointer.

## Tuning knobs (top of `miner.py`)

| Constant | Default | Effect of raising | Effect of lowering |
|---|---|---|---|
| `PER_CLASS_CONF[0]` (balaclava) | 0.05 | fewer FPs (good for FFPI) | more recall (better AP, better IoU) |
| `PER_CLASS_CONF[2]` (glove) | 0.05 | as above | as above |
| `PER_CLASS_CONF[4]` (hoodie) | 0.20 | fewer hoodie FPs | more boxes (may hurt precision) |
| `TTA_SIZES` | (1280, 1536) | better small-object recall | faster inference |
| `WBF_IOU` | 0.55 | more conservative fusion | tighter clusters |
| `NMS_IOU` | 0.45 | keeps more near-duplicates | stricter dedup |
| `MAX_DET` | 100 | more boxes survive ranking | tighter cap |
| `CLAHE_DARK_THRESHOLD` | 70 | CLAHE on more frames | only the very dark ones |

When tuning, validate against `runs/detect/crime_b/val_batch*.jpg` and the manako latest
challenge image — don't hill-climb on the synthetic benchmark alone (it's only 50 frames).

## Why these specific choices

- **The IoU pillar dominates the live score** (dashboard 0.576 ≈ baseline `overall_iou`
  0.597). IoU is the *label-agnostic* AUC-F1 placement metric — what matters most is
  whether *any* well-placed box exists for each GT. So the optimal strategy is to flood
  predictions for the rare classes; the FFPI cap (10 FP/image, currently ~6.5 preds/img
  baseline) gives generous headroom.
- **mAP@50 matters too** because secondary pillars are likely weighted in. mAP@50 is
  per-class-averaged with strict label match. Raising recall on the four near-zero classes
  even modestly (0.03 → 0.20 on balaclava) lifts the per-class mean by ~0.03 alone.
- **WBF over hard NMS**: tighter localizations → more boxes clearing the IoU≥0.5 bar.
- **Class-aware NMS**: balaclava overlaps with hoodie geometry; bat overlaps with glove
  on a held bat. Class-agnostic NMS would silently kill one of each pair.
- **CLAHE only on dark frames**: applying CLAHE to bright frames hurts hoodie/graffiti
  texture. Luma gate keeps it surgical.

## Verifying you're actually beating the king

Before committing on-chain:

1. Pull the latest annotated challenge image+predictions:
   ```bash
   curl -sL "https://console.scorevision.io/api/v2/elements/manak0%2FDetect-crime?lookback_days=7" \
     | jq '.latestAnnotatedChallenge'
   ```
2. Run your `miner.py` on that image; visually verify your boxes ≥ king's, especially on
   balaclava, glove, and spray paint.
3. Run `sv -vv run-once` (per `MINER.md`) to score yourself end-to-end on a real challenge
   without committing — confirms the chute deploys correctly and your output format matches.
4. Only after the offline score is repeatedly above 0.62 (the king + a comfortable margin)
   should you deploy and commit.

## Open questions / pending work

- **Live pillar weights for `Detect-crime`** — confirm by reading the active manifest with
  `sv -vv elements list` once `.env` is configured. The recipe above assumes IoU-dominated
  scoring; if mAP/precision/recall pillars are weighted higher, the per-class confidence
  floors should be raised (less recall, more precision).
- **Real GT vs SAM3 PGT** — confirm whether `elements[].ground_truth = true` in the live
  manifest. If real GT (Manako-internal), the synthetic_fixed dataset on HF is the closest
  proxy and we should overfit it carefully. If SAM3 PGT, the live targets are whatever
  SAM3 detects when prompted with the 6 class names — slightly fuzzier.
- **Manako data pull** — `poll_manako.py` is built but untested for `Detect-crime`. The
  endpoint shape is the same as petrol-station's; if Manako gates the API for low-traffic
  elements, fall back to using the king's ONNX as the silver labeler over Roboflow data.