# DOPER_HOPE — RTMPose keypoint models for HOPE objects

Trained on 40k synthetic images per object (Blender EEVEE, uniform SO(3) pose, 5–8 distractors).
Evaluated on the real HOPE BOP val split using PnP pose estimation.

## Pipeline

1. Build `.blend` from BOP PLY (mm → m conversion)
2. Render 40k synthetic images per object (32 workers, Blender 5.1 EEVEE)
3. Merge worker annotations, 95/5 train/val split
4. Train RTMPose-t, 300 epochs, 8 GPUs
5. Eval: RTMPose inference → PnP → BOP metrics on real HOPE val

## Repository structure

```
obj_000001/
  best_coco_AP_epoch_NNN.pth   ← best checkpoint
  obj_000001.blend             ← Blender scene with keypoint empties
  obj_000001.png               ← UV texture
  obj_000001_keypoints_3d.json ← 3D keypoint positions (meters)
  real_val_grid.jpg            ← 5×5 PnP visualisation on real val images
  synth_val_grid.jpg           ← 5×5 sample of synthetic val images
  bop_summary.json             ← per-object BOP metrics
...
```

## BOP Metrics — HOPE val split

| ID | Object | ADD-AUC | ADDS-AUC | ADD<10% | ADDS<10% | Best epoch |
|----|--------|---------|----------|---------|----------|------------|
|  1 | AlphabetSoup         | 0.4607  | 0.5672   | 0.111   | 0.222    | 200 |
|  2 | BBQSauce             | 0.4783  | 0.5977   | 0.178   | 0.356    | 300 |
|  3 | Butter               | 0.3786  | 0.4348   | 0.244   | 0.311    |  90 |
|  4 | Cherries             | 0.5712  | 0.7205   | 0.150   | 0.450    | 260 |
|  5 | ChocolatePudding     | 0.6897  | 0.7764   | 0.440   | 0.560    | 220 |
|  6 | Cookies              | 0.8272  | 0.9123   | 0.200   | 0.600    | 300 |
|  7 | Corn                 | 0.3038  | 0.4866   | 0.130   | 0.217    | 140 |
|  8 | CreamCheese          | 0.3328  | 0.4646   | 0.040   | 0.160    |  30 |
|  9 | GranolaBars          | 0.4354  | 0.5482   | 0.200   | 0.320    | 120 |
| 10 | GreenBeans           | 0.2668  | 0.3570   | 0.125   | 0.281    |  80 |
| 11 | Ketchup              | 0.5817  | 0.7010   | 0.350   | 0.575    |  80 |
| 12 | Macaroni             | 0.4228  | 0.6292   | 0.180   | 0.420    | 180 |
| 13 | Mayo                 | 0.4735  | 0.5996   | 0.167   | 0.233    | 230 |
| 14 | Milk                 | 0.7333  | 0.8359   | 0.160   | 0.440    | 100 |
| 15 | Mushrooms            | 0.1255  | 0.2801   | 0.000   | 0.030    |  90 |
| 16 | Mustard              | 0.6703  | 0.7290   | 0.340   | 0.540    | 200 |
| 17 | OrangeJuice          | 0.5128  | 0.5882   | 0.235   | 0.412    | 230 |
| 18 | Parmesan             | 0.6658  | 0.7573   | 0.333   | 0.567    | 210 |
| 19 | Peaches              | 0.5019  | 0.6263   | 0.150   | 0.250    |  70 |
| 20 | PeasAndCarrots       | 0.1679  | 0.2462   | 0.100   | 0.100    | 100 |
| 21 | Pineapple            | 0.4367  | 0.5948   | 0.167   | 0.396    | 270 |
| 22 | Popcorn              | 0.3282  | 0.4506   | 0.050   | 0.150    | 290 |
| 23 | Raisins              | 0.3327  | 0.5173   | 0.075   | 0.225    | 180 |
| 24 | SaladDressing        | 0.4628  | 0.4756   | 0.400   | 0.400    | 260 |
| 25 | Spaghetti            | 0.7319  | 0.7985   | 0.486   | 0.571    | 220 |
| 26 | TomatoSauce          | 0.4514  | 0.6269   | 0.241   | 0.345    | 290 |
| 27 | Tuna                 | 0.2392  | 0.3377   | 0.000   | 0.057    |  60 |
| 28 | Yogurt               | 0.6595  | 0.8333   | 0.167   | 0.278    | 190 |

**Mean (28/28):** ADD-AUC = 0.4722 | ADDS-AUC = 0.5924

## Notes

- **obj_000003 (Butter)**: previously failed (ADD-AUC = 0.0). Fixed by regenerating keypoints as
  8 symmetric pairs + 1 YZ-plane anchor to handle the object's YZ-plane reflection symmetry.
  Retrained from scratch; ADD-AUC improved from 0.000 → 0.379.
- **obj_000022 (Popcorn)**: previously ADD-AUC = 0.085. Fixed by regenerating keypoints as
  8 symmetric pairs + 1 Z-axis anchor to handle the object's 180-deg rotational symmetry around Z.
  Retrained from scratch; ADD-AUC improved from 0.085 → 0.328, ADDS-AUC from 0.163 → 0.451.
- All models use RTMPose-t architecture.
- Synthetic data: Blender 5.1 EEVEE, HDRI lighting, random CCO textures, 5–8 USD distractor objects.
- BOP PLY meshes are in mm; converted to meters for Blender. Evaluation uses original mm scale.