| # DOPER_HOPE — RTMPose keypoint models for HOPE objects |
| |
| Trained on 40k synthetic images per object (Blender EEVEE, uniform SO(3) pose, 5–8 distractors). |
| Evaluated on the real HOPE BOP val split using PnP pose estimation. |
| |
| ## Pipeline |
| |
| 1. Build `.blend` from BOP PLY (mm → m conversion) |
| 2. Render 40k synthetic images per object (32 workers, Blender 5.1 EEVEE) |
| 3. Merge worker annotations, 95/5 train/val split |
| 4. Train RTMPose-t, 300 epochs, 8 GPUs |
| 5. Eval: RTMPose inference → PnP → BOP metrics on real HOPE val |
| |
| ## Repository structure |
| |
| ``` |
| obj_000001/ |
| best_coco_AP_epoch_NNN.pth ← best checkpoint |
| obj_000001.blend ← Blender scene with keypoint empties |
| obj_000001.png ← UV texture |
| obj_000001_keypoints_3d.json ← 3D keypoint positions (meters) |
| real_val_grid.jpg ← 5×5 PnP visualisation on real val images |
| synth_val_grid.jpg ← 5×5 sample of synthetic val images |
| bop_summary.json ← per-object BOP metrics |
| ... |
| ``` |
| |
| ## BOP Metrics — HOPE val split |
| |
| | ID | Object | ADD-AUC | ADDS-AUC | ADD<10% | ADDS<10% | Best epoch | |
| |----|--------|---------|----------|---------|----------|------------| |
| | 1 | AlphabetSoup | 0.4607 | 0.5672 | 0.111 | 0.222 | 200 | |
| | 2 | BBQSauce | 0.4783 | 0.5977 | 0.178 | 0.356 | 300 | |
| | 3 | Butter | 0.3786 | 0.4348 | 0.244 | 0.311 | 90 | |
| | 4 | Cherries | 0.5712 | 0.7205 | 0.150 | 0.450 | 260 | |
| | 5 | ChocolatePudding | 0.6897 | 0.7764 | 0.440 | 0.560 | 220 | |
| | 6 | Cookies | 0.8272 | 0.9123 | 0.200 | 0.600 | 300 | |
| | 7 | Corn | 0.3038 | 0.4866 | 0.130 | 0.217 | 140 | |
| | 8 | CreamCheese | 0.3328 | 0.4646 | 0.040 | 0.160 | 30 | |
| | 9 | GranolaBars | 0.4354 | 0.5482 | 0.200 | 0.320 | 120 | |
| | 10 | GreenBeans | 0.2668 | 0.3570 | 0.125 | 0.281 | 80 | |
| | 11 | Ketchup | 0.5817 | 0.7010 | 0.350 | 0.575 | 80 | |
| | 12 | Macaroni | 0.4228 | 0.6292 | 0.180 | 0.420 | 180 | |
| | 13 | Mayo | 0.4735 | 0.5996 | 0.167 | 0.233 | 230 | |
| | 14 | Milk | 0.7333 | 0.8359 | 0.160 | 0.440 | 100 | |
| | 15 | Mushrooms | 0.1255 | 0.2801 | 0.000 | 0.030 | 90 | |
| | 16 | Mustard | 0.6703 | 0.7290 | 0.340 | 0.540 | 200 | |
| | 17 | OrangeJuice | 0.5128 | 0.5882 | 0.235 | 0.412 | 230 | |
| | 18 | Parmesan | 0.6658 | 0.7573 | 0.333 | 0.567 | 210 | |
| | 19 | Peaches | 0.5019 | 0.6263 | 0.150 | 0.250 | 70 | |
| | 20 | PeasAndCarrots | 0.1679 | 0.2462 | 0.100 | 0.100 | 100 | |
| | 21 | Pineapple | 0.4367 | 0.5948 | 0.167 | 0.396 | 270 | |
| | 22 | Popcorn | 0.3282 | 0.4506 | 0.050 | 0.150 | 290 | |
| | 23 | Raisins | 0.3327 | 0.5173 | 0.075 | 0.225 | 180 | |
| | 24 | SaladDressing | 0.4628 | 0.4756 | 0.400 | 0.400 | 260 | |
| | 25 | Spaghetti | 0.7319 | 0.7985 | 0.486 | 0.571 | 220 | |
| | 26 | TomatoSauce | 0.4514 | 0.6269 | 0.241 | 0.345 | 290 | |
| | 27 | Tuna | 0.2392 | 0.3377 | 0.000 | 0.057 | 60 | |
| | 28 | Yogurt | 0.6595 | 0.8333 | 0.167 | 0.278 | 190 | |
| |
| **Mean (28/28):** ADD-AUC = 0.4722 | ADDS-AUC = 0.5924 |
| |
| ## Notes |
| |
| - **obj_000003 (Butter)**: previously failed (ADD-AUC = 0.0). Fixed by regenerating keypoints as |
| 8 symmetric pairs + 1 YZ-plane anchor to handle the object's YZ-plane reflection symmetry. |
| Retrained from scratch; ADD-AUC improved from 0.000 → 0.379. |
| - **obj_000022 (Popcorn)**: previously ADD-AUC = 0.085. Fixed by regenerating keypoints as |
| 8 symmetric pairs + 1 Z-axis anchor to handle the object's 180-deg rotational symmetry around Z. |
| Retrained from scratch; ADD-AUC improved from 0.085 → 0.328, ADDS-AUC from 0.163 → 0.451. |
| - All models use RTMPose-t architecture. |
| - Synthetic data: Blender 5.1 EEVEE, HDRI lighting, random CCO textures, 5–8 USD distractor objects. |
| - BOP PLY meshes are in mm; converted to meters for Blender. Evaluation uses original mm scale. |
| |
