# Plan-Bind D2 results

Source: `/home/marinelab/code/plan-bind-d2/experiments/261028-plan-bind-d2-latency/outputs/full`. Success and stage rates in percent; 50 seeded episodes per cell unless noted.

## Inference time of one call (batch 1)

A call: preprocess, encoders on the observation frames, then the head (bsp10: DDIM denoise and spline decode; planbind: plan and bind, the slower call). Latency frames: the call's p95 rounded up to frames at 30 fps, the latency each policy is evaluated at below.

| Bench | Hardware | Policy | Call p50 (ms) | Call p95 (ms) | Bind-only p50 (ms) | Latency frames | select_action mean (ms) | Stage p50 (ms) |
|---|---|---|---|---|---|---|---|---|
| cpu | cpu x4 | bsp10 | 590.0 | 628.9 | - | 19 | 27.1 | preprocess 0.7, encode 58.0, denoise 528.4, decode 0.2 |
| cpu | cpu x4 | onestep | 125.9 | 130.1 | - | 4 | 32.0 | preprocess 0.8, encode 60.2, onestep 64.5 |
| cpu | cpu x4 | planbind | 162.2 | 164.7 | 113.0 | 5 | 30.2 | preprocess 1.0, encode 50.0, plan 49.0, bind 62.1 |
| cuda | NVIDIA GeForce RTX 5050 | bsp10 | 169.7 | 170.2 | - | 6 | 8.4 | preprocess 1.3, encode 4.2, denoise 163.9, decode 0.3 |
| cuda | NVIDIA GeForce RTX 5050 | onestep | 22.1 | 22.2 | - | 1 | 5.2 | preprocess 1.3, encode 4.2, onestep 16.6 |
| cuda | NVIDIA GeForce RTX 5050 | planbind | 38.7 | 38.9 | 22.1 | 2 | 5.8 | preprocess 1.3, encode 4.2, plan 16.5, bind 16.6 |
| cuda_amp | NVIDIA GeForce RTX 5050 | bsp10 | 189.7 | 190.1 | - | 6 | 8.6 | preprocess 1.3, encode 3.7, denoise 184.4, decode 0.3 |
| cuda_amp | NVIDIA GeForce RTX 5050 | onestep | 23.7 | 23.8 | - | 1 | 5.6 | preprocess 1.3, encode 3.7, onestep 18.6 |
| cuda_amp | NVIDIA GeForce RTX 5050 | planbind | 42.4 | 42.5 | 23.8 | 2 | 6.2 | preprocess 1.3, encode 3.7, plan 18.6, bind 18.7 |

## Each policy at its measured latency

| Bench | Policy | Latency (frames) | Latency (ms) | Success |
|---|---|---|---|---|
| cpu | bsp10 | 19 | 633 | 22 |
| cpu | onestep | 4 | 133 | 32 |
| cpu | planbind | 5 | 167 | 100 |
| cuda | bsp10 | 6 | 200 | 52 |
| cuda | onestep | 1 | 33 | 32 |
| cuda | planbind | 2 | 67 | 94 |

## Update rate without latency

Latency 0; a call every 4 frames is D1's rate. In D1 a call could not start before the previous one
swapped in, so latency 0, 3, 6 and 9 also meant a call every 4, 4, 8 and 12 frames: these cells
tell rate from latency apart.

| Policy | Commit | Every 4 | Every 8 | Every 12 |
|---|---|---|---|---|
| onestep | latency | 66 | 58 | 70 |
| onestep | none | 36 | 50 | 62 |
| planbind | latency | 92 | 100 | 98 |
| planbind | none | 88 | 98 | 100 |

## Commit modes

| Policy | Latency (ms) | latency | now | none | now vs latency |
|---|---|---|---|---|---|
| onestep | 0 | 66 | 68 | 36 | +2 (+1/-0) |
| onestep | 200 | 78 | 82 | 76 | +4 (+9/-7) |
| planbind | 0 | 92 | 94 | 88 | +2 (+1/-0) |
| planbind | 200 | 100 | 100 | 100 | +0 (+0/-0) |

## Plan-Bind replan threshold (latency 6, commit latency)

| Threshold | Success | vs off |
|---|---|---|
| off | 100 | - |
| 0.5 | 70 | -30 (+0/-15) |
| 1.0 | 100 | +0 (+0/-0) |

## bsp10 through the spline runtime

`classic` is the action-queue policy (D1's bsp10_L0): it plans 32 frames and replans when they run out.

| Mode | Latency (ms) | Commit | Success | vs classic |
|---|---|---|---|---|
| classic | 0 | - | 100 | - |
| runtime | 0 | latency | 80 | -20 (+0/-10) |
| runtime | 0 | none | 58 | -42 (+0/-21) |
| runtime | 200 | latency | 52 | -48 (+0/-24) |
| runtime | 300 | latency | 36 | -64 (+0/-32) |
| runtime | 633 | latency | 22 | -78 (+0/-39) |

## The bottom bowl moves when the middle bowl is lifted

| Cell | Shift | Static | Shifted | Shifted vs static | Middle nested (shifted) |
|---|---|---|---|---|---|
| bsp10_L0 | shift3cm | 100 | 90 | -10 (+0/-5) | 98 |
| bsp10_L0_latency | shift3cm | 80 | 76 | -4 (+4/-6) | 96 |
| bsp10_L6_latency | shift3cm | 52 | 34 | -18 (+2/-11) | 66 |
| onestep_L0_latency | shift3cm | 66 | 56 | -10 (+2/-7) | 92 |
| onestep_L6_latency | shift3cm | 78 | 62 | -16 (+1/-9) | 86 |
| planbind_L0_latency | shift3cm | 92 | 74 | -18 (+2/-11) | 94 |
| planbind_L6_latency | shift3cm | 100 | 84 | -16 (+0/-8) | 100 |

## All cells

| Cell | Episodes | Success | 95% CI | nested_middle | nested_top | picked_middle | picked_top | released_middle | released_top |
|---|---|---|---|---|---|---|---|---|---|
| bsp10_L0 | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| bsp10_L0_latency | 50 | 80 | 67-89 | 90 | 84 | 96 | 88 | 90 | 82 |
| bsp10_L0_none | 50 | 58 | 44-71 | 92 | 62 | 92 | 64 | 92 | 60 |
| bsp10_L6_latency | 50 | 52 | 39-65 | 74 | 56 | 76 | 62 | 72 | 56 |
| bsp10_L9_latency | 50 | 36 | 24-50 | 68 | 42 | 80 | 46 | 68 | 40 |
| bsp10_L19_latency | 50 | 22 | 13-35 | 78 | 24 | 80 | 28 | 74 | 24 |
| bsp10_L0_shift3cm | 50 | 90 | 79-96 | 98 | 92 | 98 | 92 | 98 | 92 |
| bsp10_L0_latency_shift3cm | 50 | 76 | 63-86 | 96 | 84 | 100 | 86 | 92 | 82 |
| bsp10_L6_latency_shift3cm | 50 | 34 | 22-48 | 66 | 46 | 68 | 50 | 66 | 44 |
| onestep_L0_latency | 50 | 66 | 52-78 | 92 | 66 | 92 | 66 | 92 | 66 |
| onestep_L0_latency_every8 | 50 | 58 | 44-71 | 84 | 60 | 88 | 64 | 84 | 60 |
| onestep_L0_latency_every12 | 50 | 70 | 56-81 | 88 | 72 | 92 | 72 | 88 | 72 |
| onestep_L0_none | 50 | 36 | 24-50 | 70 | 38 | 72 | 38 | 70 | 36 |
| onestep_L0_none_every8 | 50 | 50 | 37-63 | 78 | 50 | 80 | 52 | 78 | 50 |
| onestep_L0_none_every12 | 50 | 62 | 48-74 | 88 | 62 | 88 | 64 | 88 | 62 |
| onestep_L0_now | 50 | 68 | 54-79 | 92 | 68 | 92 | 68 | 92 | 68 |
| onestep_L1_latency | 50 | 32 | 21-46 | 56 | 32 | 66 | 46 | 54 | 32 |
| onestep_L4_latency | 50 | 32 | 21-46 | 60 | 36 | 60 | 48 | 60 | 36 |
| onestep_L6_latency | 50 | 78 | 65-87 | 84 | 84 | 86 | 84 | 84 | 84 |
| onestep_L6_none | 50 | 76 | 63-86 | 98 | 78 | 98 | 80 | 98 | 78 |
| onestep_L6_now | 50 | 82 | 69-90 | 96 | 84 | 98 | 88 | 96 | 84 |
| onestep_L0_latency_shift3cm | 50 | 56 | 42-69 | 92 | 60 | 92 | 62 | 92 | 60 |
| onestep_L6_latency_shift3cm | 50 | 62 | 48-74 | 86 | 70 | 88 | 74 | 86 | 70 |
| planbind_L0_latency | 50 | 92 | 81-97 | 92 | 92 | 94 | 92 | 92 | 92 |
| planbind_L0_latency_every8 | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| planbind_L0_latency_every12 | 50 | 98 | 90-100 | 98 | 98 | 98 | 98 | 98 | 98 |
| planbind_L0_none | 50 | 88 | 76-94 | 90 | 88 | 90 | 88 | 88 | 88 |
| planbind_L0_none_every8 | 50 | 98 | 90-100 | 100 | 98 | 100 | 100 | 100 | 98 |
| planbind_L0_none_every12 | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| planbind_L0_now | 50 | 94 | 84-98 | 94 | 94 | 94 | 94 | 94 | 94 |
| planbind_L2_latency | 50 | 94 | 84-98 | 94 | 94 | 98 | 94 | 94 | 94 |
| planbind_L5_latency | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| planbind_L6_latency | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| planbind_L6_latency_tau0.5 | 50 | 70 | 56-81 | 80 | 74 | 80 | 80 | 80 | 72 |
| planbind_L6_latency_tau1.0 | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| planbind_L6_none | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| planbind_L6_now | 50 | 100 | 93-100 | 100 | 100 | 100 | 100 | 100 | 100 |
| planbind_L0_latency_shift3cm | 50 | 74 | 60-84 | 94 | 92 | 96 | 94 | 94 | 92 |
| planbind_L6_latency_shift3cm | 50 | 84 | 71-92 | 100 | 100 | 100 | 100 | 100 | 100 |