Auto-sync: 2026-06-27 11:24:43

results/README.md

@@ -15,6 +15,7 @@ Use these summaries for the current paper tables:
 - `h16_policy_ckpt_near_miss_policy_field_k32_sigma0p35_summary.md`
 - `h16_policy_ckpt_near_miss_policy_fieldckpt_field_k32_sigma0p35_summary.md`
 - `h16_policy_ckpt_near_miss_policy_bc5_fieldckpt_field_k32_sigma0p35_summary.md`
+- `h16_policy_ckpt_near_miss_policy_bc5_bestpt_field_sweep_summary.md`
 
 Known stale or incomplete files:
 
@@ -26,8 +27,8 @@ Known stale or incomplete files:
   absent/incomplete because the final seed was canceled after the stronger `best.pt`
   field-checkpoint variant completed.
 
-Pending:
+Completed sweep:
 
-- Job `14837136` is sweeping K/sigma for the current best deployment-clean route
-  (`near_miss_policy_bc5` + `best.pt` + Gaussian field selection). Summary job:
-  `14837137`.
+- Job `14837136` swept K/sigma for the current best deployment-clean route
+  (`near_miss_policy_bc5` + `best.pt` + Gaussian field selection). Summary job
+  `14837137` wrote `h16_policy_ckpt_near_miss_policy_bc5_bestpt_field_sweep_summary.*`.

results/dovla_cil_run_report_2026-06-27.md

@@ -13,7 +13,7 @@ The deployed h=16 behavior-cloning policy remains weak, but the interventional f
 | Near-miss distillation policy, BC x5 | 28.29% | -1.45 pp | Stronger BC improves slightly but remains below baseline |
 | Near-miss proposal + field, best-policy checkpoint | 26.32% | -3.42 pp | Field scoring around the BC-selected checkpoint is worse |
 | Near-miss proposal + field, field checkpoint | 30.14% | +0.41 pp | Deployment-clean route begins to recover the mechanism |
-| Near-miss proposal + field, BC x5 field checkpoint | 32.64% | +2.90 pp | Best deployment-clean result so far |
+| Near-miss proposal + field, BC x5 field checkpoint | 32.93% | +3.19 pp | Best deployment-clean result so far (`k64_sigma0.50`) |
 | Lattice field selection, no expert candidates | 56.99% | +27.25 pp | Conservative, reviewer-safe result |
 | Lattice field selection, near-miss only | 55.94% | +26.20 pp | Minimal local counterfactual proposal family |
 | Lattice field selection, no expert and no near-miss | 25.57% | -4.17 pp | Confirms near-miss proposals carry the gain |
@@ -36,6 +36,7 @@ Canonical summaries:
 - `results/h16_policy_ckpt_near_miss_policy_field_k32_sigma0p35_summary.md`
 - `results/h16_policy_ckpt_near_miss_policy_fieldckpt_field_k32_sigma0p35_summary.md`
 - `results/h16_policy_ckpt_near_miss_policy_bc5_fieldckpt_field_k32_sigma0p35_summary.md`
+- `results/h16_policy_ckpt_near_miss_policy_bc5_bestpt_field_sweep_summary.md`
 
 Non-canonical/stale summaries:
 
@@ -70,8 +71,8 @@ Non-canonical/stale summaries:
 | 14836240 | Summarize near-miss `best.pt` field selection | Completed |
 | 14836241 | Evaluate near-miss BC x5 `best.pt` with field Gaussian selection | Completed |
 | 14836242 | Summarize near-miss BC x5 `best.pt` field selection | Completed |
-| 14837136 | Sweep near-miss BC x5 `best.pt` field selection over K/sigma | Pending |
-| 14837137 | Summarize near-miss BC x5 `best.pt` field sweep | Pending |
+| 14837136 | Sweep near-miss BC x5 `best.pt` field selection over K/sigma | Completed |
+| 14837137 | Summarize near-miss BC x5 `best.pt` field sweep | Completed |
 
 ## Interpretation
 
@@ -93,7 +94,7 @@ Deployment-clean distillation is partially but not fully successful. Directly tr
 policy to imitate near-miss candidates stays below baseline (27.48%, or 28.29% with BC x5).
 Using the BC-selected `best_policy.pt` checkpoint as a Gaussian proposal center also fails
 (26.32%). However, selecting the field-ranked checkpoint `best.pt` and increasing BC weight
-does produce a small clean gain: 32.64%, +2.90 pp over the h=16 policy. This is promising
+does produce a small clean gain: 32.93%, +3.19 pp over the h=16 policy. This is promising
 as a deployment path, but it is still far below the same-state no-expert lattice result
 (56.99%), so the paper should frame it as evidence for the proposal bottleneck rather than
 as the main result.
@@ -107,11 +108,10 @@ as the main result.
 - Negative but useful ablation: Gaussian field search fails, showing the field is not a generic black-box action optimizer off the data manifold.
 - Negative but useful ablation: nearest train-state lattice retrieval fails, showing the gain is not from a generic action library.
 - Negative checkpoint ablation: selecting checkpoint by lower BC validation loss does not improve online rollout.
-- Deployment-clean positive control: near-miss proposal policy plus field selection reaches 32.64% when using the field-selected checkpoint and BC x5, but the gap to 56.99% shows that proposal quality remains the bottleneck.
+- Deployment-clean positive control: near-miss proposal policy plus field selection reaches 32.93% when using the field-selected checkpoint and BC x5, but the gap to 56.99% shows that proposal quality remains the bottleneck.
 
 ## Next Best Experiments
 
 1. Generate an on-the-fly non-expert proposal lattice that does not depend on stored validation candidates, then evaluate with `selection_mode=lattice`.
-2. Finish job `14837136` to tune K/sigma for the promising near-miss BC x5 `best.pt` proposal+field route.
-3. Train a proposal model to imitate successful non-expert lattice candidates so deployment does not require an explicit lattice store.
-4. Add final paper tables separating `policy`, `proposal_field`, `field_lattice_no_expert`, `field_lattice_full`, and `oracle`.
+2. Train a proposal model to imitate successful non-expert lattice candidates so deployment does not require an explicit lattice store.
+3. Add final paper tables separating `policy`, `proposal_field`, `field_lattice_no_expert`, `field_lattice_full`, and `oracle`.

results/h16_policy_ckpt_near_miss_policy_bc5_bestpt_field_sweep_summary.json

@@ -0,0 +1,1150 @@
+{
+  "run_root": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt",
+  "baseline_h4_policy_success": 0.2967,
+  "baseline_h16_policy_success": 0.29739130434782607,
+  "num_result_files": 15,
+  "configs": [
+    {
+      "config": "k64_sigma0.50",
+      "completed_seeds": [
+        0,
+        1,
+        2
+      ],
+      "num_completed": 3,
+      "mean_success": 0.3292753623188406,
+      "std_success": 0.01233843284277841,
+      "mean_progress": 0.5483272921758826,
+      "mean_action_mse_to_best": 0.4194377801362155,
+      "absolute_gain_vs_h4_baseline": 0.03257536231884056,
+      "absolute_gain_vs_h16_policy": 0.03188405797101451
+    },
+    {
+      "config": "k32_sigma0.35",
+      "completed_seeds": [
+        0,
+        1,
+        2
+      ],
+      "num_completed": 3,
+      "mean_success": 0.32811594202898553,
+      "std_success": 0.016159257814221877,
+      "mean_progress": 0.5461846023357074,
+      "mean_action_mse_to_best": 0.4120204289451889,
+      "absolute_gain_vs_h4_baseline": 0.03141594202898551,
+      "absolute_gain_vs_h16_policy": 0.03072463768115946
+    },
+    {
+      "config": "k64_sigma0.35",
+      "completed_seeds": [
+        0,
+        1,
+        2
+      ],
+      "num_completed": 3,
+      "mean_success": 0.32753623188405795,
+      "std_success": 0.020306411464506676,
+      "mean_progress": 0.5457847676730542,
+      "mean_action_mse_to_best": 0.4149229990820522,
+      "absolute_gain_vs_h4_baseline": 0.03083623188405793,
+      "absolute_gain_vs_h16_policy": 0.03014492753623188
+    },
+    {
+      "config": "k16_sigma0.20",
+      "completed_seeds": [
+        0,
+        1,
+        2
+      ],
+      "num_completed": 3,
+      "mean_success": 0.3263768115942029,
+      "std_success": 0.01859595934184978,
+      "mean_progress": 0.5471449069725687,
+      "mean_action_mse_to_best": 0.40453435571999224,
+      "absolute_gain_vs_h4_baseline": 0.02967681159420288,
+      "absolute_gain_vs_h16_policy": 0.02898550724637683
+    },
+    {
+      "config": "k32_sigma0.20",
+      "completed_seeds": [
+        0,
+        1,
+        2
+      ],
+      "num_completed": 3,
+      "mean_success": 0.32231884057971016,
+      "std_success": 0.020748440774693663,
+      "mean_progress": 0.5414390760636103,
+      "mean_action_mse_to_best": 0.40601564239008703,
+      "absolute_gain_vs_h4_baseline": 0.025618840579710145,
+      "absolute_gain_vs_h16_policy": 0.024927536231884095
+    }
+  ],
+  "best": {
+    "config": "k64_sigma0.50",
+    "completed_seeds": [
+      0,
+      1,
+      2
+    ],
+    "num_completed": 3,
+    "mean_success": 0.3292753623188406,
+    "std_success": 0.01233843284277841,
+    "mean_progress": 0.5483272921758826,
+    "mean_action_mse_to_best": 0.4194377801362155,
+    "absolute_gain_vs_h4_baseline": 0.03257536231884056,
+    "absolute_gain_vs_h16_policy": 0.03188405797101451
+  },
+  "rows": [
+    {
+      "config": "k16_sigma0.20",
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k16_sigma0.20/seed_0/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 16,
+      "candidate_sigma": 0.2,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.33043478260869563,
+      "policy_rollout_progress": 0.544193181817764,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.3899358766955202,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.32247859436884213,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.07642500809173,
+          "policy_oracle_regret": 1.1028404858001728,
+          "policy_rollout_progress": 0.5733718855172089,
+          "policy_rollout_success_rate": 0.2268041237113402,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3079392760633849,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 0.9954389036507704,
+          "policy_oracle_regret": 1.0088617710772758,
+          "policy_rollout_progress": 0.601058509406777,
+          "policy_rollout_success_rate": 0.32211538461538464,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6287145626235318,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.28835389711485887,
+          "policy_oracle_regret": 0.4758843815524143,
+          "policy_rollout_progress": 0.31379556818789645,
+          "policy_rollout_success_rate": 0.19480519480519481,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3850305494158826,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.5150462303389894,
+          "policy_oracle_regret": 0.6138637159416016,
+          "policy_rollout_progress": 0.6964525897452172,
+          "policy_rollout_success_rate": 0.6702127659574468,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.44724689957431774,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 0.9836689957765617,
+          "policy_oracle_regret": 1.157828590334064,
+          "policy_rollout_progress": 0.43075792431229293,
+          "policy_rollout_success_rate": 0.23232323232323232,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k16_sigma0.20",
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k16_sigma0.20/seed_1/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 16,
+      "candidate_sigma": 0.2,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3060869565217391,
+      "policy_rollout_progress": 0.5316903001021431,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.39861446821495244,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.33289363966282226,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 1.048153557608613,
+          "policy_oracle_regret": 1.1386048915111913,
+          "policy_rollout_progress": 0.5989050643633952,
+          "policy_rollout_success_rate": 0.19469026548672566,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3264182775443339,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.1100710043076263,
+          "policy_oracle_regret": 1.1100950087825565,
+          "policy_rollout_progress": 0.5545998569780394,
+          "policy_rollout_success_rate": 0.266304347826087,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.5894409823966654,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.3966844152708195,
+          "policy_oracle_regret": 0.66795516218745,
+          "policy_rollout_progress": 0.2253140292596072,
+          "policy_rollout_success_rate": 0.10526315789473684,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3773898981377348,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.3984359223563392,
+          "policy_oracle_regret": 0.4819212639117026,
+          "policy_rollout_progress": 0.7703309883405497,
+          "policy_rollout_success_rate": 0.7477477477477478,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.4927206760231446,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.1646305985830643,
+          "policy_oracle_regret": 1.3239166384542382,
+          "policy_rollout_progress": 0.36668868870525567,
+          "policy_rollout_success_rate": 0.15384615384615385,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k16_sigma0.20",
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k16_sigma0.20/seed_2/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 16,
+      "candidate_sigma": 0.2,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3426086956521739,
+      "policy_rollout_progress": 0.5655512389977989,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.425052722249504,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.3572826692106901,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.8819095164847871,
+          "policy_oracle_regret": 0.9881842317990959,
+          "policy_rollout_progress": 0.6433409581271311,
+          "policy_rollout_success_rate": 0.3020833333333333,
+          "restore_max_error": 3.5762786865234375e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.33731283482653324,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.0088076980295355,
+          "policy_oracle_regret": 1.015891148558244,
+          "policy_rollout_progress": 0.6228325308554552,
+          "policy_rollout_success_rate": 0.32323232323232326,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6599690684013897,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.3202306208821634,
+          "policy_oracle_regret": 0.5816346477613681,
+          "policy_rollout_progress": 0.31432361036746037,
+          "policy_rollout_success_rate": 0.2,
+          "restore_max_error": 4.0978193283081055e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.39746504446657577,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.4742610312924527,
+          "policy_oracle_regret": 0.5289383319639923,
+          "policy_rollout_progress": 0.7482893908082848,
+          "policy_rollout_success_rate": 0.7227722772277227,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.4864116900703973,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.1857868118418589,
+          "policy_oracle_regret": 1.3348589769668049,
+          "policy_rollout_progress": 0.40271151032712726,
+          "policy_rollout_success_rate": 0.14444444444444443,
+          "restore_max_error": 4.76837158203125e-07
+        }
+      }
+    },
+    {
+      "config": "k32_sigma0.20",
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k32_sigma0.20/seed_0/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 32,
+      "candidate_sigma": 0.2,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.30782608695652175,
+      "policy_rollout_progress": 0.5236520005676198,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.3907121669816906,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.32411609940979746,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.096901917734097,
+          "policy_oracle_regret": 1.12331739544254,
+          "policy_rollout_progress": 0.5632042542253572,
+          "policy_rollout_success_rate": 0.21649484536082475,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3089638287559725,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 0.9997012414521753,
+          "policy_oracle_regret": 1.0131241088786807,
+          "policy_rollout_progress": 0.594855451822066,
+          "policy_rollout_success_rate": 0.3173076923076923,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.620038845709392,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.34222419101954143,
+          "policy_oracle_regret": 0.6057183034637363,
+          "policy_rollout_progress": 0.21818976864638404,
+          "policy_rollout_success_rate": 0.09090909090909091,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3883775261012798,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.5517117910562678,
+          "policy_oracle_regret": 0.6486242381816215,
+          "policy_rollout_progress": 0.6828119799177698,
+          "policy_rollout_success_rate": 0.648936170212766,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.4515684789629898,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 1.0139289311688355,
+          "policy_oracle_regret": 1.1769277733383756,
+          "policy_rollout_progress": 0.4217597514089912,
+          "policy_rollout_success_rate": 0.2222222222222222,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k32_sigma0.20",
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k32_sigma0.20/seed_1/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 32,
+      "candidate_sigma": 0.2,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3130434782608696,
+      "policy_rollout_progress": 0.5412584817804316,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.4014194884442765,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.3384467677122592,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 1.025868605855292,
+          "policy_oracle_regret": 1.105070760951633,
+          "policy_rollout_progress": 0.6147400798787058,
+          "policy_rollout_success_rate": 0.21238938053097345,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.32736685282915184,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.11036436841317,
+          "policy_oracle_regret": 1.1103883728881,
+          "policy_rollout_progress": 0.552188413313833,
+          "policy_rollout_success_rate": 0.2608695652173913,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.593048064920463,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.29840909580142533,
+          "policy_oracle_regret": 0.5478567443129386,
+          "policy_rollout_progress": 0.28396381365937967,
+          "policy_rollout_success_rate": 0.17105263157894737,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3798570418814281,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.4019931071513408,
+          "policy_oracle_regret": 0.4871569598043287,
+          "policy_rollout_progress": 0.7650952924479235,
+          "policy_rollout_success_rate": 0.7477477477477478,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.4956091824243521,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.183210695019135,
+          "policy_oracle_regret": 1.342819583776233,
+          "policy_rollout_progress": 0.36976376536128286,
+          "policy_rollout_success_rate": 0.13186813186813187,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k32_sigma0.20",
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k32_sigma0.20/seed_2/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 32,
+      "candidate_sigma": 0.2,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.34608695652173915,
+      "policy_rollout_progress": 0.5594067458427795,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.425915271744294,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.35779982555444195,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.8545110411942005,
+          "policy_oracle_regret": 0.9607857565085093,
+          "policy_rollout_progress": 0.6394894334177176,
+          "policy_rollout_success_rate": 0.3333333333333333,
+          "restore_max_error": 3.5762786865234375e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3381687065755779,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.00870233607676,
+          "policy_oracle_regret": 1.0157857866054683,
+          "policy_rollout_progress": 0.6229378928082309,
+          "policy_rollout_success_rate": 0.32323232323232326,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6543993772731886,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.3404962347147779,
+          "policy_oracle_regret": 0.5860447995105965,
+          "policy_rollout_progress": 0.3017854992972894,
+          "policy_rollout_success_rate": 0.18888888888888888,
+          "restore_max_error": 4.0978193283081055e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.4016442125534067,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.47577363680494894,
+          "policy_oracle_regret": 0.5299919200415658,
+          "policy_rollout_progress": 0.7472358027307113,
+          "policy_rollout_success_rate": 0.7227722772277227,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.49036760750330155,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.1946870706147619,
+          "policy_oracle_regret": 1.356807397802671,
+          "policy_rollout_progress": 0.38105199403233,
+          "policy_rollout_success_rate": 0.14444444444444443,
+          "restore_max_error": 4.76837158203125e-07
+        }
+      }
+    },
+    {
+      "config": "k32_sigma0.35",
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k32_sigma0.35/seed_0/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 32,
+      "candidate_sigma": 0.35,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3234782608695652,
+      "policy_rollout_progress": 0.5365680560134893,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.39553123654797673,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.3222495113672285,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.0924583412322801,
+          "policy_oracle_regret": 1.118873818940723,
+          "policy_rollout_progress": 0.5676478307271741,
+          "policy_rollout_success_rate": 0.21649484536082475,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.31544953831829703,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 0.9939068040470235,
+          "policy_oracle_regret": 1.007329671473529,
+          "policy_rollout_progress": 0.5977829167028316,
+          "policy_rollout_success_rate": 0.3269230769230769,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6550499460139831,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.2455574702068305,
+          "policy_oracle_regret": 0.4835427300238486,
+          "policy_rollout_progress": 0.29021572308752125,
+          "policy_rollout_success_rate": 0.16883116883116883,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3813233734603892,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.5716793151929024,
+          "policy_oracle_regret": 0.6685689863689402,
+          "policy_rollout_progress": 0.6824948434182938,
+          "policy_rollout_success_rate": 0.648936170212766,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.44722738882733715,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 0.9838731827759983,
+          "policy_oracle_regret": 1.1580327773335004,
+          "policy_rollout_progress": 0.43055373731285634,
+          "policy_rollout_success_rate": 0.23232323232323232,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k32_sigma0.35",
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k32_sigma0.35/seed_1/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 32,
+      "candidate_sigma": 0.35,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.31478260869565217,
+      "policy_rollout_progress": 0.5412338370824422,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.4067689944188232,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.33181332383898243,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 1.004089695426215,
+          "policy_oracle_regret": 1.0832918505225562,
+          "policy_rollout_progress": 0.6276694327856587,
+          "policy_rollout_success_rate": 0.22123893805309736,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.33753558987265697,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.1125094104487367,
+          "policy_oracle_regret": 1.112533414923667,
+          "policy_rollout_progress": 0.5523134896521578,
+          "policy_rollout_success_rate": 0.266304347826087,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6109341694728324,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.3739064343829165,
+          "policy_oracle_regret": 0.6021578795881124,
+          "policy_rollout_progress": 0.25026103445704573,
+          "policy_rollout_success_rate": 0.14473684210526316,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3871889783254078,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.4335025233459902,
+          "policy_oracle_regret": 0.4999616445453317,
+          "policy_rollout_progress": 0.7612996167159295,
+          "policy_rollout_success_rate": 0.7387387387387387,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.49320598007026284,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.1323802955858,
+          "policy_oracle_regret": 1.3045280497152727,
+          "policy_rollout_progress": 0.38607727744422116,
+          "policy_rollout_success_rate": 0.15384615384615385,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k32_sigma0.35",
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k32_sigma0.35/seed_2/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 32,
+      "candidate_sigma": 0.35,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.34608695652173915,
+      "policy_rollout_progress": 0.5607519139111908,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.4337610558687669,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.3546609722592014,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.8603796567767859,
+          "policy_oracle_regret": 0.9666543720910946,
+          "policy_rollout_progress": 0.644037484501799,
+          "policy_rollout_success_rate": 0.3229166666666667,
+          "restore_max_error": 3.5762786865234375e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.34091578990296284,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.0118255193599246,
+          "policy_oracle_regret": 1.0189089698886329,
+          "policy_rollout_progress": 0.6198147095250662,
+          "policy_rollout_success_rate": 0.32323232323232326,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.7086632399095429,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.3871518444545826,
+          "policy_oracle_regret": 0.6527724535790842,
+          "policy_rollout_progress": 0.26429835981517097,
+          "policy_rollout_success_rate": 0.15555555555555556,
+          "restore_max_error": 4.0978193283081055e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.39258715377585723,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.40123750564485494,
+          "policy_oracle_regret": 0.49343014014239356,
+          "policy_rollout_progress": 0.7639956024318638,
+          "policy_rollout_success_rate": 0.7425742574257426,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.49369814737389484,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.155931308037705,
+          "policy_oracle_regret": 1.305003473162651,
+          "policy_rollout_progress": 0.41034479190905887,
+          "policy_rollout_success_rate": 0.16666666666666666,
+          "restore_max_error": 4.76837158203125e-07
+        }
+      }
+    },
+    {
+      "config": "k64_sigma0.35",
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k64_sigma0.35/seed_0/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 64,
+      "candidate_sigma": 0.35,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3234782608695652,
+      "policy_rollout_progress": 0.5343159465103726,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.39918596936227835,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.32200342137846594,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.0924534333735396,
+          "policy_oracle_regret": 1.1188689110819827,
+          "policy_rollout_progress": 0.5676527385859146,
+          "policy_rollout_success_rate": 0.21649484536082475,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.32040938953510845,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 1.0003741416238392,
+          "policy_oracle_regret": 1.0137970090503445,
+          "policy_rollout_progress": 0.5913155791260158,
+          "policy_rollout_success_rate": 0.3269230769230769,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6566622284325686,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.2460009390846041,
+          "policy_oracle_regret": 0.4629497372528128,
+          "policy_rollout_progress": 0.29306630275033213,
+          "policy_rollout_success_rate": 0.18181818181818182,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.39111159397091005,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.5936297910644653,
+          "policy_oracle_regret": 0.6903621123192158,
+          "policy_rollout_progress": 0.6713400153403587,
+          "policy_rollout_success_rate": 0.6382978723404256,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.44772692993659563,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 0.9749961484863301,
+          "policy_oracle_regret": 1.1491557430438322,
+          "policy_rollout_progress": 0.43943077160252464,
+          "policy_rollout_success_rate": 0.23232323232323232,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k64_sigma0.35",
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k64_sigma0.35/seed_1/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 64,
+      "candidate_sigma": 0.35,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3095652173913043,
+      "policy_rollout_progress": 0.536779355444554,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.4101900245184484,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.33171445011795886,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 1.0022590173556742,
+          "policy_oracle_regret": 1.0813931261016205,
+          "policy_rollout_progress": 0.6295681572065944,
+          "policy_rollout_success_rate": 0.22123893805309736,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3401717414229136,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.1126799705215609,
+          "policy_oracle_regret": 1.1127039749964909,
+          "policy_rollout_progress": 0.5521429295793338,
+          "policy_rollout_success_rate": 0.266304347826087,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6175777537650183,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.3696385342464373,
+          "policy_oracle_regret": 0.6190723324227227,
+          "policy_rollout_progress": 0.24086668692006366,
+          "policy_rollout_success_rate": 0.11842105263157894,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3934205474483,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.4525444950606372,
+          "policy_oracle_regret": 0.5380019815386953,
+          "policy_rollout_progress": 0.741277297740584,
+          "policy_rollout_success_rate": 0.7207207207207207,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.49646528307757865,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.131820830506283,
+          "policy_oracle_regret": 1.2914297192633808,
+          "policy_rollout_progress": 0.38818659690710217,
+          "policy_rollout_success_rate": 0.16483516483516483,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k64_sigma0.35",
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k64_sigma0.35/seed_2/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 64,
+      "candidate_sigma": 0.35,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.34956521739130436,
+      "policy_rollout_progress": 0.5662590010642358,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.43539300336542985,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.3568755660720247,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.8741882933924595,
+          "policy_oracle_regret": 0.9804630087067684,
+          "policy_rollout_progress": 0.640645514552792,
+          "policy_rollout_success_rate": 0.3125,
+          "restore_max_error": 3.5762786865234375e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3404735067313669,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.0108400425600885,
+          "policy_oracle_regret": 1.0179234930887968,
+          "policy_rollout_progress": 0.6208001863249023,
+          "policy_rollout_success_rate": 0.32323232323232326,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.7115254996551408,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.38466353309858176,
+          "policy_oracle_regret": 0.6054818535947966,
+          "policy_rollout_progress": 0.30913451097698674,
+          "policy_rollout_success_rate": 0.2,
+          "restore_max_error": 4.0978193283081055e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.396122483481275,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.43801331372544317,
+          "policy_oracle_regret": 0.5106629358954949,
+          "policy_rollout_progress": 0.7566637967777724,
+          "policy_rollout_success_rate": 0.7326732673267327,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.4959055828758412,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.155905323723952,
+          "policy_oracle_regret": 1.304977488848898,
+          "policy_rollout_progress": 0.4103707762228118,
+          "policy_rollout_success_rate": 0.16666666666666666,
+          "restore_max_error": 4.76837158203125e-07
+        }
+      }
+    },
+    {
+      "config": "k64_sigma0.50",
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k64_sigma0.50/seed_0/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 64,
+      "candidate_sigma": 0.5,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3269565217391304,
+      "policy_rollout_progress": 0.5399010283508054,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.4025516371615231,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.32251464834283156,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.0791288545143973,
+          "policy_oracle_regret": 1.1055443322228402,
+          "policy_rollout_progress": 0.5707339151311166,
+          "policy_rollout_success_rate": 0.2268041237113402,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3262789388760351,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 0.9832543106995917,
+          "policy_oracle_regret": 0.9966771781260971,
+          "policy_rollout_progress": 0.6036277177425711,
+          "policy_rollout_success_rate": 0.3317307692307692,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6741709269873508,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.21915740969699699,
+          "policy_oracle_regret": 0.43092472498099527,
+          "policy_rollout_progress": 0.32208714377763986,
+          "policy_rollout_success_rate": 0.2077922077922078,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3830847614543869,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.6339274873124793,
+          "policy_oracle_regret": 0.7308171584885171,
+          "policy_rollout_progress": 0.6521615649157382,
+          "policy_rollout_success_rate": 0.6170212765957447,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.448445679168358,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 0.9758063234163054,
+          "policy_oracle_regret": 1.1499659179738073,
+          "policy_rollout_progress": 0.4386205966725494,
+          "policy_rollout_success_rate": 0.23232323232323232,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k64_sigma0.50",
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k64_sigma0.50/seed_1/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 64,
+      "candidate_sigma": 0.5,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3182608695652174,
+      "policy_rollout_progress": 0.5455222901711647,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.41242632866877577,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.3303305860361562,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 0.9963025565432236,
+          "policy_oracle_regret": 1.0867589768846477,
+          "policy_rollout_progress": 0.624202306423567,
+          "policy_rollout_success_rate": 0.22123893805309736,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.33823254261084873,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.0973767103475474,
+          "policy_oracle_regret": 1.0974007148224774,
+          "policy_rollout_progress": 0.5641294867033144,
+          "policy_rollout_success_rate": 0.27717391304347827,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6391991545495234,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.3485522249125336,
+          "policy_oracle_regret": 0.5971745297214702,
+          "policy_rollout_progress": 0.2562023993875635,
+          "policy_rollout_success_rate": 0.14473684210526316,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3912716278688865,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.4345502409043613,
+          "policy_oracle_regret": 0.5010873861946501,
+          "policy_rollout_progress": 0.7601738750666112,
+          "policy_rollout_success_rate": 0.7387387387387387,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.5007989943170776,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.1284581424443276,
+          "policy_oracle_regret": 1.3006058965738003,
+          "policy_rollout_progress": 0.38999943058569353,
+          "policy_rollout_success_rate": 0.15384615384615385,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "config": "k64_sigma0.50",
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt/k64_sigma0.50/seed_2/online_rollout.json",
+      "selection_mode": "field",
+      "num_candidates": 64,
+      "candidate_sigma": 0.5,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.3426086956521739,
+      "policy_rollout_progress": 0.5595585580056777,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.44333537457834765,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.35361564260771655,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.8470402403424183,
+          "policy_oracle_regret": 0.9533149556567272,
+          "policy_rollout_progress": 0.6469602342694998,
+          "policy_rollout_success_rate": 0.3333333333333333,
+          "restore_max_error": 3.5762786865234375e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.34493389291067916,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.007205421227322,
+          "policy_oracle_regret": 1.0142888717560306,
+          "policy_rollout_progress": 0.6244348076576687,
+          "policy_rollout_success_rate": 0.32323232323232326,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.75293321294917,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.4486654932610691,
+          "policy_oracle_regret": 0.7107819825928244,
+          "policy_rollout_progress": 0.250149907451123,
+          "policy_rollout_success_rate": 0.13333333333333333,
+          "restore_max_error": 4.0978193283081055e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.3872078667477806,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.4194727140192938,
+          "policy_oracle_regret": 0.5103210249159595,
+          "policy_rollout_progress": 0.7570057077573078,
+          "policy_rollout_success_rate": 0.7326732673267327,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.50890937987715,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.1548446744680405,
+          "policy_oracle_regret": 1.3039168395929865,
+          "policy_rollout_progress": 0.4114314254787233,
+          "policy_rollout_success_rate": 0.16666666666666666,
+          "restore_max_error": 4.76837158203125e-07
+        }
+      }
+    }
+  ]
+}

results/h16_policy_ckpt_near_miss_policy_bc5_bestpt_field_sweep_summary.md

@@ -0,0 +1,19 @@
+# h=16 Field-Guided Rollout Sweep
+
+Result root: `/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_field_sweep/near_miss_policy_bc5_bestpt`
+Completed result files: 15
+Baseline h=4 policy success: 29.67%
+Baseline h=16 policy success: 29.74%
+
+| config | seeds | mean success | gain vs h=16 | progress | action MSE |
+|---|---:|---:|---:|---:|---:|
+| k64_sigma0.50 | 3 | 32.93% | +3.19% | 54.83% | 0.419 |
+| k32_sigma0.35 | 3 | 32.81% | +3.07% | 54.62% | 0.412 |
+| k64_sigma0.35 | 3 | 32.75% | +3.01% | 54.58% | 0.415 |
+| k16_sigma0.20 | 3 | 32.64% | +2.90% | 54.71% | 0.405 |
+| k32_sigma0.20 | 3 | 32.23% | +2.49% | 54.14% | 0.406 |
+
+Best config:
+- k64_sigma0.50
+- mean success: 32.93%
+- gain vs h=16 policy: +3.19%