Auto-sync: 2026-06-28 00:54:07 (part 2)

results/README.md

@@ -38,7 +38,7 @@ Completed sweep:
 
 Pending field-optimization jobs:
 
-Last checked: `2026-06-28 04:44 UTC`.
+Last checked: `2026-06-28 04:51 UTC`.
 
 - Job `14842523` is a small runtime smoke test for `selection_mode=field_optim`
   (`MAX_GROUPS=64`, `near_miss_policy_bc5` + `best.pt`); it is pending on
@@ -98,6 +98,7 @@ on the project `.venv` and include `retrieval_neighbors`.
 - Job `14857114` reruns KNN4 `retrieval_residual` smoke, `14857115` runs full
   validation after that smoke, and `14857116` summarizes it.
 - Job `14857117` rebuilds `paper_table_status.*` after both residual summaries.
+  It completed successfully.
 - Jobs `14857692`/`14857693`/`14857694` evaluate the nearest-1
   `retrieval_residual` diagnostic restricted to transferred `residual_near_miss`
   proposals.
@@ -105,6 +106,9 @@ on the project `.venv` and include `retrieval_neighbors`.
   near-miss residual diagnostic.
 - Job `14857698` rebuilds `paper_table_status.*` after those near-miss residual
   summaries.
+- Nearest-1 residual v2 completed at 32.12% success; KNN4 residual completed at
+  29.91%. The near-miss-only residual smokes reached only 14.06%, so the full
+  near-miss-only residual jobs were canceled.
 
 Paper table automation:
 

results/dovla_cil_run_report_2026-06-27.md

@@ -97,21 +97,21 @@ Non-canonical/stale summaries:
 | 14842619 | Patched summary for KNN `retrieval_residual` rollout | Pending |
 | 14842646 | CPU unit smoke for KNN `retrieval_residual` helper | Completed |
 | 14857111 | Rerun fixed nearest-1 `retrieval_residual` smoke | Pending |
-| 14857112 | Rerun fixed nearest-1 `retrieval_residual` full rollout | Pending |
-| 14857113 | Summarize fixed nearest-1 `retrieval_residual` rollout | Pending |
-| 14857114 | Rerun fixed KNN4 `retrieval_residual` smoke | Pending |
-| 14857115 | Rerun fixed KNN4 `retrieval_residual` full rollout | Pending |
-| 14857116 | Summarize fixed KNN4 `retrieval_residual` rollout | Pending |
-| 14857117 | Rebuild paper table after fixed residual summaries | Pending |
-| 14857692 | Smoke-test nearest-1 transferred near-miss residual retrieval | Running |
-| 14857693 | Evaluate nearest-1 transferred near-miss residual retrieval | Pending |
-| 14857694 | Summarize nearest-1 transferred near-miss residual retrieval | Pending |
-| 14857695 | Smoke-test KNN4 transferred near-miss residual retrieval | Running |
-| 14857696 | Evaluate KNN4 transferred near-miss residual retrieval | Pending |
-| 14857697 | Summarize KNN4 transferred near-miss residual retrieval | Pending |
-| 14857698 | Rebuild paper table after near-miss residual summaries | Pending |
-
-Last queue check: `2026-06-28 04:44 UTC`. The earlier queue backlog cleared.
+| 14857112 | Rerun fixed nearest-1 `retrieval_residual` full rollout | Completed |
+| 14857113 | Summarize fixed nearest-1 `retrieval_residual` rollout | Completed locally and via Slurm |
+| 14857114 | Rerun fixed KNN4 `retrieval_residual` smoke | Completed |
+| 14857115 | Rerun fixed KNN4 `retrieval_residual` full rollout | Completed |
+| 14857116 | Summarize fixed KNN4 `retrieval_residual` rollout | Completed |
+| 14857117 | Rebuild paper table after fixed residual summaries | Completed |
+| 14857692 | Smoke-test nearest-1 transferred near-miss residual retrieval | Completed |
+| 14857693 | Evaluate nearest-1 transferred near-miss residual retrieval | Canceled after weak smoke |
+| 14857694 | Summarize nearest-1 transferred near-miss residual retrieval | Canceled |
+| 14857695 | Smoke-test KNN4 transferred near-miss residual retrieval | Completed |
+| 14857696 | Evaluate KNN4 transferred near-miss residual retrieval | Canceled after weak smoke |
+| 14857697 | Summarize KNN4 transferred near-miss residual retrieval | Canceled |
+| 14857698 | Rebuild paper table after near-miss residual summaries | Canceled |
+
+Last queue check: `2026-06-28 04:51 UTC`. The earlier queue backlog cleared.
 `field_optim`, `nonexpert_policy_bc5`, and their summaries completed; fixed
 `retrieval_residual` v2 jobs are now pending on priority/dependencies.
 
@@ -147,8 +147,13 @@ failed because the evaluator did not pass `retrieval_neighbors` into
 `14857111`-`14857117` were submitted.
 The first fixed residual smokes passed and full rollouts started. Because the
 smoke JSON showed the field often selected `policy_residual` and non-near-miss
-residuals, jobs `14857692`-`14857698` test the sharper diagnostic where only
-transferred `residual_near_miss` proposals remain eligible.
+residuals, jobs `14857692` and `14857695` tested a sharper smoke diagnostic
+where only transferred `residual_near_miss` proposals remain eligible.
+Nearest-1 residual retrieval completed at 32.12% (+2.38 pp over h=16 policy),
+close to but below the clean best 32.93%. KNN4 residual retrieval completed at
+29.91%, indicating that adding more retrieved tangent neighborhoods dilutes the
+signal. The near-miss-only residual smoke was only 14.06%, so the full
+near-miss-only residual jobs were canceled.
 
 Paper table automation now lives in `scripts/build_paper_table_status.py` with a
 CPU wrapper at `scripts/slurm/build_paper_table_status.sbatch`. The table builder

results/h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_knn4_v2_summary.json

@@ -0,0 +1,277 @@
+{
+  "run_root": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs",
+  "objective": "near_miss_policy_bc5",
+  "out_name": "retrieval_residual_knn4_v2_rollout.json",
+  "num_completed": 3,
+  "baseline_h4_policy_success": 0.2967,
+  "baseline_h16_rank_checkpoint_success": 0.29739130434782607,
+  "mean_success": 0.2991304347826087,
+  "std_success": 0.02005663059942746,
+  "mean_progress": 0.5339705883982875,
+  "mean_action_mse_to_best": 0.6740692569633973,
+  "gain_vs_h4": 0.0024304347826086814,
+  "gain_vs_h16_rank_checkpoint": 0.001739130434782632,
+  "rows": [
+    {
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs/near_miss_policy_bc5/seed_0/retrieval_residual_knn4_v2_rollout.json",
+      "num_groups": 575,
+      "selection_mode": "retrieval_residual",
+      "num_candidates": 64,
+      "candidate_sigma": 0.0,
+      "field_optim_steps": 0,
+      "field_optim_step_size": 0.0,
+      "field_optim_trust_radius": 0.0,
+      "field_optim_l2_penalty": 0.0,
+      "retrieval_neighbors": 4,
+      "policy_rollout_success_rate": 0.2782608695652174,
+      "policy_rollout_progress": 0.5109179835960916,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.8115562080672901,
+      "best_policy_val": {
+        "bc_loss": 0.13721593966086706,
+        "field_effect_loss": 0.009290305380192068,
+        "field_potential_loss": 0.2666468388504452,
+        "field_preference_loss": 0.5130573478009965,
+        "lattice_edges": 3833.3333333333335,
+        "progress_mae": 0.1933159919248687,
+        "rank_acc": 0.8265031774838766,
+        "rank_loss": 0.5130523675017886,
+        "regret_mae": 0.3756548762321472,
+        "success_accuracy": 0.8773836526605818,
+        "total_loss": 1.5581054819954767
+      },
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 1.4539887857882632,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.32526028494245,
+          "policy_oracle_regret": 1.3648747405440538,
+          "policy_rollout_progress": 0.44535824933002904,
+          "policy_rollout_success_rate": 0.09278350515463918,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.43021411278123894,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 1.0523346415452228,
+          "policy_oracle_regret": 1.0657575089717284,
+          "policy_rollout_progress": 0.5662605132674798,
+          "policy_rollout_success_rate": 0.2932692307692308,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.7097426858041194,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.2187154905797986,
+          "policy_oracle_regret": 0.4380442425608635,
+          "policy_rollout_progress": 0.3043108030960157,
+          "policy_rollout_success_rate": 0.19480519480519481,
+          "restore_max_error": 2.3096799850463867e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.4324036994354522,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.5528638410441419,
+          "policy_oracle_regret": 0.6304144875166264,
+          "policy_rollout_progress": 0.6993727465259268,
+          "policy_rollout_success_rate": 0.6702127659574468,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 1.4224973475240699,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 1.0807228234380182,
+          "policy_oracle_regret": 1.2590619327142984,
+          "policy_rollout_progress": 0.44063569304316935,
+          "policy_rollout_success_rate": 0.12121212121212122,
+          "restore_max_error": 2.384185791015625e-07
+        }
+      }
+    },
+    {
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs/near_miss_policy_bc5/seed_1/retrieval_residual_knn4_v2_rollout.json",
+      "num_groups": 575,
+      "selection_mode": "retrieval_residual",
+      "num_candidates": 64,
+      "candidate_sigma": 0.0,
+      "field_optim_steps": 0,
+      "field_optim_step_size": 0.0,
+      "field_optim_trust_radius": 0.0,
+      "field_optim_l2_penalty": 0.0,
+      "retrieval_neighbors": 4,
+      "policy_rollout_success_rate": 0.3008695652173913,
+      "policy_rollout_progress": 0.5379841797219832,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.6072114229510012,
+      "best_policy_val": {
+        "bc_loss": 0.1315989200439718,
+        "field_effect_loss": 0.010110370814800262,
+        "field_potential_loss": 0.31546536915832096,
+        "field_preference_loss": 0.4960637440284093,
+        "lattice_edges": 3833.3333333333335,
+        "progress_mae": 0.19023226698239645,
+        "rank_acc": 0.8255469501018524,
+        "rank_loss": 0.4960531195004781,
+        "regret_mae": 0.42156532737943864,
+        "success_accuracy": 0.8807473679383596,
+        "total_loss": 1.561984618504842
+      },
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.5623966725048634,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 1.090754540620652,
+          "policy_oracle_regret": 1.1814605609505577,
+          "policy_rollout_progress": 0.5801568753951418,
+          "policy_rollout_success_rate": 0.1592920353982301,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.5474449685450805,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.1541079988060317,
+          "policy_oracle_regret": 1.1541320032809619,
+          "policy_rollout_progress": 0.5378125531107957,
+          "policy_rollout_success_rate": 0.2391304347826087,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.7271176308001343,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.32360691885084014,
+          "policy_oracle_regret": 0.5946264473237343,
+          "policy_rollout_progress": 0.2581635246537381,
+          "policy_rollout_success_rate": 0.14473684210526316,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.4844838827177211,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.3500694275439323,
+          "policy_oracle_regret": 0.4335452922292658,
+          "policy_rollout_progress": 0.7916799329959594,
+          "policy_rollout_success_rate": 0.7747747747747747,
+          "restore_max_error": 2.3795291781425476e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.8332661436154292,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.120737257894579,
+          "policy_oracle_regret": 1.2803991050510617,
+          "policy_rollout_progress": 0.4102062221084322,
+          "policy_rollout_success_rate": 0.15384615384615385,
+          "restore_max_error": 1.9371509552001953e-07
+        }
+      }
+    },
+    {
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs/near_miss_policy_bc5/seed_2/retrieval_residual_knn4_v2_rollout.json",
+      "num_groups": 575,
+      "selection_mode": "retrieval_residual",
+      "num_candidates": 64,
+      "candidate_sigma": 0.0,
+      "field_optim_steps": 0,
+      "field_optim_step_size": 0.0,
+      "field_optim_trust_radius": 0.0,
+      "field_optim_l2_penalty": 0.0,
+      "retrieval_neighbors": 4,
+      "policy_rollout_success_rate": 0.3182608695652174,
+      "policy_rollout_progress": 0.5530096018767875,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.6034401398719005,
+      "best_policy_val": {
+        "bc_loss": 0.11367896075050037,
+        "field_effect_loss": 0.009670218582161598,
+        "field_potential_loss": 0.2641640139950646,
+        "field_preference_loss": 0.5130490180518892,
+        "lattice_edges": 3833.3333333333335,
+        "progress_mae": 0.2021729110015763,
+        "rank_acc": 0.8333857821093665,
+        "rank_loss": 0.5130119257503085,
+        "regret_mae": 0.3958987047274907,
+        "success_accuracy": 0.8680730561415354,
+        "total_loss": 1.4394984311527677
+      },
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.4755813370817729,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.9551507406868041,
+          "policy_oracle_regret": 1.0750064556486905,
+          "policy_rollout_progress": 0.5981854009442031,
+          "policy_rollout_success_rate": 0.2604166666666667,
+          "restore_max_error": 1.955777406692505e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.511378369145942,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.1163382293995132,
+          "policy_oracle_regret": 1.1305679249285598,
+          "policy_rollout_progress": 0.5797244202665458,
+          "policy_rollout_success_rate": 0.23737373737373738,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.7437843493289418,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.2959972145035863,
+          "policy_oracle_regret": 0.5239301502083739,
+          "policy_rollout_progress": 0.32425515037029984,
+          "policy_rollout_success_rate": 0.2222222222222222,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.43701646425346335,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.3824794600800713,
+          "policy_oracle_regret": 0.4372355530757715,
+          "policy_rollout_progress": 0.7904872192014561,
+          "policy_rollout_success_rate": 0.7722772277227723,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.9887788960710168,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.1929329228070047,
+          "policy_oracle_regret": 1.3292692128154966,
+          "policy_rollout_progress": 0.4083012744784355,
+          "policy_rollout_success_rate": 0.14444444444444443,
+          "restore_max_error": 2.2351741790771484e-07
+        }
+      }
+    }
+  ]
+}

results/h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_knn4_v2_summary.md

@@ -0,0 +1,19 @@
+# h=16 Best-Policy Checkpoint Rollout
+
+Run root: `/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs`
+Objective: `near_miss_policy_bc5`
+Result file: `retrieval_residual_knn4_v2_rollout.json`
+Completed seeds: 3
+Baseline h=4 policy success: 29.67%
+Baseline h=16 rank-checkpoint success: 29.74%
+
+Mean success: 29.91% +/- 2.01%
+Gain vs h=16 rank checkpoint: +0.17%
+Mean progress: 53.40%
+Mean action MSE to best: 0.674
+
+| seed | mode | k | retrieval K | sigma | opt steps | trust | success | progress | oracle | action MSE |
+|---:|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| 0 | retrieval_residual | 64 | 4 | 0.00 | 0 | 0.00 | 27.83% | 51.09% | 85.74% | 0.812 |
+| 1 | retrieval_residual | 64 | 4 | 0.00 | 0 | 0.00 | 30.09% | 53.80% | 86.96% | 0.607 |
+| 2 | retrieval_residual | 64 | 4 | 0.00 | 0 | 0.00 | 31.83% | 55.30% | 87.65% | 0.603 |

results/h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_v2_summary.json

@@ -0,0 +1,277 @@
+{
+  "run_root": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs",
+  "objective": "near_miss_policy_bc5",
+  "out_name": "retrieval_residual_v2_rollout.json",
+  "num_completed": 3,
+  "baseline_h4_policy_success": 0.2967,
+  "baseline_h16_rank_checkpoint_success": 0.29739130434782607,
+  "mean_success": 0.32115942028985506,
+  "std_success": 0.012581179370556922,
+  "mean_progress": 0.5482713204283484,
+  "mean_action_mse_to_best": 0.5593383192142769,
+  "gain_vs_h4": 0.02445942028985504,
+  "gain_vs_h16_rank_checkpoint": 0.02376811594202899,
+  "rows": [
+    {
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs/near_miss_policy_bc5/seed_0/retrieval_residual_v2_rollout.json",
+      "num_groups": 575,
+      "selection_mode": "retrieval_residual",
+      "num_candidates": 16,
+      "candidate_sigma": 0.0,
+      "field_optim_steps": 0,
+      "field_optim_step_size": 0.0,
+      "field_optim_trust_radius": 0.0,
+      "field_optim_l2_penalty": 0.0,
+      "retrieval_neighbors": 1,
+      "policy_rollout_success_rate": 0.31478260869565217,
+      "policy_rollout_progress": 0.5324433289079562,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.6325278520742026,
+      "best_policy_val": {
+        "bc_loss": 0.13721593966086706,
+        "field_effect_loss": 0.009290305380192068,
+        "field_potential_loss": 0.2666468388504452,
+        "field_preference_loss": 0.5130573478009965,
+        "lattice_edges": 3833.3333333333335,
+        "progress_mae": 0.1933159919248687,
+        "rank_acc": 0.8265031774838766,
+        "rank_loss": 0.5130523675017886,
+        "regret_mae": 0.3756548762321472,
+        "success_accuracy": 0.8773836526605818,
+        "total_loss": 1.5581054819954767
+      },
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 1.1169110205139696,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.2637317341934775,
+          "policy_oracle_regret": 1.303346189795081,
+          "policy_rollout_progress": 0.4759589650274552,
+          "policy_rollout_success_rate": 0.12371134020618557,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3548992721334798,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 0.9826834486403431,
+          "policy_oracle_regret": 0.9961063160668485,
+          "policy_rollout_progress": 0.6041985798018197,
+          "policy_rollout_success_rate": 0.3317307692307692,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6653474590608052,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.24013747659499768,
+          "policy_oracle_regret": 0.4356162056520388,
+          "policy_rollout_progress": 0.31756056129158317,
+          "policy_rollout_success_rate": 0.2077922077922078,
+          "restore_max_error": 2.3096799850463867e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.4073208732966413,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.5322399554734535,
+          "policy_oracle_regret": 0.6097906019459379,
+          "policy_rollout_progress": 0.7093583342242749,
+          "policy_rollout_success_rate": 0.6808510638297872,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.9295371809240544,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 1.0120462096399732,
+          "policy_oracle_regret": 1.194266588850455,
+          "policy_rollout_progress": 0.4361792250414087,
+          "policy_rollout_success_rate": 0.20202020202020202,
+          "restore_max_error": 2.384185791015625e-07
+        }
+      }
+    },
+    {
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs/near_miss_policy_bc5/seed_1/retrieval_residual_v2_rollout.json",
+      "num_groups": 575,
+      "selection_mode": "retrieval_residual",
+      "num_candidates": 16,
+      "candidate_sigma": 0.0,
+      "field_optim_steps": 0,
+      "field_optim_step_size": 0.0,
+      "field_optim_trust_radius": 0.0,
+      "field_optim_l2_penalty": 0.0,
+      "retrieval_neighbors": 1,
+      "policy_rollout_success_rate": 0.3130434782608696,
+      "policy_rollout_progress": 0.5482881942201082,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.5078381675854325,
+      "best_policy_val": {
+        "bc_loss": 0.1315989200439718,
+        "field_effect_loss": 0.010110370814800262,
+        "field_potential_loss": 0.31546536915832096,
+        "field_preference_loss": 0.4960637440284093,
+        "lattice_edges": 3833.3333333333335,
+        "progress_mae": 0.19023226698239645,
+        "rank_acc": 0.8255469501018524,
+        "rank_loss": 0.4960531195004781,
+        "regret_mae": 0.42156532737943864,
+        "success_accuracy": 0.8807473679383596,
+        "total_loss": 1.561984618504842
+      },
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.445948240330723,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 1.053844725268077,
+          "policy_oracle_regret": 1.1443010205979895,
+          "policy_rollout_progress": 0.6020584927987208,
+          "policy_rollout_success_rate": 0.18584070796460178,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.4437538628142489,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.1499858849160576,
+          "policy_oracle_regret": 1.1500098893909878,
+          "policy_rollout_progress": 0.5419346670007698,
+          "policy_rollout_success_rate": 0.2391304347826087,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6825062574720696,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.3223650339457795,
+          "policy_oracle_regret": 0.5709751525458782,
+          "policy_rollout_progress": 0.268611280869472,
+          "policy_rollout_success_rate": 0.15789473684210525,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.4619921058550611,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.33068269638864844,
+          "policy_oracle_regret": 0.3805773960308986,
+          "policy_rollout_progress": 0.8176208021672996,
+          "policy_rollout_success_rate": 0.8018018018018018,
+          "restore_max_error": 2.3795291781425476e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.6243130233916607,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.131528569283066,
+          "policy_oracle_regret": 1.291190416439549,
+          "policy_rollout_progress": 0.3994149107199449,
+          "policy_rollout_success_rate": 0.15384615384615385,
+          "restore_max_error": 1.9371509552001953e-07
+        }
+      }
+    },
+    {
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_policy_ckpt_runs/near_miss_policy_bc5/seed_2/retrieval_residual_v2_rollout.json",
+      "num_groups": 575,
+      "selection_mode": "retrieval_residual",
+      "num_candidates": 16,
+      "candidate_sigma": 0.0,
+      "field_optim_steps": 0,
+      "field_optim_step_size": 0.0,
+      "field_optim_trust_radius": 0.0,
+      "field_optim_l2_penalty": 0.0,
+      "retrieval_neighbors": 1,
+      "policy_rollout_success_rate": 0.3356521739130435,
+      "policy_rollout_progress": 0.5640824381569806,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.5376489379831955,
+      "best_policy_val": {
+        "bc_loss": 0.11367896075050037,
+        "field_effect_loss": 0.009670218582161598,
+        "field_potential_loss": 0.2641640139950646,
+        "field_preference_loss": 0.5130490180518892,
+        "lattice_edges": 3833.3333333333335,
+        "progress_mae": 0.2021729110015763,
+        "rank_acc": 0.8333857821093665,
+        "rank_loss": 0.5130119257503085,
+        "regret_mae": 0.3958987047274907,
+        "success_accuracy": 0.8680730561415354,
+        "total_loss": 1.4394984311527677
+      },
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.4202862553211162,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.9022893918057283,
+          "policy_oracle_regret": 1.0085590817034245,
+          "policy_rollout_progress": 0.6229661082228025,
+          "policy_rollout_success_rate": 0.3020833333333333,
+          "restore_max_error": 1.955777406692505e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.4408898874929156,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.0846060385287861,
+          "policy_oracle_regret": 1.1059332543683023,
+          "policy_rollout_progress": 0.5912901466110289,
+          "policy_rollout_success_rate": 0.25757575757575757,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.7155702132317755,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.29389671767130493,
+          "policy_oracle_regret": 0.5223780753815339,
+          "policy_rollout_progress": 0.32725070962268443,
+          "policy_rollout_success_rate": 0.2222222222222222,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.4149936126305325,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.38247763963028936,
+          "policy_oracle_regret": 0.43723373262598964,
+          "policy_rollout_progress": 0.7904890396512381,
+          "policy_rollout_success_rate": 0.7722772277227723,
+          "restore_max_error": 2.384185791015625e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.8354309671041038,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.1548425344957245,
+          "policy_oracle_regret": 1.2911788245042166,
+          "policy_rollout_progress": 0.42416944056749345,
+          "policy_rollout_success_rate": 0.16666666666666666,
+          "restore_max_error": 2.2351741790771484e-07
+        }
+      }
+    }
+  ]
+}