Auto-sync: 2026-06-27 10:39:02

results/dovla_cil_run_report_2026-06-27.md

@@ -10,15 +10,27 @@ The deployed h=16 behavior-cloning policy remains weak, but the interventional f
 | h=16 policy, best-policy checkpoint | 27.01% | -2.72 pp | Lower val BC did not improve rollout |
 | Gaussian field search | 29.10% | -0.64 pp | Off-manifold candidates hurt |
 | Lattice field selection, no expert candidates | 56.99% | +27.25 pp | Conservative, reviewer-safe result |
+| Lattice field selection, near-miss only | 55.94% | +26.20 pp | Minimal local counterfactual proposal family |
+| Lattice field selection, no expert and no near-miss | 25.57% | -4.17 pp | Confirms near-miss proposals carry the gain |
 | Lattice field selection, full lattice | 69.33% | +39.59 pp | Includes expert proposal in candidate set |
+| Retrieval lattice from nearest train state | 28.93% | -0.81 pp | Same-task action-library retrieval does not transfer |
+| Retrieval lattice, no expert | 27.13% | -2.61 pp | No same-state proposals, no gain |
 
 Canonical summaries:
 
 - `results/h16_field_sweep_summary.md`
 - `results/h16_lattice_summary.md`
 - `results/h16_lattice_no_expert_summary.md`
+- `results/h16_lattice_near_miss_only_v2_summary.md`
+- `results/h16_lattice_no_near_miss_no_expert_v2_summary.md`
+- `results/h16_retrieval_lattice_summary.md`
+- `results/h16_retrieval_lattice_no_expert_summary.md`
 - `results/h16_policy_ckpt_summary.md`
 
+Non-canonical/stale summaries:
+
+- `results/h16_lattice_no_near_miss_no_expert_summary.md` was produced before fixing Slurm comma handling for multi-type exclusion. Use the `_v2` file instead.
+
 ## Jobs Submitted
 
 | Job | Purpose | Status |
@@ -43,11 +55,20 @@ The better story is:
 
 This supports a clean novelty claim: DoVLA-CIL is not just more demonstrations or a larger VLA. It is a counterfactual data engine plus a path-independent action-utility field, and the field must be deployed on intervention-lattice proposals rather than arbitrary action noise.
 
+The mechanism is now sharper: the gain is almost entirely carried by local `near_miss`
+counterfactuals. Near-miss-only selection reaches 55.94%, while removing both expert and
+near-miss candidates drops to 25.57%. Retrieval from nearest train-state lattices also stays
+near baseline (28.93% full, 27.13% no-expert), showing that the useful proposals must be
+local same-state counterfactuals rather than a generic action library.
+
 ## Reviewer-Safe Claims
 
 - Strong conservative result: no-expert lattice selection improves success from 29.74% to 56.99%.
+- Minimal-proposal result: near-miss-only selection reaches 55.94%, essentially preserving the conservative gain.
+- Mechanism ablation: removing near-miss proposals drops success to 25.57%, below baseline.
 - Strong upper result: full lattice selection improves success to 69.33%, but must be labeled as including expert proposals.
 - Negative but useful ablation: Gaussian field search fails, showing the field is not a generic black-box action optimizer off the data manifold.
+- Negative but useful ablation: nearest train-state lattice retrieval fails, showing the gain is not from a generic action library.
 - Negative checkpoint ablation: selecting checkpoint by lower BC validation loss does not improve online rollout.
 
 ## Next Best Experiments

results/h16_lattice_near_miss_only_v2_summary.json

@@ -0,0 +1,223 @@
+{
+  "run_root": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs",
+  "out_name": "lattice_near_miss_only_v2_rollout.json",
+  "num_completed": 3,
+  "baseline_h4_policy_success": 0.2967,
+  "baseline_h16_policy_success": 0.29739130434782607,
+  "mean_success": 0.5594202898550724,
+  "std_success": 0.032921207801740716,
+  "mean_progress": 0.7515200069436927,
+  "mean_oracle_success": 0.8678260869565217,
+  "mean_action_mse_to_best": 0.3473810441786781,
+  "gain_vs_h4": 0.2627202898550724,
+  "gain_vs_h16_policy": 0.26202898550724635,
+  "selected_candidate_type_counts": {
+    "lattice_near_miss": 1725
+  },
+  "rows": [
+    {
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs/seed_0/lattice_near_miss_only_v2_rollout.json",
+      "selection_mode": "lattice",
+      "num_candidates": 16,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.5217391304347826,
+      "policy_rollout_progress": 0.7402807480075242,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.33686841549711183,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.28381826125232973,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 0.46978291792353405,
+          "policy_oracle_regret": 0.4830270939573799,
+          "policy_rollout_progress": 0.8529790917929915,
+          "policy_rollout_success_rate": 0.5670103092783505,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.29220151433792824,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 0.6119791001905329,
+          "policy_oracle_regret": 0.6254019676170384,
+          "policy_rollout_progress": 0.7854622084683237,
+          "policy_rollout_success_rate": 0.5144230769230769,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.5234726118044807,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.20568561372930197,
+          "policy_oracle_regret": 0.5280835723012399,
+          "policy_rollout_progress": 0.23136528778129353,
+          "policy_rollout_success_rate": 0.1038961038961039,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.29275905471443736,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.17897391620468586,
+          "policy_oracle_regret": 0.21774373758346477,
+          "policy_rollout_progress": 0.8904324348302598,
+          "policy_rollout_success_rate": 0.8723404255319149,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.37943747813103135,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 0.5016614573471474,
+          "policy_oracle_regret": 0.547894100799705,
+          "policy_rollout_progress": 0.7881873010685949,
+          "policy_rollout_success_rate": 0.48484848484848486,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs/seed_1/lattice_near_miss_only_v2_rollout.json",
+      "selection_mode": "lattice",
+      "num_candidates": 16,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.5826086956521739,
+      "policy_rollout_progress": 0.7557949085549815,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.338796136389365,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.2927715086885027,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 0.3441992188449455,
+          "policy_oracle_regret": 0.3671226171793136,
+          "policy_rollout_progress": 0.8900700404053241,
+          "policy_rollout_success_rate": 0.6637168141592921,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.3072718674033556,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 0.48501841450595984,
+          "policy_oracle_regret": 0.4925553194733094,
+          "policy_rollout_progress": 0.8276301623807977,
+          "policy_rollout_success_rate": 0.6032608695652174,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.4901314919912501,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.23106697291184805,
+          "policy_oracle_regret": 0.516092440003137,
+          "policy_rollout_progress": 0.2737633452726234,
+          "policy_rollout_success_rate": 0.15789473684210525,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.2843499049829604,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.2667342086096068,
+          "policy_oracle_regret": 0.36706711418993837,
+          "policy_rollout_progress": 0.8311310840082599,
+          "policy_rollout_success_rate": 0.8018018018018018,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.39971144500464856,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 0.4863413238263392,
+          "policy_oracle_regret": 0.549977686378982,
+          "policy_rollout_progress": 0.7544905869187889,
+          "policy_rollout_success_rate": 0.5274725274725275,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs/seed_2/lattice_near_miss_only_v2_rollout.json",
+      "selection_mode": "lattice",
+      "num_candidates": 16,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.5739130434782609,
+      "policy_rollout_progress": 0.7584843642685725,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.36647858064955746,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.3152333604769713,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 0.33117461173484725,
+          "policy_oracle_regret": 0.37181698189427453,
+          "policy_rollout_progress": 0.8922190756226579,
+          "policy_rollout_success_rate": 0.65625,
+          "restore_max_error": 3.5762786865234375e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.32152757409847144,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 0.46628229395307674,
+          "policy_oracle_regret": 0.47343207752764827,
+          "policy_rollout_progress": 0.8408821377940853,
+          "policy_rollout_success_rate": 0.6262626262626263,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.5553114101911585,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.25806876574125553,
+          "policy_oracle_regret": 0.5503819031847847,
+          "policy_rollout_progress": 0.31111136451363564,
+          "policy_rollout_success_rate": 0.18888888888888888,
+          "restore_max_error": 4.0978193283081055e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.28991338288374613,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.3124116746210816,
+          "policy_oracle_regret": 0.3663207845817698,
+          "policy_rollout_progress": 0.8316990173984282,
+          "policy_rollout_success_rate": 0.801980198019802,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 0.41712270008607044,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 0.5690274155802197,
+          "policy_oracle_regret": 0.6078866597678926,
+          "policy_rollout_progress": 0.799768792755074,
+          "policy_rollout_success_rate": 0.5,
+          "restore_max_error": 4.76837158203125e-07
+        }
+      }
+    }
+  ]
+}

results/h16_lattice_near_miss_only_v2_summary.md

@@ -0,0 +1,20 @@
+# h=16 Lattice-Selected Rollout
+
+Run root: `/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs`
+Completed seeds: 3
+Baseline h=4 policy success: 29.67%
+Baseline h=16 policy success: 29.74%
+
+Mean success: 55.94% +/- 3.29%
+Gain vs h=16 policy: +26.20%
+Mean oracle success: 86.78%
+Mean progress: 75.15%
+
+| seed | success | progress | oracle | candidates | action MSE |
+|---:|---:|---:|---:|---:|---:|
+| 0 | 52.17% | 74.03% | 85.74% | 16 | 0.337 |
+| 1 | 58.26% | 75.58% | 86.96% | 16 | 0.339 |
+| 2 | 57.39% | 75.85% | 87.65% | 16 | 0.366 |
+
+Selected candidate types:
+- lattice_near_miss: 1725

results/h16_lattice_no_near_miss_no_expert_v2_summary.json

@@ -0,0 +1,226 @@
+{
+  "run_root": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs",
+  "out_name": "lattice_no_near_miss_no_expert_v2_rollout.json",
+  "num_completed": 3,
+  "baseline_h4_policy_success": 0.2967,
+  "baseline_h16_policy_success": 0.29739130434782607,
+  "mean_success": 0.25565217391304346,
+  "std_success": 0.028629700231572734,
+  "mean_progress": 0.507481863628803,
+  "mean_oracle_success": 0.8678260869565217,
+  "mean_action_mse_to_best": 0.7650326310357322,
+  "gain_vs_h4": -0.04104782608695656,
+  "gain_vs_h16_policy": -0.04173913043478261,
+  "selected_candidate_type_counts": {
+    "lattice_no_op": 515,
+    "lattice_random_negative": 314,
+    "lattice_wrong_direction": 620,
+    "lattice_wrong_gripper": 276
+  },
+  "rows": [
+    {
+      "seed": 0,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs/seed_0/lattice_no_near_miss_no_expert_v2_rollout.json",
+      "selection_mode": "lattice",
+      "num_candidates": 16,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.2382608695652174,
+      "policy_rollout_progress": 0.5087076342559379,
+      "oracle_success_rate": 0.8573913043478261,
+      "action_mse_to_best": 0.7311759229781835,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.4629675186786455,
+          "expert_success_rate": 0.8865979381443299,
+          "num_groups": 97,
+          "oracle_success_rate": 0.9278350515463918,
+          "policy_expert_regret": 1.0916482767800695,
+          "policy_oracle_regret": 1.1049537313045914,
+          "policy_rollout_progress": 0.5403308049612439,
+          "policy_rollout_success_rate": 0.25773195876288657,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.7160787453266004,
+          "expert_success_rate": 0.9375,
+          "num_groups": 208,
+          "oracle_success_rate": 0.9471153846153846,
+          "policy_expert_regret": 1.170773381868807,
+          "policy_oracle_regret": 1.1774399559228466,
+          "policy_rollout_progress": 0.5266934509317462,
+          "policy_rollout_success_rate": 0.22115384615384615,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.6624402255787478,
+          "expert_success_rate": 0.19480519480519481,
+          "num_groups": 77,
+          "oracle_success_rate": 0.36363636363636365,
+          "policy_expert_regret": 0.27485672640916586,
+          "policy_oracle_regret": 0.45024229619990697,
+          "policy_rollout_progress": 0.28314845375232883,
+          "policy_rollout_success_rate": 0.12987012987012986,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.574349947471885,
+          "expert_success_rate": 0.8617021276595744,
+          "num_groups": 94,
+          "oracle_success_rate": 0.9893617021276596,
+          "policy_expert_regret": 0.6354690522272536,
+          "policy_oracle_regret": 0.7731607213933417,
+          "policy_rollout_progress": 0.6310945977555945,
+          "policy_rollout_success_rate": 0.5957446808510638,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 1.2280518680886188,
+          "expert_success_rate": 0.6767676767676768,
+          "num_groups": 99,
+          "oracle_success_rate": 0.8585858585858586,
+          "policy_expert_regret": 1.1099242578552226,
+          "policy_oracle_regret": 1.3217504209641255,
+          "policy_rollout_progress": 0.4991640474158104,
+          "policy_rollout_success_rate": 0.0,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "seed": 1,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs/seed_1/lattice_no_near_miss_no_expert_v2_rollout.json",
+      "selection_mode": "lattice",
+      "num_candidates": 16,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.24,
+      "policy_rollout_progress": 0.4924592710918059,
+      "oracle_success_rate": 0.8695652173913043,
+      "action_mse_to_best": 0.7720995902658805,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.5445958682525475,
+          "expert_success_rate": 0.8584070796460177,
+          "num_groups": 113,
+          "oracle_success_rate": 0.9380530973451328,
+          "policy_expert_regret": 1.1100166487482797,
+          "policy_oracle_regret": 1.1915242880319072,
+          "policy_rollout_progress": 0.5372224030768977,
+          "policy_rollout_success_rate": 0.20353982300884957,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.848305003616311,
+          "expert_success_rate": 0.9402173913043478,
+          "num_groups": 184,
+          "oracle_success_rate": 0.9456521739130435,
+          "policy_expert_regret": 1.1529376383000256,
+          "policy_oracle_regret": 1.1529616427749558,
+          "policy_rollout_progress": 0.53135427386176,
+          "policy_rollout_success_rate": 0.2391304347826087,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.626066874884265,
+          "expert_success_rate": 0.25,
+          "num_groups": 76,
+          "oracle_success_rate": 0.40789473684210525,
+          "policy_expert_regret": 0.32680029648748277,
+          "policy_oracle_regret": 0.5785002729144731,
+          "policy_rollout_progress": 0.26434124054044095,
+          "policy_rollout_success_rate": 0.10526315789473684,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.53622070122678,
+          "expert_success_rate": 0.8198198198198198,
+          "num_groups": 111,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.7703769668802485,
+          "policy_oracle_regret": 0.8709201557679219,
+          "policy_rollout_progress": 0.5885393036915375,
+          "policy_rollout_success_rate": 0.5405405405405406,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 1.3102003329402798,
+          "expert_success_rate": 0.6923076923076923,
+          "num_groups": 91,
+          "oracle_success_rate": 0.8571428571428571,
+          "policy_expert_regret": 1.169783660671213,
+          "policy_oracle_regret": 1.3548147910898858,
+          "policy_rollout_progress": 0.4315490763593506,
+          "policy_rollout_success_rate": 0.03296703296703297,
+          "restore_max_error": 3.948807716369629e-07
+        }
+      }
+    },
+    {
+      "seed": 2,
+      "path": "/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs/seed_2/lattice_no_near_miss_no_expert_v2_rollout.json",
+      "selection_mode": "lattice",
+      "num_candidates": 16,
+      "num_groups": 575,
+      "policy_rollout_success_rate": 0.288695652173913,
+      "policy_rollout_progress": 0.5212786855386651,
+      "oracle_success_rate": 0.8765217391304347,
+      "action_mse_to_best": 0.7918223798631326,
+      "per_task": {
+        "LiftPegUpright-v1": {
+          "action_mse_to_best": 0.6275289737386629,
+          "expert_success_rate": 0.8229166666666666,
+          "num_groups": 96,
+          "oracle_success_rate": 0.9270833333333334,
+          "policy_expert_regret": 1.0027583745929103,
+          "policy_oracle_regret": 1.1099423215103645,
+          "policy_rollout_progress": 0.5574810882098973,
+          "policy_rollout_success_rate": 0.23958333333333334,
+          "restore_max_error": 3.5762786865234375e-07
+        },
+        "PickCube-v1": {
+          "action_mse_to_best": 0.7933213269770748,
+          "expert_success_rate": 0.9444444444444444,
+          "num_groups": 198,
+          "oracle_success_rate": 0.9595959595959596,
+          "policy_expert_regret": 1.0354422351364234,
+          "policy_oracle_regret": 1.0424736577340148,
+          "policy_rollout_progress": 0.5698203555675168,
+          "policy_rollout_success_rate": 0.3282828282828283,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "PullCube-v1": {
+          "action_mse_to_best": 0.7853237770290838,
+          "expert_success_rate": 0.24444444444444444,
+          "num_groups": 90,
+          "oracle_success_rate": 0.4666666666666667,
+          "policy_expert_regret": 0.32428360626929337,
+          "policy_oracle_regret": 0.5771468290438254,
+          "policy_rollout_progress": 0.30629591482381024,
+          "policy_rollout_success_rate": 0.16666666666666666,
+          "restore_max_error": 4.0978193283081055e-07
+        },
+        "PushCube-v1": {
+          "action_mse_to_best": 0.4587447434885077,
+          "expert_success_rate": 0.8514851485148515,
+          "num_groups": 101,
+          "oracle_success_rate": 1.0,
+          "policy_expert_regret": 0.6443573544816216,
+          "policy_oracle_regret": 0.7373619361384081,
+          "policy_rollout_progress": 0.6487766777229781,
+          "policy_rollout_success_rate": 0.6138613861386139,
+          "restore_max_error": 4.76837158203125e-07
+        },
+        "StackCube-v1": {
+          "action_mse_to_best": 1.3440567241774666,
+          "expert_success_rate": 0.7666666666666667,
+          "num_groups": 90,
+          "oracle_success_rate": 0.9111111111111111,
+          "policy_expert_regret": 1.2523939018448194,
+          "policy_oracle_regret": 1.423131014737818,
+          "policy_rollout_progress": 0.4477728058894475,
+          "policy_rollout_success_rate": 0.011111111111111112,
+          "restore_max_error": 4.76837158203125e-07
+        }
+      }
+    }
+  ]
+}

results/h16_lattice_no_near_miss_no_expert_v2_summary.md

@@ -0,0 +1,23 @@
+# h=16 Lattice-Selected Rollout
+
+Run root: `/scratch/knguy52/dovla/experiments/dovla_h16_rollout_runs`
+Completed seeds: 3
+Baseline h=4 policy success: 29.67%
+Baseline h=16 policy success: 29.74%
+
+Mean success: 25.57% +/- 2.86%
+Gain vs h=16 policy: -4.17%
+Mean oracle success: 86.78%
+Mean progress: 50.75%
+
+| seed | success | progress | oracle | candidates | action MSE |
+|---:|---:|---:|---:|---:|---:|
+| 0 | 23.83% | 50.87% | 85.74% | 16 | 0.731 |
+| 1 | 24.00% | 49.25% | 86.96% | 16 | 0.772 |
+| 2 | 28.87% | 52.13% | 87.65% | 16 | 0.792 |
+
+Selected candidate types:
+- lattice_no_op: 515
+- lattice_random_negative: 314
+- lattice_wrong_direction: 620
+- lattice_wrong_gripper: 276

results/paper_core_results.md

@@ -0,0 +1,33 @@
+# Paper Core Results
+
+All rows use 3 seeds and 575 validation groups per seed unless noted. The direct policy
+baseline is the h=16 rank-checkpoint online rollout (`29.74%`).
+
+| Method | Uses same-state proposals | Uses expert proposal | Success | Gain vs policy | Interpretation |
+|---|---:|---:|---:|---:|---|
+| Direct h=16 policy | No | No | 29.74% | -- | BC policy cannot exploit high oracle ceiling |
+| Best-policy checkpoint | No | No | 27.01% | -2.72 pp | Lower validation BC is not enough |
+| Gaussian field search | No | No | 29.10% | -0.64 pp | Field does not optimize off-manifold noise |
+| Retrieval lattice | No | Yes | 28.93% | -0.81 pp | Nearest train-state action library does not transfer |
+| Retrieval lattice, no expert | No | No | 27.13% | -2.61 pp | Conservative retrieval also fails |
+| Lattice, no expert/no near-miss | Yes | No | 25.57% | -4.17 pp | Non-local negatives do not help |
+| Lattice, near-miss only | Yes | No | 55.94% | +26.20 pp | Local counterfactual proposals carry the gain |
+| Lattice, no expert | Yes | No | 56.99% | +27.25 pp | Reviewer-safe main result |
+| Lattice, full | Yes | Yes | 69.33% | +39.59 pp | Upper deployment result with expert proposal |
+| Oracle ceiling | Yes | Yes | 86.78% | +57.04 pp | Remaining headroom |
+
+Suggested main-table rows:
+
+1. Direct h=16 policy
+2. Gaussian field search
+3. Retrieval lattice, no expert
+4. Lattice, near-miss only
+5. Lattice, no expert
+6. Lattice, full
+7. Oracle ceiling
+
+Suggested claim:
+
+> DoVLA-CIL is not a better behavior-cloning policy; it is a local counterfactual action
+> selection rule. The learned field only improves rollout when queried on same-state
+> intervention proposals, and the effect is isolated to near-miss counterfactuals.