anhtld commited on
Commit
b6c6ce5
·
verified ·
1 Parent(s): 701c8ce

Auto-sync: 2026-06-28 12:50:49

Browse files
results/README.md CHANGED
@@ -38,6 +38,7 @@ Use these summaries for the current paper tables:
38
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p50_safe_types_summary.md`,
39
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p35_safe_types_margin0p20_summary.md`,
40
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_knn2_scale0p40_safe_types_margin0p20_summary.md`,
 
41
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_policy_anchor_scale0p35_safe_noexpert_summary.md`,
42
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p35_type_success010_summary.md`,
43
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p35_type_success025_summary.md`
@@ -88,7 +89,7 @@ Completed deployment-clean diagnostics:
88
 
89
  Completed residual transport jobs:
90
 
91
- Last checked: `2026-06-28 12:25 UTC`. No DoVLA jobs are currently queued.
92
 
93
  - Code supports `--retrieval-residual-scale`, defaulting to `1.0` so the
94
  completed 32.12% result is unchanged.
@@ -128,6 +129,12 @@ Last checked: `2026-06-28 12:25 UTC`. No DoVLA jobs are currently queued.
128
  - Jobs `14862857`-`14862939` completed KNN-with-abstention sweeps. The current
129
  clean best is K2 retrieval, scale `0.40`, safe residual families, margin
130
  `0.20`: 35.01%.
 
 
 
 
 
 
131
 
132
  Paper table automation:
133
 
 
38
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p50_safe_types_summary.md`,
39
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p35_safe_types_margin0p20_summary.md`,
40
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_knn2_scale0p40_safe_types_margin0p20_summary.md`,
41
+ `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_k4s040_safe_margin0p20_mean_by_type_summary.md`,
42
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_policy_anchor_scale0p35_safe_noexpert_summary.md`,
43
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p35_type_success010_summary.md`,
44
  `h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_scale0p35_type_success025_summary.md`
 
89
 
90
  Completed residual transport jobs:
91
 
92
+ Last checked: `2026-06-28 16:46 UTC`. No DoVLA jobs are currently queued.
93
 
94
  - Code supports `--retrieval-residual-scale`, defaulting to `1.0` so the
95
  completed 32.12% result is unchanged.
 
129
  - Jobs `14862857`-`14862939` completed KNN-with-abstention sweeps. The current
130
  clean best is K2 retrieval, scale `0.40`, safe residual families, margin
131
  `0.20`: 35.01%.
132
+ - Jobs `14868661`-`14868668` completed same-state no-expert lattice with a
133
+ prepended policy baseline candidate. Margin `0.00` reaches 40.70%, below the
134
+ 56.99% no-expert lattice, so policy fallback is a negative mechanism diagnostic.
135
+ - Jobs `14868693`-`14868700` completed the clean mean-by-type tangent consensus
136
+ sweep. K4, scale `0.40`, margin `0.20` reaches 34.96%, close to but below the
137
+ 35.01% K2 raw residual best.
138
 
139
  Paper table automation:
140
 
results/dovla_cil_run_report_2026-06-27.md CHANGED
@@ -36,6 +36,7 @@ The deployed h=16 behavior-cloning policy remains weak, but the interventional f
36
  | Train-state residual retrieval, policy/no-op/wrong-gripper, scale 0.35 | 33.74% | +4.00 pp | Typed tangent transport before abstention |
37
  | Train-state residual retrieval, safe residuals + advantage margin 0.20 | 34.84% | +5.10 pp | Abstention improves typed tangent transport |
38
  | K2 train-state residual retrieval, safe residuals + advantage margin 0.20 | 35.01% | +5.28 pp | Current best deployment-clean diagnostic |
 
39
  | Policy-relative residual anchor, safe residuals | 33.74% | +4.00 pp | Anchor change ties but does not improve expert-relative residuals |
40
  | Train-state residual retrieval, z-score metric | 32.23% | +2.49 pp | State normalization hurts retrieval here |
41
  | Train-state residual retrieval, repaired train-family reliability prior | 33.28-33.33% | +3.54-3.59 pp | Train success thresholds do not recover the typed safe mask |
@@ -44,6 +45,7 @@ The deployed h=16 behavior-cloning policy remains weak, but the interventional f
44
  | Residual+Gaussian hybrid, K32 sigma0.35 | 31.30% | +1.57 pp | Hybrid proposals dilute residual transport |
45
  | Residual+Gaussian hybrid, K64 sigma0.50 | 30.90% | +1.16 pp | Larger hybrid search is worse |
46
  | KNN train-state residual retrieval | 29.91% | +0.17 pp | More retrieved tangent neighborhoods dilute the signal |
 
47
 
48
  Canonical summaries:
49
 
@@ -127,6 +129,11 @@ Completed since the first report:
127
  residual transport at scale `0.35` with margin `0.20`/`0.22` reaches 34.84%.
128
  - KNN-with-abstention completed as the new clean best: K2 retrieval at scale
129
  `0.40`, margin `0.20` reaches 35.01%.
 
 
 
 
 
130
  - Z-score retrieval completed as a negative diagnostic: 32.23-32.81%, below raw
131
  residual retrieval.
132
  - Train-family reliability priors completed as neutral/negative diagnostics:
 
36
  | Train-state residual retrieval, policy/no-op/wrong-gripper, scale 0.35 | 33.74% | +4.00 pp | Typed tangent transport before abstention |
37
  | Train-state residual retrieval, safe residuals + advantage margin 0.20 | 34.84% | +5.10 pp | Abstention improves typed tangent transport |
38
  | K2 train-state residual retrieval, safe residuals + advantage margin 0.20 | 35.01% | +5.28 pp | Current best deployment-clean diagnostic |
39
+ | K4 train-state residual retrieval, mean-by-type tangent consensus | 34.96% | +5.22 pp | Near-tie clean diagnostic; does not beat K2 raw residuals |
40
  | Policy-relative residual anchor, safe residuals | 33.74% | +4.00 pp | Anchor change ties but does not improve expert-relative residuals |
41
  | Train-state residual retrieval, z-score metric | 32.23% | +2.49 pp | State normalization hurts retrieval here |
42
  | Train-state residual retrieval, repaired train-family reliability prior | 33.28-33.33% | +3.54-3.59 pp | Train success thresholds do not recover the typed safe mask |
 
45
  | Residual+Gaussian hybrid, K32 sigma0.35 | 31.30% | +1.57 pp | Hybrid proposals dilute residual transport |
46
  | Residual+Gaussian hybrid, K64 sigma0.50 | 30.90% | +1.16 pp | Larger hybrid search is worse |
47
  | KNN train-state residual retrieval | 29.91% | +0.17 pp | More retrieved tangent neighborhoods dilute the signal |
48
+ | Same-state no-expert lattice + policy baseline candidate | 40.70% | +10.96 pp | Policy fallback collapses the 56.99% no-expert lattice mechanism |
49
 
50
  Canonical summaries:
51
 
 
129
  residual transport at scale `0.35` with margin `0.20`/`0.22` reaches 34.84%.
130
  - KNN-with-abstention completed as the new clean best: K2 retrieval at scale
131
  `0.40`, margin `0.20` reaches 35.01%.
132
+ - Mean-by-type tangent consensus completed as a near-tie clean diagnostic: K4
133
+ retrieval at scale `0.40`, margin `0.20` reaches 34.96%, below K2 raw residuals.
134
+ - Same-state policy-baseline fallback completed as a negative mechanism
135
+ diagnostic: prepending the policy candidate drops no-expert lattice selection
136
+ from 56.99% to 40.70%.
137
  - Z-score retrieval completed as a negative diagnostic: 32.23-32.81%, below raw
138
  residual retrieval.
139
  - Train-family reliability priors completed as neutral/negative diagnostics: