anhtld commited on
Commit
be9dc11
·
verified ·
1 Parent(s): 6a54bab

Auto-sync: 2026-06-30 03:55:26 (part 2)

Browse files
results/paper_core_results.md CHANGED
@@ -5,10 +5,10 @@ baseline is the h=16 rank-checkpoint online rollout (`29.74%`).
5
 
6
  For paired seed deltas, per-task gaps, and selection histograms, regenerate and
7
  read `paper_analysis.md` with `python3 scripts/build_paper_analysis.py`. Current
8
- paired analysis: best clean K4 masked composed type-consensus transport with
9
- an exact near-miss/no-op compatibility mask is `+5.86 pp` over canonical h=16, same-state no-expert
10
  lattice is `+27.25 pp`, and the remaining clean-to-same-state proposal gap is
11
- `+21.39 pp`.
12
 
13
  | Method | Uses same-state proposals | Uses expert proposal | Success | Gain vs policy | Interpretation |
14
  |---|---:|---:|---:|---:|---|
@@ -56,14 +56,15 @@ lattice is `+27.25 pp`, and the remaining clean-to-same-state proposal gap is
56
  | K4 composed type-consensus residual retrieval, masked | No | No | 35.30% | +5.57 pp | Clean anti-goal composite masking removes the confound; tangent composition alone near-ties but does not beat mean-by-type scale-grid transport |
57
  | K4 composed type-consensus residual retrieval, masked + no-op prior 0.03 | No | No | 35.54% | +5.80 pp | Previous clean best; masked local tangent composition adds one success over the scale-grid no-op row while staying sparse |
58
  | K4 composed type-consensus residual retrieval, exact compatibility mask without typed prior | No | No | 35.48% | +5.74 pp | Isolates the algebraic compatibility mask: it improves pure masked composition but does not fully replace the sparse typed no-op prior |
59
- | K4 composed type-consensus residual retrieval, no-op prior + exact near-miss/no-op incompatibility mask | No | No | 35.59% | +5.86 pp | Current best clean diagnostic; dropping only the weak `near_miss+no_op` composite preserves useful singleton/composite tangents and lowers action MSE |
60
  | K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/near-miss singleton priors | No | No | 35.59% | +5.86 pp | Exact singleton near-miss priors 0.01/0.02 tie the top row and raise progress slightly; transferred near-miss tangents are high-precision but too sparse to close the proposal gap |
61
- | K4 compatible residual retrieval, unique candidate-oracle prefix K=8 | No | No | pending | pending | Diagnostic-only measured oracle over generated clean candidate prefix; first non-unique run was archived because duplicate policy branches inflated headroom |
62
- | K4 compatible residual retrieval, margin 0.10 | No | No | 34.67% | +4.93 pp | Naively lowering abstention selects too many bad nonzero tangents and falls below the 35.59% top row |
 
63
  | K4 composed type-consensus residual retrieval, exact compatibility mask + source-score prior | No | No | 35.48% | +5.74 pp | Measured train-source reward confidence does not replace the sparse typed prior on the compatible chart; useful negative calibration |
64
  | K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/source-score priors | No | No | 35.54% | +5.80 pp | Adding measured train-source reward confidence to the current typed-prior chart ties near-best but does not beat exact compatibility masking alone |
65
  | K4 composed type-consensus residual retrieval, masked + component-wise no-op prior | No | No | 35.36% | +5.62 pp | Propagating the no-op prior into every composite over-selects composed tangents and drops below the exact-prior row; useful negative calibration |
66
- | K4 composed type-consensus residual retrieval, masked + composite trust penalty | No | No | 35.48-35.54% | +5.74-5.80 pp | Composite-only L2 lowers action MSE but does not improve success; combined with the exact compatibility mask it reaches 35.54% with the lowest MSE, one seed below the 35.59% top row |
67
  | K4 repair-tangent residual transport | No | No | 34.14-34.43% | +4.41-4.70 pp | Reversing residuals into failure-to-expert repair tangents is a clean negative diagnostic; the current gain is not recovered by transporting near-miss-to-expert vectors |
68
  | K4 mean-by-type residual retrieval + source-score prior 0.025 | No | No | 35.19% | +5.45 pp | A stronger reward-score prior drops below the plateau |
69
  | K4 mean-by-type residual retrieval, no-op-only residuals | No | No | 35.19% | +5.45 pp | Removing wrong-gripper residuals loses one success versus the fixed-scale safe-family plateau; the core gain is sparse no-op/tangent repair, with wrong-gripper acting only as a marginal helper |
@@ -112,9 +113,10 @@ Suggested main-table rows:
112
  14. K4 mean-by-type residual retrieval + scale-grid no-op prior 0.03
113
  15. K4 masked composed type-consensus residual retrieval + no-op prior 0.03
114
  16. K4 masked composed type-consensus residual retrieval + exact near-miss/no-op compatibility mask, with and without typed no-op prior
115
- 17. K4 masked composed type-consensus residual retrieval + source-score prior on the exact compatibility chart
116
- 18. K4 masked composed type-consensus residual retrieval + component-wise no-op prior diagnostic
117
- 19. K4 masked composed type-consensus residual retrieval + composite trust penalty diagnostic
 
118
  20. K4 mean-by-type residual retrieval + upper/wide tangent-length diagnostics
119
  21. K4 mean-by-type residual retrieval + minimum-energy action penalty diagnostics
120
  22. K4 mean-by-type residual retrieval + source-progress/source-score/source-advantage prior diagnostics
@@ -142,10 +144,12 @@ Suggested claim:
142
  > DoVLA-CIL is not a better behavior-cloning policy; it is a local counterfactual action
143
  > selection rule. Deployment-clean K4 consensus residual transport with advantage
144
  > abstention, a small typed no-op prior, field-gated tangent length calibration,
145
- > and masked local tangent composition gives the strongest clean gain so far;
146
- > the top row improves by removing only the semantically incompatible
147
- > `near_miss+no_op` composite while preserving the useful singleton and
148
- > near-miss/wrong-gripper tangents. A train-source score prior on the same
 
 
149
  > compatible chart drops to 35.48%, so the result is not explained by generic
150
  > train-confidence weighting.
151
  > extending the scale grid upward and adding minimum-energy action regularization
 
5
 
6
  For paired seed deltas, per-task gaps, and selection histograms, regenerate and
7
  read `paper_analysis.md` with `python3 scripts/build_paper_analysis.py`. Current
8
+ paired analysis: best clean K4 compatible tangent transport with a trace-motivated
9
+ near-miss challenger gate is `+6.26 pp` over canonical h=16, same-state no-expert
10
  lattice is `+27.25 pp`, and the remaining clean-to-same-state proposal gap is
11
+ `+20.99 pp`.
12
 
13
  | Method | Uses same-state proposals | Uses expert proposal | Success | Gain vs policy | Interpretation |
14
  |---|---:|---:|---:|---:|---|
 
56
  | K4 composed type-consensus residual retrieval, masked | No | No | 35.30% | +5.57 pp | Clean anti-goal composite masking removes the confound; tangent composition alone near-ties but does not beat mean-by-type scale-grid transport |
57
  | K4 composed type-consensus residual retrieval, masked + no-op prior 0.03 | No | No | 35.54% | +5.80 pp | Previous clean best; masked local tangent composition adds one success over the scale-grid no-op row while staying sparse |
58
  | K4 composed type-consensus residual retrieval, exact compatibility mask without typed prior | No | No | 35.48% | +5.74 pp | Isolates the algebraic compatibility mask: it improves pure masked composition but does not fully replace the sparse typed no-op prior |
59
+ | K4 composed type-consensus residual retrieval, no-op prior + exact near-miss/no-op incompatibility mask | No | No | 35.59% | +5.86 pp | Previous clean best; dropping only the weak `near_miss+no_op` composite preserves useful singleton/composite tangents and lowers action MSE |
60
  | K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/near-miss singleton priors | No | No | 35.59% | +5.86 pp | Exact singleton near-miss priors 0.01/0.02 tie the top row and raise progress slightly; transferred near-miss tangents are high-precision but too sparse to close the proposal gap |
61
+ | K4 compatible residual retrieval, unique candidate-oracle prefix K=8 | No | No | 43.07% diagnostic | +13.33 pp diagnostic | Diagnostic-only measured oracle over generated clean candidate prefix; unique-action trace shows real selector headroom, not duplicate-action inflation |
62
+ | K4 compatible residual retrieval + near-miss challenger gate 0.02 | No | No | 36.00% | +6.26 pp | Current best clean deployment row; a two-stage selector keeps the robust compatible-chart anchor and only lets singleton near-miss tangents override when the field margin is positive |
63
+ | K4 compatible residual retrieval, margin 0.10 | No | No | 34.67% | +4.93 pp | Naively lowering abstention selects too many bad nonzero tangents and falls below the 36.00% top row |
64
  | K4 composed type-consensus residual retrieval, exact compatibility mask + source-score prior | No | No | 35.48% | +5.74 pp | Measured train-source reward confidence does not replace the sparse typed prior on the compatible chart; useful negative calibration |
65
  | K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/source-score priors | No | No | 35.54% | +5.80 pp | Adding measured train-source reward confidence to the current typed-prior chart ties near-best but does not beat exact compatibility masking alone |
66
  | K4 composed type-consensus residual retrieval, masked + component-wise no-op prior | No | No | 35.36% | +5.62 pp | Propagating the no-op prior into every composite over-selects composed tangents and drops below the exact-prior row; useful negative calibration |
67
+ | K4 composed type-consensus residual retrieval, masked + composite trust penalty | No | No | 35.48-35.54% | +5.74-5.80 pp | Composite-only L2 lowers action MSE but does not improve success; combined with the exact compatibility mask it reaches 35.54% with the lowest MSE, one seed below the previous 35.59% compatibility row |
68
  | K4 repair-tangent residual transport | No | No | 34.14-34.43% | +4.41-4.70 pp | Reversing residuals into failure-to-expert repair tangents is a clean negative diagnostic; the current gain is not recovered by transporting near-miss-to-expert vectors |
69
  | K4 mean-by-type residual retrieval + source-score prior 0.025 | No | No | 35.19% | +5.45 pp | A stronger reward-score prior drops below the plateau |
70
  | K4 mean-by-type residual retrieval, no-op-only residuals | No | No | 35.19% | +5.45 pp | Removing wrong-gripper residuals loses one success versus the fixed-scale safe-family plateau; the core gain is sparse no-op/tangent repair, with wrong-gripper acting only as a marginal helper |
 
113
  14. K4 mean-by-type residual retrieval + scale-grid no-op prior 0.03
114
  15. K4 masked composed type-consensus residual retrieval + no-op prior 0.03
115
  16. K4 masked composed type-consensus residual retrieval + exact near-miss/no-op compatibility mask, with and without typed no-op prior
116
+ 17. K4 compatible tangent near-miss challenger gate 0.02
117
+ 18. K4 masked composed type-consensus residual retrieval + source-score prior on the exact compatibility chart
118
+ 19. K4 masked composed type-consensus residual retrieval + component-wise no-op prior diagnostic
119
+ 20. K4 masked composed type-consensus residual retrieval + composite trust penalty diagnostic
120
  20. K4 mean-by-type residual retrieval + upper/wide tangent-length diagnostics
121
  21. K4 mean-by-type residual retrieval + minimum-energy action penalty diagnostics
122
  22. K4 mean-by-type residual retrieval + source-progress/source-score/source-advantage prior diagnostics
 
144
  > DoVLA-CIL is not a better behavior-cloning policy; it is a local counterfactual action
145
  > selection rule. Deployment-clean K4 consensus residual transport with advantage
146
  > abstention, a small typed no-op prior, field-gated tangent length calibration,
147
+ > masked local tangent composition, and a trace-motivated near-miss challenger
148
+ > gate give the strongest clean gain so far. The previous top row improves by
149
+ > removing only the semantically incompatible `near_miss+no_op` composite while
150
+ > preserving useful singleton and near-miss/wrong-gripper tangents; the current
151
+ > clean top row adds a second-stage selector that lets singleton near-miss
152
+ > tangents override the robust anchor only under a positive field margin. A train-source score prior on the same
153
  > compatible chart drops to 35.48%, so the result is not explained by generic
154
  > train-confidence weighting.
155
  > extending the scale grid upward and adding minimum-energy action regularization
results/paper_story_memo.md CHANGED
@@ -16,7 +16,7 @@ when queried on proposal geometry that matches those local counterfactuals.
16
  | Same-state local counterfactual proposals are the mechanism | near-miss-only lattice is 55.94%; removing expert+near_miss drops to 25.57% | Strongly supported |
17
  | Conservative same-state result is large | no-expert lattice is 56.99% vs 29.74% policy | Main result |
18
  | Full lattice gives upper result | full lattice is 69.33%, oracle is 86.78% | Strong but label expert proposal clearly |
19
- | Deployment-clean proposal is currently a bottleneck | best clean K4 masked composed type-consensus transport with an exact near-miss/no-op compatibility mask is 35.59%, far below 56.99% | Supported |
20
  | Gradient-based field optimization does not solve the clean proposal gap | `field_optim` best observed result is 25.39% | Negative diagnostic |
21
  | A broader non-expert proposal target does not reduce the proposal gap | direct broad non-expert policy is 27.88%; with field scoring it is 26.49% | Negative diagnostic |
22
  | Counterfactual residuals transfer better than absolute retrieved actions | nearest residual retrieval is 32.12% vs absolute retrieval 28.93%; KNN4 residual drops to 29.91% | Supported as a clean bridge |
@@ -29,10 +29,11 @@ when queried on proposal geometry that matches those local counterfactuals.
29
  | Clean residual transport behaves like sparse intervention | the best clean row abstains to zero-residual policy on 93.5% of states, while selected nonzero residuals succeed at 50.9% vs 34.5% for abstention | Stronger clean-mechanism framing |
30
  | Tangent consensus is close but needs sparse typing | K4 mean-by-type residual consensus reaches 34.96%; a small no-op residual prior plateau at 0.025-0.035 raises fixed-scale transport to 35.25% | Fixed-scale clean diagnostic |
31
  | Field-gated tangent length calibration improves the clean bridge | K4 mean-by-type scale grid 0.35/0.40/0.45 with no-op bonus 0.03 reaches 35.42%; the source-score version reaches 35.30%. Upper 0.40/0.45/0.50 nearly ties at 35.36%, while wide 0.35/0.45/0.55 drops to 35.13% | Previous clean best; local scale calibration, not a larger-step effect |
32
- | Masked tangent composition gives a small clean lift | K4 composed type-consensus transport is 35.30% after anti-goal composite masking; exact `near_miss+no_op` masking alone reaches 35.48%; typed no-op prior reaches 35.54%; combining the exact compatibility mask with the typed prior reaches 35.59% | Current best clean result; controlled compatible tangent-chart composition, not unmasked proposal accumulation |
33
- | Train-source score priors do not explain the compatible tangent-chart gain | replacing the typed no-op prior with a measured train-source reward-score prior on the exact compatibility chart reaches 35.48%; adding it to the typed no-op row reaches 35.54%, still below the 35.59% top row | Negative diagnostic: compatibility masking and sparse typed abstention matter more than continuous train-source confidence on this chart |
34
- | Singleton near-miss priors do not open a new clean route | adding a small exact `residual_near_miss` singleton prior 0.01 or 0.02 on the compatible chart ties the 35.59% top row and slightly raises progress to 57.10% | Tie diagnostic: revived near-miss tangents are high-precision but too sparse to close the clean-to-same-state proposal gap |
35
- | Candidate-prefix oracle needs unique-action hygiene | the first K=8 prefix diagnostic exposed apparent headroom, but raw rows showed duplicate zero-residual/policy branches across scale slots; summaries were archived as `_nonunique` and a unique-action rerun is pending | In progress: use the deduplicated diagnostic before making ranking/abstention claims |
 
36
  | Component-wise composite priors do not add the gain | propagating the no-op bonus into composite types reaches 35.36%, below the exact-prior masked composition row at 35.54% and the exact compatibility row at 35.59% | Negative/near-tie diagnostic: sparse exact priors are better than broadly rewarding every no-op-containing composite |
37
  | Composite trust-radius penalty explains but does not improve the success top line | composite-only L2 penalty 0.02 ties 35.54% while lowering action MSE from 0.4106 to 0.4079; with the exact compatibility mask it lowers MSE further to 0.4048 but drops back to 35.54% | Tie/negative diagnostic: composed tangents need a local trust radius, but compatibility masking gives the success optimum |
38
  | Minimum-energy residual regularization does not add the gain | action L2 penalty 0.05 ties the previous 35.42% scale-grid row, while 0.10/0.20 reach 35.36% | Negative/tie diagnostic: the clean bridge is not explained by shortest-action bias |
@@ -46,7 +47,7 @@ when queried on proposal geometry that matches those local counterfactuals.
46
  | Typed no-op residual prior improves the clean bridge | CPU smoke `14883591` passed; bonuses 0.025/0.03/0.035 tie at 35.25%, while 0.01/0.02/0.05/0.08 are slightly lower | Fixed-scale clean diagnostic |
47
  | Wrong-gripper typed prior does not add a new clean bridge | wrong-gripper-only reaches 35.19%; no-op+wrong-gripper 0.02 ties 35.25%; no-op+wrong-gripper 0.04 drops to 35.13% | Negative/tie diagnostic |
48
  | No-op-only residuals nearly preserve the fixed-scale clean bridge | excluding wrong-gripper residuals gives 35.19% with either no-op bonus 0.03 or source-score bonus 0.02, one success below the 35.25% fixed-scale safe-family plateau | Mechanism sharpened: wrong-gripper is marginal, not core |
49
- | The proposal gap is now quantified | `paper_analysis.md` reports best clean +5.86 pp over canonical h16, same-state no-expert +27.25 pp, leaving a +21.39 pp clean-to-same-state gap | Core paper tension |
50
  | Policy fallback is not the same-state mechanism | adding a policy baseline candidate to the no-expert same-state lattice drops 56.99% to 40.70% even with margin 0.00 | Negative diagnostic |
51
  | Z-score retrieval metric does not help | z-score rows reach 32.23-32.81%, below raw retrieval | Negative diagnostic |
52
  | Task-relative actor-pose retrieval metric does not improve tangent transfer | K2 task-relative residual retrieval reaches 34.26% vs raw K2 35.01%; K4 task-relative mean-by-type + no-op reaches 34.43% vs raw K4 35.25% | Negative diagnostic |
@@ -81,8 +82,9 @@ clean proposal result, the intended main rows are:
81
  15. K4 mean-by-type tangent consensus + typed no-op prior 0.025-0.035: 35.25%
82
  16. K4 mean-by-type tangent consensus + scale-grid typed no-op prior: 35.42%
83
  17. K4 masked composed type-consensus tangent transport: 35.30%; with exact near-miss/no-op compatibility mask only: 35.48%; with typed no-op prior: 35.54%; with exact compatibility mask + typed no-op prior: 35.59%; adding singleton near-miss prior 0.01/0.02 ties 35.59%; with source-score prior on that exact chart: 35.48%; with typed no-op + source-score: 35.54%
84
- 18. K4 compatible tangent unique-action candidate-oracle prefix K=8 diagnostic: pending rerun after archiving the non-unique oracle summary
85
- 19. K4 compatible tangent abstention margin 0.10 diagnostic: 34.67%; naive lower abstention hurts despite apparent non-unique oracle headroom
 
86
  20. K4 masked composed type-consensus tangent transport + component-wise no-op prior: 35.36%
87
  21. K4 masked composed type-consensus tangent transport + composite L2 trust penalty: 35.54% at 0.02; 35.48% at 0.05; 35.54% with exact compatibility mask + L2 0.02
88
  22. K4 mean-by-type tangent consensus + upper/wide scale diagnostics: 35.36% for 0.40/0.45/0.50; 35.13% for 0.35/0.45/0.55
@@ -139,8 +141,8 @@ test-time search. The cleaner novelty is:
139
 
140
  ## Job Status
141
 
142
- Last checked: `2026-06-30 06:34 UTC`. The K4 masked composed type-consensus
143
- sweep completed and produced a new clean best, 35.59%, when the exact
144
  `residual_near_miss+residual_no_op` composite is additionally masked. Pure
145
  masked composition reached 35.30%, exact compatibility masking without typed
146
  or source-score priors reached 35.48%, masked composition with the exact typed
@@ -152,21 +154,23 @@ no-op prior row. Composite-only L2 trust penalties reduce action MSE but do not
152
  improve success: standalone L2 0.02 ties 35.54%, L2 0.05 reaches 35.48%, and
153
  exact compatibility + L2 0.02 reaches 35.54% with the lowest MSE.
154
  On the same exact compatibility chart, a train-measured source-score prior
155
- without the typed no-op prior completed at 35.48%, below the 35.59% top row;
156
  adding source-score to the typed no-op exact-mask row completed at 35.54%, also
157
  below the top row.
158
  The singleton near-miss revival diagnostic completed: exact `residual_near_miss`
159
- bonuses 0.01 and 0.02 both tie the 35.59% top row and slightly raise mean
160
  progress to 57.10%, but do not improve success.
161
- The original candidate-oracle prefix diagnostic (`14935471`) completed but is
162
- now treated as non-unique because the prefix contained duplicate policy/zero
163
- residual actions across residual-scale slots; its summaries were archived with
164
- the `_nonunique` suffix rather than used as current evidence. The evaluator now
165
- deduplicates candidate actions before oracle selection and blocks invalid padding
166
- branches from winning. The unique-action rerun is pending as GPU array
167
- `14953513`, with summary `14953522` and rebuild `14953524`. The clean
168
- compatible-chart margin `0.10` test completed at 34.67%, below the 35.59% top
169
- row, so simple lower abstention is not the answer.
 
 
170
 
171
  - `14858328`-`14858333`: completed train-split `field_selected_noexpert_bc5`;
172
  direct rollout is 26.84%, field-guided best is 27.65%.
@@ -334,7 +338,7 @@ row, so simple lower abstention is not the answer.
334
  array, summary, and rebuild. Dropping only the weak
335
  `near_miss+no_op` composite raises the clean row to 35.59% with seeds
336
  35.30%/34.78%/36.70%, progress 57.07%, and action MSE 0.4064. This is the
337
- current best clean deployment diagnostic.
338
  - `14915146`: completed CPU smoke for the exact compatibility mask combined
339
  with composite-only L2 trust penalty 0.02.
340
  - `14915213`/`14915217`/`14915251`: completed the exact compatibility + L2 0.02
@@ -424,12 +428,13 @@ row, so simple lower abstention is not the answer.
424
 
425
  - Promote same-state no-expert lattice (56.99%) as the conservative mechanism
426
  result.
427
- - Use K4 masked composed type-consensus residual transport with advantage
428
- abstention, a small typed no-op prior, field-gated tangent length calibration
429
- over `0.35/0.40/0.45`, and an exact near-miss/no-op compatibility mask as the
430
- current best clean deployment diagnostic, 35.59%, not as a SOTA claim. The
431
- previous mean-by-type scale-grid no-op row remains 35.42%; pure masked
432
- composition is 35.30%, the exact typed-prior composition row is 35.54%, and
 
433
  component-wise propagation of the no-op prior reaches 35.36%, so composition
434
  helps only when sparse exact priors are preserved and incompatible/anti-goal
435
  composite parts are masked. Composite-only L2 trust penalties lower action
 
16
  | Same-state local counterfactual proposals are the mechanism | near-miss-only lattice is 55.94%; removing expert+near_miss drops to 25.57% | Strongly supported |
17
  | Conservative same-state result is large | no-expert lattice is 56.99% vs 29.74% policy | Main result |
18
  | Full lattice gives upper result | full lattice is 69.33%, oracle is 86.78% | Strong but label expert proposal clearly |
19
+ | Deployment-clean proposal is currently a bottleneck | best clean K4 compatible tangent transport with a near-miss challenger gate is 36.00%, far below 56.99% | Supported |
20
  | Gradient-based field optimization does not solve the clean proposal gap | `field_optim` best observed result is 25.39% | Negative diagnostic |
21
  | A broader non-expert proposal target does not reduce the proposal gap | direct broad non-expert policy is 27.88%; with field scoring it is 26.49% | Negative diagnostic |
22
  | Counterfactual residuals transfer better than absolute retrieved actions | nearest residual retrieval is 32.12% vs absolute retrieval 28.93%; KNN4 residual drops to 29.91% | Supported as a clean bridge |
 
29
  | Clean residual transport behaves like sparse intervention | the best clean row abstains to zero-residual policy on 93.5% of states, while selected nonzero residuals succeed at 50.9% vs 34.5% for abstention | Stronger clean-mechanism framing |
30
  | Tangent consensus is close but needs sparse typing | K4 mean-by-type residual consensus reaches 34.96%; a small no-op residual prior plateau at 0.025-0.035 raises fixed-scale transport to 35.25% | Fixed-scale clean diagnostic |
31
  | Field-gated tangent length calibration improves the clean bridge | K4 mean-by-type scale grid 0.35/0.40/0.45 with no-op bonus 0.03 reaches 35.42%; the source-score version reaches 35.30%. Upper 0.40/0.45/0.50 nearly ties at 35.36%, while wide 0.35/0.45/0.55 drops to 35.13% | Previous clean best; local scale calibration, not a larger-step effect |
32
+ | Masked tangent composition gives a small clean lift | K4 composed type-consensus transport is 35.30% after anti-goal composite masking; exact `near_miss+no_op` masking alone reaches 35.48%; typed no-op prior reaches 35.54%; combining the exact compatibility mask with the typed prior reaches 35.59% | Previous clean best; controlled compatible tangent-chart composition, not unmasked proposal accumulation |
33
+ | Train-source score priors do not explain the compatible tangent-chart gain | replacing the typed no-op prior with a measured train-source reward-score prior on the exact compatibility chart reaches 35.48%; adding it to the typed no-op row reaches 35.54%, still below the previous 35.59% compatibility row | Negative diagnostic: compatibility masking and sparse typed abstention matter more than continuous train-source confidence on this chart |
34
+ | Singleton near-miss priors do not open a new clean route | adding a small exact `residual_near_miss` singleton prior 0.01 or 0.02 on the compatible chart ties the previous 35.59% compatibility row and slightly raises progress to 57.10% | Tie diagnostic: revived near-miss tangents are high-precision but too sparse to close the clean-to-same-state proposal gap |
35
+ | Candidate-prefix oracle needs unique-action hygiene | the first K=8 prefix diagnostic was archived as `_nonunique`; the deduplicated unique-action trace reaches 43.07% candidate-oracle success with mean best branch rank 2.85 | Supported diagnostic: proposal headroom is real, but branch success falls with rank, so selector calibration must be conditional |
36
+ | Near-miss challenger calibration improves clean deployment | the trace-motivated two-stage selector keeps the compatible-chart anchor and lets singleton near-miss tangents override only at margin 0.02; it reaches 36.00% and 57.42% progress | Current best clean row; small but story-aligned selector-calibration gain |
37
  | Component-wise composite priors do not add the gain | propagating the no-op bonus into composite types reaches 35.36%, below the exact-prior masked composition row at 35.54% and the exact compatibility row at 35.59% | Negative/near-tie diagnostic: sparse exact priors are better than broadly rewarding every no-op-containing composite |
38
  | Composite trust-radius penalty explains but does not improve the success top line | composite-only L2 penalty 0.02 ties 35.54% while lowering action MSE from 0.4106 to 0.4079; with the exact compatibility mask it lowers MSE further to 0.4048 but drops back to 35.54% | Tie/negative diagnostic: composed tangents need a local trust radius, but compatibility masking gives the success optimum |
39
  | Minimum-energy residual regularization does not add the gain | action L2 penalty 0.05 ties the previous 35.42% scale-grid row, while 0.10/0.20 reach 35.36% | Negative/tie diagnostic: the clean bridge is not explained by shortest-action bias |
 
47
  | Typed no-op residual prior improves the clean bridge | CPU smoke `14883591` passed; bonuses 0.025/0.03/0.035 tie at 35.25%, while 0.01/0.02/0.05/0.08 are slightly lower | Fixed-scale clean diagnostic |
48
  | Wrong-gripper typed prior does not add a new clean bridge | wrong-gripper-only reaches 35.19%; no-op+wrong-gripper 0.02 ties 35.25%; no-op+wrong-gripper 0.04 drops to 35.13% | Negative/tie diagnostic |
49
  | No-op-only residuals nearly preserve the fixed-scale clean bridge | excluding wrong-gripper residuals gives 35.19% with either no-op bonus 0.03 or source-score bonus 0.02, one success below the 35.25% fixed-scale safe-family plateau | Mechanism sharpened: wrong-gripper is marginal, not core |
50
+ | The proposal gap is now quantified | `paper_analysis.md` reports best clean +6.26 pp over canonical h16, same-state no-expert +27.25 pp, leaving a +20.99 pp clean-to-same-state gap | Core paper tension |
51
  | Policy fallback is not the same-state mechanism | adding a policy baseline candidate to the no-expert same-state lattice drops 56.99% to 40.70% even with margin 0.00 | Negative diagnostic |
52
  | Z-score retrieval metric does not help | z-score rows reach 32.23-32.81%, below raw retrieval | Negative diagnostic |
53
  | Task-relative actor-pose retrieval metric does not improve tangent transfer | K2 task-relative residual retrieval reaches 34.26% vs raw K2 35.01%; K4 task-relative mean-by-type + no-op reaches 34.43% vs raw K4 35.25% | Negative diagnostic |
 
82
  15. K4 mean-by-type tangent consensus + typed no-op prior 0.025-0.035: 35.25%
83
  16. K4 mean-by-type tangent consensus + scale-grid typed no-op prior: 35.42%
84
  17. K4 masked composed type-consensus tangent transport: 35.30%; with exact near-miss/no-op compatibility mask only: 35.48%; with typed no-op prior: 35.54%; with exact compatibility mask + typed no-op prior: 35.59%; adding singleton near-miss prior 0.01/0.02 ties 35.59%; with source-score prior on that exact chart: 35.48%; with typed no-op + source-score: 35.54%
85
+ 18. K4 compatible tangent unique-action candidate-oracle prefix K=8 diagnostic: 43.07%; branch trace shows mean best branch rank 2.85, unique count 8.00, and real but conditional selector headroom
86
+ 19. K4 compatible tangent near-miss challenger gate 0.02: 36.00%; current best clean deployment row from trace-motivated two-stage selector calibration
87
+ 20. K4 compatible tangent abstention margin 0.10 diagnostic: 34.67%; naive lower abstention hurts despite candidate-prefix oracle headroom
88
  20. K4 masked composed type-consensus tangent transport + component-wise no-op prior: 35.36%
89
  21. K4 masked composed type-consensus tangent transport + composite L2 trust penalty: 35.54% at 0.02; 35.48% at 0.05; 35.54% with exact compatibility mask + L2 0.02
90
  22. K4 mean-by-type tangent consensus + upper/wide scale diagnostics: 35.36% for 0.40/0.45/0.50; 35.13% for 0.35/0.45/0.55
 
141
 
142
  ## Job Status
143
 
144
+ Last checked: `2026-06-30 03:45 UTC`. The K4 masked composed type-consensus
145
+ sweep produced the previous clean best, 35.59%, when the exact
146
  `residual_near_miss+residual_no_op` composite is additionally masked. Pure
147
  masked composition reached 35.30%, exact compatibility masking without typed
148
  or source-score priors reached 35.48%, masked composition with the exact typed
 
154
  improve success: standalone L2 0.02 ties 35.54%, L2 0.05 reaches 35.48%, and
155
  exact compatibility + L2 0.02 reaches 35.54% with the lowest MSE.
156
  On the same exact compatibility chart, a train-measured source-score prior
157
+ without the typed no-op prior completed at 35.48%, below the previous 35.59% compatibility row;
158
  adding source-score to the typed no-op exact-mask row completed at 35.54%, also
159
  below the top row.
160
  The singleton near-miss revival diagnostic completed: exact `residual_near_miss`
161
+ bonuses 0.01 and 0.02 both tie the previous 35.59% compatibility row and slightly raise mean
162
  progress to 57.10%, but do not improve success.
163
+ The original candidate-oracle prefix diagnostic (`14935471`) is treated as
164
+ non-unique because the prefix contained duplicate policy/zero residual actions
165
+ across residual-scale slots; its summaries were archived with the `_nonunique`
166
+ suffix. The evaluator now deduplicates candidate actions before oracle selection
167
+ and blocks invalid padding branches from winning. The unique-action rerun
168
+ `14953513`/`14953522` completed at 43.07% candidate-oracle success, and the
169
+ branch-trace rerun `14953960`/`14953961` confirmed unique count 8.00 and mean
170
+ best branch rank 2.85. A trace-motivated clean near-miss challenger gate
171
+ (`14954280`/`14954281`) completed at 36.00%, the current clean best. The clean
172
+ compatible-chart margin `0.10` test completed at 34.67%, so simple lower
173
+ abstention is not the answer.
174
 
175
  - `14858328`-`14858333`: completed train-split `field_selected_noexpert_bc5`;
176
  direct rollout is 26.84%, field-guided best is 27.65%.
 
338
  array, summary, and rebuild. Dropping only the weak
339
  `near_miss+no_op` composite raises the clean row to 35.59% with seeds
340
  35.30%/34.78%/36.70%, progress 57.07%, and action MSE 0.4064. This is the
341
+ previous best clean deployment diagnostic before the near-miss challenger gate.
342
  - `14915146`: completed CPU smoke for the exact compatibility mask combined
343
  with composite-only L2 trust penalty 0.02.
344
  - `14915213`/`14915217`/`14915251`: completed the exact compatibility + L2 0.02
 
428
 
429
  - Promote same-state no-expert lattice (56.99%) as the conservative mechanism
430
  result.
431
+ - Use K4 compatible residual transport with advantage abstention, a small typed
432
+ no-op prior, field-gated tangent length calibration over `0.35/0.40/0.45`, an
433
+ exact near-miss/no-op compatibility mask, and the trace-motivated singleton
434
+ near-miss challenger gate as the current best clean deployment diagnostic,
435
+ 36.00%, not as a SOTA claim. The previous mean-by-type scale-grid no-op row
436
+ remains 35.42%; pure masked composition is 35.30%, the exact typed-prior
437
+ composition row is 35.54%, exact compatibility reaches 35.59%, and
438
  component-wise propagation of the no-op prior reaches 35.36%, so composition
439
  helps only when sparse exact priors are preserved and incompatible/anti-goal
440
  composite parts are masked. Composite-only L2 trust penalties lower action
scripts/build_paper_analysis.py CHANGED
@@ -401,6 +401,30 @@ METHODS = [
401
  "k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p02_summary.json"
402
  ),
403
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
404
  MethodSpec(
405
  key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
406
  label="K4 composed compatible tangents, no-op bonus 0.03, singleton near-miss bonus 0.01",
 
401
  "k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p02_summary.json"
402
  ),
403
  ),
404
+ MethodSpec(
405
+ key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger001",
406
+ label="K4 compatible tangents, no-op bonus 0.03, near-miss challenger gate 0.01",
407
+ summary_path=(
408
+ "h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_"
409
+ "k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p01_summary.json"
410
+ ),
411
+ ),
412
+ MethodSpec(
413
+ key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger003",
414
+ label="K4 compatible tangents, no-op bonus 0.03, near-miss challenger gate 0.03",
415
+ summary_path=(
416
+ "h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_"
417
+ "k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p03_summary.json"
418
+ ),
419
+ ),
420
+ MethodSpec(
421
+ key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmwgchallenger001",
422
+ label="K4 compatible tangents, no-op bonus 0.03, near-miss/wrong-gripper challenger gate 0.01",
423
+ summary_path=(
424
+ "h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_"
425
+ "k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmwgchallenger0p01_summary.json"
426
+ ),
427
+ ),
428
  MethodSpec(
429
  key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
430
  label="K4 composed compatible tangents, no-op bonus 0.03, singleton near-miss bonus 0.01",
scripts/build_paper_table_status.py CHANGED
@@ -755,6 +755,36 @@ SPECS = [
755
  story_role="trace-motivated two-stage selector calibration over the compatible local tangent chart",
756
  pending_job="14954280/14954281",
757
  ),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
758
  ResultSpec(
759
  key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
760
  label="K4 composed compatible residual retrieval, no-op bonus 0.03, singleton near-miss bonus 0.01",
 
755
  story_role="trace-motivated two-stage selector calibration over the compatible local tangent chart",
756
  pending_job="14954280/14954281",
757
  ),
758
+ ResultSpec(
759
+ key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger001",
760
+ label="K4 compatible residual retrieval, near-miss challenger gate 0.01",
761
+ path="h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p01_summary.json",
762
+ clean_deployment="yes",
763
+ same_state_proposals="no",
764
+ expert_proposal="no",
765
+ story_role="lower-margin sensitivity for trace-motivated singleton near-miss challenger calibration",
766
+ pending_job="14954526/14954527",
767
+ ),
768
+ ResultSpec(
769
+ key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger003",
770
+ label="K4 compatible residual retrieval, near-miss challenger gate 0.03",
771
+ path="h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p03_summary.json",
772
+ clean_deployment="yes",
773
+ same_state_proposals="no",
774
+ expert_proposal="no",
775
+ story_role="upper-margin sensitivity for trace-motivated singleton near-miss challenger calibration",
776
+ pending_job="14954528/14954529",
777
+ ),
778
+ ResultSpec(
779
+ key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmwgchallenger001",
780
+ label="K4 compatible residual retrieval, near-miss/wrong-gripper challenger gate 0.01",
781
+ path="h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmwgchallenger0p01_summary.json",
782
+ clean_deployment="yes",
783
+ same_state_proposals="no",
784
+ expert_proposal="no",
785
+ story_role="trace-motivated challenger calibration test for whether wrong-gripper residuals carry conditional selector headroom",
786
+ pending_job="14954530/14954531",
787
+ ),
788
  ResultSpec(
789
  key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
790
  label="K4 composed compatible residual retrieval, no-op bonus 0.03, singleton near-miss bonus 0.01",