Auto-sync: 2026-06-30 03:55:26 (part 2)
Browse files- results/paper_core_results.md +18 -14
- results/paper_story_memo.md +33 -28
- scripts/build_paper_analysis.py +24 -0
- scripts/build_paper_table_status.py +30 -0
results/paper_core_results.md
CHANGED
|
@@ -5,10 +5,10 @@ baseline is the h=16 rank-checkpoint online rollout (`29.74%`).
|
|
| 5 |
|
| 6 |
For paired seed deltas, per-task gaps, and selection histograms, regenerate and
|
| 7 |
read `paper_analysis.md` with `python3 scripts/build_paper_analysis.py`. Current
|
| 8 |
-
paired analysis: best clean K4
|
| 9 |
-
|
| 10 |
lattice is `+27.25 pp`, and the remaining clean-to-same-state proposal gap is
|
| 11 |
-
`+
|
| 12 |
|
| 13 |
| Method | Uses same-state proposals | Uses expert proposal | Success | Gain vs policy | Interpretation |
|
| 14 |
|---|---:|---:|---:|---:|---|
|
|
@@ -56,14 +56,15 @@ lattice is `+27.25 pp`, and the remaining clean-to-same-state proposal gap is
|
|
| 56 |
| K4 composed type-consensus residual retrieval, masked | No | No | 35.30% | +5.57 pp | Clean anti-goal composite masking removes the confound; tangent composition alone near-ties but does not beat mean-by-type scale-grid transport |
|
| 57 |
| K4 composed type-consensus residual retrieval, masked + no-op prior 0.03 | No | No | 35.54% | +5.80 pp | Previous clean best; masked local tangent composition adds one success over the scale-grid no-op row while staying sparse |
|
| 58 |
| K4 composed type-consensus residual retrieval, exact compatibility mask without typed prior | No | No | 35.48% | +5.74 pp | Isolates the algebraic compatibility mask: it improves pure masked composition but does not fully replace the sparse typed no-op prior |
|
| 59 |
-
| K4 composed type-consensus residual retrieval, no-op prior + exact near-miss/no-op incompatibility mask | No | No | 35.59% | +5.86 pp |
|
| 60 |
| K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/near-miss singleton priors | No | No | 35.59% | +5.86 pp | Exact singleton near-miss priors 0.01/0.02 tie the top row and raise progress slightly; transferred near-miss tangents are high-precision but too sparse to close the proposal gap |
|
| 61 |
-
| K4 compatible residual retrieval, unique candidate-oracle prefix K=8 | No | No |
|
| 62 |
-
| K4 compatible residual retrieval
|
|
|
|
| 63 |
| K4 composed type-consensus residual retrieval, exact compatibility mask + source-score prior | No | No | 35.48% | +5.74 pp | Measured train-source reward confidence does not replace the sparse typed prior on the compatible chart; useful negative calibration |
|
| 64 |
| K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/source-score priors | No | No | 35.54% | +5.80 pp | Adding measured train-source reward confidence to the current typed-prior chart ties near-best but does not beat exact compatibility masking alone |
|
| 65 |
| K4 composed type-consensus residual retrieval, masked + component-wise no-op prior | No | No | 35.36% | +5.62 pp | Propagating the no-op prior into every composite over-selects composed tangents and drops below the exact-prior row; useful negative calibration |
|
| 66 |
-
| K4 composed type-consensus residual retrieval, masked + composite trust penalty | No | No | 35.48-35.54% | +5.74-5.80 pp | Composite-only L2 lowers action MSE but does not improve success; combined with the exact compatibility mask it reaches 35.54% with the lowest MSE, one seed below the 35.59%
|
| 67 |
| K4 repair-tangent residual transport | No | No | 34.14-34.43% | +4.41-4.70 pp | Reversing residuals into failure-to-expert repair tangents is a clean negative diagnostic; the current gain is not recovered by transporting near-miss-to-expert vectors |
|
| 68 |
| K4 mean-by-type residual retrieval + source-score prior 0.025 | No | No | 35.19% | +5.45 pp | A stronger reward-score prior drops below the plateau |
|
| 69 |
| K4 mean-by-type residual retrieval, no-op-only residuals | No | No | 35.19% | +5.45 pp | Removing wrong-gripper residuals loses one success versus the fixed-scale safe-family plateau; the core gain is sparse no-op/tangent repair, with wrong-gripper acting only as a marginal helper |
|
|
@@ -112,9 +113,10 @@ Suggested main-table rows:
|
|
| 112 |
14. K4 mean-by-type residual retrieval + scale-grid no-op prior 0.03
|
| 113 |
15. K4 masked composed type-consensus residual retrieval + no-op prior 0.03
|
| 114 |
16. K4 masked composed type-consensus residual retrieval + exact near-miss/no-op compatibility mask, with and without typed no-op prior
|
| 115 |
-
17. K4
|
| 116 |
-
18. K4 masked composed type-consensus residual retrieval +
|
| 117 |
-
19. K4 masked composed type-consensus residual retrieval +
|
|
|
|
| 118 |
20. K4 mean-by-type residual retrieval + upper/wide tangent-length diagnostics
|
| 119 |
21. K4 mean-by-type residual retrieval + minimum-energy action penalty diagnostics
|
| 120 |
22. K4 mean-by-type residual retrieval + source-progress/source-score/source-advantage prior diagnostics
|
|
@@ -142,10 +144,12 @@ Suggested claim:
|
|
| 142 |
> DoVLA-CIL is not a better behavior-cloning policy; it is a local counterfactual action
|
| 143 |
> selection rule. Deployment-clean K4 consensus residual transport with advantage
|
| 144 |
> abstention, a small typed no-op prior, field-gated tangent length calibration,
|
| 145 |
-
>
|
| 146 |
-
> the
|
| 147 |
-
> `near_miss+no_op` composite while
|
| 148 |
-
> near-miss/wrong-gripper tangents
|
|
|
|
|
|
|
| 149 |
> compatible chart drops to 35.48%, so the result is not explained by generic
|
| 150 |
> train-confidence weighting.
|
| 151 |
> extending the scale grid upward and adding minimum-energy action regularization
|
|
|
|
| 5 |
|
| 6 |
For paired seed deltas, per-task gaps, and selection histograms, regenerate and
|
| 7 |
read `paper_analysis.md` with `python3 scripts/build_paper_analysis.py`. Current
|
| 8 |
+
paired analysis: best clean K4 compatible tangent transport with a trace-motivated
|
| 9 |
+
near-miss challenger gate is `+6.26 pp` over canonical h=16, same-state no-expert
|
| 10 |
lattice is `+27.25 pp`, and the remaining clean-to-same-state proposal gap is
|
| 11 |
+
`+20.99 pp`.
|
| 12 |
|
| 13 |
| Method | Uses same-state proposals | Uses expert proposal | Success | Gain vs policy | Interpretation |
|
| 14 |
|---|---:|---:|---:|---:|---|
|
|
|
|
| 56 |
| K4 composed type-consensus residual retrieval, masked | No | No | 35.30% | +5.57 pp | Clean anti-goal composite masking removes the confound; tangent composition alone near-ties but does not beat mean-by-type scale-grid transport |
|
| 57 |
| K4 composed type-consensus residual retrieval, masked + no-op prior 0.03 | No | No | 35.54% | +5.80 pp | Previous clean best; masked local tangent composition adds one success over the scale-grid no-op row while staying sparse |
|
| 58 |
| K4 composed type-consensus residual retrieval, exact compatibility mask without typed prior | No | No | 35.48% | +5.74 pp | Isolates the algebraic compatibility mask: it improves pure masked composition but does not fully replace the sparse typed no-op prior |
|
| 59 |
+
| K4 composed type-consensus residual retrieval, no-op prior + exact near-miss/no-op incompatibility mask | No | No | 35.59% | +5.86 pp | Previous clean best; dropping only the weak `near_miss+no_op` composite preserves useful singleton/composite tangents and lowers action MSE |
|
| 60 |
| K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/near-miss singleton priors | No | No | 35.59% | +5.86 pp | Exact singleton near-miss priors 0.01/0.02 tie the top row and raise progress slightly; transferred near-miss tangents are high-precision but too sparse to close the proposal gap |
|
| 61 |
+
| K4 compatible residual retrieval, unique candidate-oracle prefix K=8 | No | No | 43.07% diagnostic | +13.33 pp diagnostic | Diagnostic-only measured oracle over generated clean candidate prefix; unique-action trace shows real selector headroom, not duplicate-action inflation |
|
| 62 |
+
| K4 compatible residual retrieval + near-miss challenger gate 0.02 | No | No | 36.00% | +6.26 pp | Current best clean deployment row; a two-stage selector keeps the robust compatible-chart anchor and only lets singleton near-miss tangents override when the field margin is positive |
|
| 63 |
+
| K4 compatible residual retrieval, margin 0.10 | No | No | 34.67% | +4.93 pp | Naively lowering abstention selects too many bad nonzero tangents and falls below the 36.00% top row |
|
| 64 |
| K4 composed type-consensus residual retrieval, exact compatibility mask + source-score prior | No | No | 35.48% | +5.74 pp | Measured train-source reward confidence does not replace the sparse typed prior on the compatible chart; useful negative calibration |
|
| 65 |
| K4 composed type-consensus residual retrieval, exact compatibility mask + no-op/source-score priors | No | No | 35.54% | +5.80 pp | Adding measured train-source reward confidence to the current typed-prior chart ties near-best but does not beat exact compatibility masking alone |
|
| 66 |
| K4 composed type-consensus residual retrieval, masked + component-wise no-op prior | No | No | 35.36% | +5.62 pp | Propagating the no-op prior into every composite over-selects composed tangents and drops below the exact-prior row; useful negative calibration |
|
| 67 |
+
| K4 composed type-consensus residual retrieval, masked + composite trust penalty | No | No | 35.48-35.54% | +5.74-5.80 pp | Composite-only L2 lowers action MSE but does not improve success; combined with the exact compatibility mask it reaches 35.54% with the lowest MSE, one seed below the previous 35.59% compatibility row |
|
| 68 |
| K4 repair-tangent residual transport | No | No | 34.14-34.43% | +4.41-4.70 pp | Reversing residuals into failure-to-expert repair tangents is a clean negative diagnostic; the current gain is not recovered by transporting near-miss-to-expert vectors |
|
| 69 |
| K4 mean-by-type residual retrieval + source-score prior 0.025 | No | No | 35.19% | +5.45 pp | A stronger reward-score prior drops below the plateau |
|
| 70 |
| K4 mean-by-type residual retrieval, no-op-only residuals | No | No | 35.19% | +5.45 pp | Removing wrong-gripper residuals loses one success versus the fixed-scale safe-family plateau; the core gain is sparse no-op/tangent repair, with wrong-gripper acting only as a marginal helper |
|
|
|
|
| 113 |
14. K4 mean-by-type residual retrieval + scale-grid no-op prior 0.03
|
| 114 |
15. K4 masked composed type-consensus residual retrieval + no-op prior 0.03
|
| 115 |
16. K4 masked composed type-consensus residual retrieval + exact near-miss/no-op compatibility mask, with and without typed no-op prior
|
| 116 |
+
17. K4 compatible tangent near-miss challenger gate 0.02
|
| 117 |
+
18. K4 masked composed type-consensus residual retrieval + source-score prior on the exact compatibility chart
|
| 118 |
+
19. K4 masked composed type-consensus residual retrieval + component-wise no-op prior diagnostic
|
| 119 |
+
20. K4 masked composed type-consensus residual retrieval + composite trust penalty diagnostic
|
| 120 |
20. K4 mean-by-type residual retrieval + upper/wide tangent-length diagnostics
|
| 121 |
21. K4 mean-by-type residual retrieval + minimum-energy action penalty diagnostics
|
| 122 |
22. K4 mean-by-type residual retrieval + source-progress/source-score/source-advantage prior diagnostics
|
|
|
|
| 144 |
> DoVLA-CIL is not a better behavior-cloning policy; it is a local counterfactual action
|
| 145 |
> selection rule. Deployment-clean K4 consensus residual transport with advantage
|
| 146 |
> abstention, a small typed no-op prior, field-gated tangent length calibration,
|
| 147 |
+
> masked local tangent composition, and a trace-motivated near-miss challenger
|
| 148 |
+
> gate give the strongest clean gain so far. The previous top row improves by
|
| 149 |
+
> removing only the semantically incompatible `near_miss+no_op` composite while
|
| 150 |
+
> preserving useful singleton and near-miss/wrong-gripper tangents; the current
|
| 151 |
+
> clean top row adds a second-stage selector that lets singleton near-miss
|
| 152 |
+
> tangents override the robust anchor only under a positive field margin. A train-source score prior on the same
|
| 153 |
> compatible chart drops to 35.48%, so the result is not explained by generic
|
| 154 |
> train-confidence weighting.
|
| 155 |
> extending the scale grid upward and adding minimum-energy action regularization
|
results/paper_story_memo.md
CHANGED
|
@@ -16,7 +16,7 @@ when queried on proposal geometry that matches those local counterfactuals.
|
|
| 16 |
| Same-state local counterfactual proposals are the mechanism | near-miss-only lattice is 55.94%; removing expert+near_miss drops to 25.57% | Strongly supported |
|
| 17 |
| Conservative same-state result is large | no-expert lattice is 56.99% vs 29.74% policy | Main result |
|
| 18 |
| Full lattice gives upper result | full lattice is 69.33%, oracle is 86.78% | Strong but label expert proposal clearly |
|
| 19 |
-
| Deployment-clean proposal is currently a bottleneck | best clean K4
|
| 20 |
| Gradient-based field optimization does not solve the clean proposal gap | `field_optim` best observed result is 25.39% | Negative diagnostic |
|
| 21 |
| A broader non-expert proposal target does not reduce the proposal gap | direct broad non-expert policy is 27.88%; with field scoring it is 26.49% | Negative diagnostic |
|
| 22 |
| Counterfactual residuals transfer better than absolute retrieved actions | nearest residual retrieval is 32.12% vs absolute retrieval 28.93%; KNN4 residual drops to 29.91% | Supported as a clean bridge |
|
|
@@ -29,10 +29,11 @@ when queried on proposal geometry that matches those local counterfactuals.
|
|
| 29 |
| Clean residual transport behaves like sparse intervention | the best clean row abstains to zero-residual policy on 93.5% of states, while selected nonzero residuals succeed at 50.9% vs 34.5% for abstention | Stronger clean-mechanism framing |
|
| 30 |
| Tangent consensus is close but needs sparse typing | K4 mean-by-type residual consensus reaches 34.96%; a small no-op residual prior plateau at 0.025-0.035 raises fixed-scale transport to 35.25% | Fixed-scale clean diagnostic |
|
| 31 |
| Field-gated tangent length calibration improves the clean bridge | K4 mean-by-type scale grid 0.35/0.40/0.45 with no-op bonus 0.03 reaches 35.42%; the source-score version reaches 35.30%. Upper 0.40/0.45/0.50 nearly ties at 35.36%, while wide 0.35/0.45/0.55 drops to 35.13% | Previous clean best; local scale calibration, not a larger-step effect |
|
| 32 |
-
| Masked tangent composition gives a small clean lift | K4 composed type-consensus transport is 35.30% after anti-goal composite masking; exact `near_miss+no_op` masking alone reaches 35.48%; typed no-op prior reaches 35.54%; combining the exact compatibility mask with the typed prior reaches 35.59% |
|
| 33 |
-
| Train-source score priors do not explain the compatible tangent-chart gain | replacing the typed no-op prior with a measured train-source reward-score prior on the exact compatibility chart reaches 35.48%; adding it to the typed no-op row reaches 35.54%, still below the 35.59%
|
| 34 |
-
| Singleton near-miss priors do not open a new clean route | adding a small exact `residual_near_miss` singleton prior 0.01 or 0.02 on the compatible chart ties the 35.59%
|
| 35 |
-
| Candidate-prefix oracle needs unique-action hygiene | the first K=8 prefix diagnostic
|
|
|
|
| 36 |
| Component-wise composite priors do not add the gain | propagating the no-op bonus into composite types reaches 35.36%, below the exact-prior masked composition row at 35.54% and the exact compatibility row at 35.59% | Negative/near-tie diagnostic: sparse exact priors are better than broadly rewarding every no-op-containing composite |
|
| 37 |
| Composite trust-radius penalty explains but does not improve the success top line | composite-only L2 penalty 0.02 ties 35.54% while lowering action MSE from 0.4106 to 0.4079; with the exact compatibility mask it lowers MSE further to 0.4048 but drops back to 35.54% | Tie/negative diagnostic: composed tangents need a local trust radius, but compatibility masking gives the success optimum |
|
| 38 |
| Minimum-energy residual regularization does not add the gain | action L2 penalty 0.05 ties the previous 35.42% scale-grid row, while 0.10/0.20 reach 35.36% | Negative/tie diagnostic: the clean bridge is not explained by shortest-action bias |
|
|
@@ -46,7 +47,7 @@ when queried on proposal geometry that matches those local counterfactuals.
|
|
| 46 |
| Typed no-op residual prior improves the clean bridge | CPU smoke `14883591` passed; bonuses 0.025/0.03/0.035 tie at 35.25%, while 0.01/0.02/0.05/0.08 are slightly lower | Fixed-scale clean diagnostic |
|
| 47 |
| Wrong-gripper typed prior does not add a new clean bridge | wrong-gripper-only reaches 35.19%; no-op+wrong-gripper 0.02 ties 35.25%; no-op+wrong-gripper 0.04 drops to 35.13% | Negative/tie diagnostic |
|
| 48 |
| No-op-only residuals nearly preserve the fixed-scale clean bridge | excluding wrong-gripper residuals gives 35.19% with either no-op bonus 0.03 or source-score bonus 0.02, one success below the 35.25% fixed-scale safe-family plateau | Mechanism sharpened: wrong-gripper is marginal, not core |
|
| 49 |
-
| The proposal gap is now quantified | `paper_analysis.md` reports best clean +
|
| 50 |
| Policy fallback is not the same-state mechanism | adding a policy baseline candidate to the no-expert same-state lattice drops 56.99% to 40.70% even with margin 0.00 | Negative diagnostic |
|
| 51 |
| Z-score retrieval metric does not help | z-score rows reach 32.23-32.81%, below raw retrieval | Negative diagnostic |
|
| 52 |
| Task-relative actor-pose retrieval metric does not improve tangent transfer | K2 task-relative residual retrieval reaches 34.26% vs raw K2 35.01%; K4 task-relative mean-by-type + no-op reaches 34.43% vs raw K4 35.25% | Negative diagnostic |
|
|
@@ -81,8 +82,9 @@ clean proposal result, the intended main rows are:
|
|
| 81 |
15. K4 mean-by-type tangent consensus + typed no-op prior 0.025-0.035: 35.25%
|
| 82 |
16. K4 mean-by-type tangent consensus + scale-grid typed no-op prior: 35.42%
|
| 83 |
17. K4 masked composed type-consensus tangent transport: 35.30%; with exact near-miss/no-op compatibility mask only: 35.48%; with typed no-op prior: 35.54%; with exact compatibility mask + typed no-op prior: 35.59%; adding singleton near-miss prior 0.01/0.02 ties 35.59%; with source-score prior on that exact chart: 35.48%; with typed no-op + source-score: 35.54%
|
| 84 |
-
18. K4 compatible tangent unique-action candidate-oracle prefix K=8 diagnostic:
|
| 85 |
-
19. K4 compatible tangent
|
|
|
|
| 86 |
20. K4 masked composed type-consensus tangent transport + component-wise no-op prior: 35.36%
|
| 87 |
21. K4 masked composed type-consensus tangent transport + composite L2 trust penalty: 35.54% at 0.02; 35.48% at 0.05; 35.54% with exact compatibility mask + L2 0.02
|
| 88 |
22. K4 mean-by-type tangent consensus + upper/wide scale diagnostics: 35.36% for 0.40/0.45/0.50; 35.13% for 0.35/0.45/0.55
|
|
@@ -139,8 +141,8 @@ test-time search. The cleaner novelty is:
|
|
| 139 |
|
| 140 |
## Job Status
|
| 141 |
|
| 142 |
-
Last checked: `2026-06-30
|
| 143 |
-
sweep
|
| 144 |
`residual_near_miss+residual_no_op` composite is additionally masked. Pure
|
| 145 |
masked composition reached 35.30%, exact compatibility masking without typed
|
| 146 |
or source-score priors reached 35.48%, masked composition with the exact typed
|
|
@@ -152,21 +154,23 @@ no-op prior row. Composite-only L2 trust penalties reduce action MSE but do not
|
|
| 152 |
improve success: standalone L2 0.02 ties 35.54%, L2 0.05 reaches 35.48%, and
|
| 153 |
exact compatibility + L2 0.02 reaches 35.54% with the lowest MSE.
|
| 154 |
On the same exact compatibility chart, a train-measured source-score prior
|
| 155 |
-
without the typed no-op prior completed at 35.48%, below the 35.59%
|
| 156 |
adding source-score to the typed no-op exact-mask row completed at 35.54%, also
|
| 157 |
below the top row.
|
| 158 |
The singleton near-miss revival diagnostic completed: exact `residual_near_miss`
|
| 159 |
-
bonuses 0.01 and 0.02 both tie the 35.59%
|
| 160 |
progress to 57.10%, but do not improve success.
|
| 161 |
-
The original candidate-oracle prefix diagnostic (`14935471`)
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
|
|
|
|
|
|
| 170 |
|
| 171 |
- `14858328`-`14858333`: completed train-split `field_selected_noexpert_bc5`;
|
| 172 |
direct rollout is 26.84%, field-guided best is 27.65%.
|
|
@@ -334,7 +338,7 @@ row, so simple lower abstention is not the answer.
|
|
| 334 |
array, summary, and rebuild. Dropping only the weak
|
| 335 |
`near_miss+no_op` composite raises the clean row to 35.59% with seeds
|
| 336 |
35.30%/34.78%/36.70%, progress 57.07%, and action MSE 0.4064. This is the
|
| 337 |
-
|
| 338 |
- `14915146`: completed CPU smoke for the exact compatibility mask combined
|
| 339 |
with composite-only L2 trust penalty 0.02.
|
| 340 |
- `14915213`/`14915217`/`14915251`: completed the exact compatibility + L2 0.02
|
|
@@ -424,12 +428,13 @@ row, so simple lower abstention is not the answer.
|
|
| 424 |
|
| 425 |
- Promote same-state no-expert lattice (56.99%) as the conservative mechanism
|
| 426 |
result.
|
| 427 |
-
- Use K4
|
| 428 |
-
|
| 429 |
-
|
| 430 |
-
current best clean deployment diagnostic,
|
| 431 |
-
previous mean-by-type scale-grid no-op row
|
| 432 |
-
composition is 35.30%, the exact typed-prior
|
|
|
|
| 433 |
component-wise propagation of the no-op prior reaches 35.36%, so composition
|
| 434 |
helps only when sparse exact priors are preserved and incompatible/anti-goal
|
| 435 |
composite parts are masked. Composite-only L2 trust penalties lower action
|
|
|
|
| 16 |
| Same-state local counterfactual proposals are the mechanism | near-miss-only lattice is 55.94%; removing expert+near_miss drops to 25.57% | Strongly supported |
|
| 17 |
| Conservative same-state result is large | no-expert lattice is 56.99% vs 29.74% policy | Main result |
|
| 18 |
| Full lattice gives upper result | full lattice is 69.33%, oracle is 86.78% | Strong but label expert proposal clearly |
|
| 19 |
+
| Deployment-clean proposal is currently a bottleneck | best clean K4 compatible tangent transport with a near-miss challenger gate is 36.00%, far below 56.99% | Supported |
|
| 20 |
| Gradient-based field optimization does not solve the clean proposal gap | `field_optim` best observed result is 25.39% | Negative diagnostic |
|
| 21 |
| A broader non-expert proposal target does not reduce the proposal gap | direct broad non-expert policy is 27.88%; with field scoring it is 26.49% | Negative diagnostic |
|
| 22 |
| Counterfactual residuals transfer better than absolute retrieved actions | nearest residual retrieval is 32.12% vs absolute retrieval 28.93%; KNN4 residual drops to 29.91% | Supported as a clean bridge |
|
|
|
|
| 29 |
| Clean residual transport behaves like sparse intervention | the best clean row abstains to zero-residual policy on 93.5% of states, while selected nonzero residuals succeed at 50.9% vs 34.5% for abstention | Stronger clean-mechanism framing |
|
| 30 |
| Tangent consensus is close but needs sparse typing | K4 mean-by-type residual consensus reaches 34.96%; a small no-op residual prior plateau at 0.025-0.035 raises fixed-scale transport to 35.25% | Fixed-scale clean diagnostic |
|
| 31 |
| Field-gated tangent length calibration improves the clean bridge | K4 mean-by-type scale grid 0.35/0.40/0.45 with no-op bonus 0.03 reaches 35.42%; the source-score version reaches 35.30%. Upper 0.40/0.45/0.50 nearly ties at 35.36%, while wide 0.35/0.45/0.55 drops to 35.13% | Previous clean best; local scale calibration, not a larger-step effect |
|
| 32 |
+
| Masked tangent composition gives a small clean lift | K4 composed type-consensus transport is 35.30% after anti-goal composite masking; exact `near_miss+no_op` masking alone reaches 35.48%; typed no-op prior reaches 35.54%; combining the exact compatibility mask with the typed prior reaches 35.59% | Previous clean best; controlled compatible tangent-chart composition, not unmasked proposal accumulation |
|
| 33 |
+
| Train-source score priors do not explain the compatible tangent-chart gain | replacing the typed no-op prior with a measured train-source reward-score prior on the exact compatibility chart reaches 35.48%; adding it to the typed no-op row reaches 35.54%, still below the previous 35.59% compatibility row | Negative diagnostic: compatibility masking and sparse typed abstention matter more than continuous train-source confidence on this chart |
|
| 34 |
+
| Singleton near-miss priors do not open a new clean route | adding a small exact `residual_near_miss` singleton prior 0.01 or 0.02 on the compatible chart ties the previous 35.59% compatibility row and slightly raises progress to 57.10% | Tie diagnostic: revived near-miss tangents are high-precision but too sparse to close the clean-to-same-state proposal gap |
|
| 35 |
+
| Candidate-prefix oracle needs unique-action hygiene | the first K=8 prefix diagnostic was archived as `_nonunique`; the deduplicated unique-action trace reaches 43.07% candidate-oracle success with mean best branch rank 2.85 | Supported diagnostic: proposal headroom is real, but branch success falls with rank, so selector calibration must be conditional |
|
| 36 |
+
| Near-miss challenger calibration improves clean deployment | the trace-motivated two-stage selector keeps the compatible-chart anchor and lets singleton near-miss tangents override only at margin 0.02; it reaches 36.00% and 57.42% progress | Current best clean row; small but story-aligned selector-calibration gain |
|
| 37 |
| Component-wise composite priors do not add the gain | propagating the no-op bonus into composite types reaches 35.36%, below the exact-prior masked composition row at 35.54% and the exact compatibility row at 35.59% | Negative/near-tie diagnostic: sparse exact priors are better than broadly rewarding every no-op-containing composite |
|
| 38 |
| Composite trust-radius penalty explains but does not improve the success top line | composite-only L2 penalty 0.02 ties 35.54% while lowering action MSE from 0.4106 to 0.4079; with the exact compatibility mask it lowers MSE further to 0.4048 but drops back to 35.54% | Tie/negative diagnostic: composed tangents need a local trust radius, but compatibility masking gives the success optimum |
|
| 39 |
| Minimum-energy residual regularization does not add the gain | action L2 penalty 0.05 ties the previous 35.42% scale-grid row, while 0.10/0.20 reach 35.36% | Negative/tie diagnostic: the clean bridge is not explained by shortest-action bias |
|
|
|
|
| 47 |
| Typed no-op residual prior improves the clean bridge | CPU smoke `14883591` passed; bonuses 0.025/0.03/0.035 tie at 35.25%, while 0.01/0.02/0.05/0.08 are slightly lower | Fixed-scale clean diagnostic |
|
| 48 |
| Wrong-gripper typed prior does not add a new clean bridge | wrong-gripper-only reaches 35.19%; no-op+wrong-gripper 0.02 ties 35.25%; no-op+wrong-gripper 0.04 drops to 35.13% | Negative/tie diagnostic |
|
| 49 |
| No-op-only residuals nearly preserve the fixed-scale clean bridge | excluding wrong-gripper residuals gives 35.19% with either no-op bonus 0.03 or source-score bonus 0.02, one success below the 35.25% fixed-scale safe-family plateau | Mechanism sharpened: wrong-gripper is marginal, not core |
|
| 50 |
+
| The proposal gap is now quantified | `paper_analysis.md` reports best clean +6.26 pp over canonical h16, same-state no-expert +27.25 pp, leaving a +20.99 pp clean-to-same-state gap | Core paper tension |
|
| 51 |
| Policy fallback is not the same-state mechanism | adding a policy baseline candidate to the no-expert same-state lattice drops 56.99% to 40.70% even with margin 0.00 | Negative diagnostic |
|
| 52 |
| Z-score retrieval metric does not help | z-score rows reach 32.23-32.81%, below raw retrieval | Negative diagnostic |
|
| 53 |
| Task-relative actor-pose retrieval metric does not improve tangent transfer | K2 task-relative residual retrieval reaches 34.26% vs raw K2 35.01%; K4 task-relative mean-by-type + no-op reaches 34.43% vs raw K4 35.25% | Negative diagnostic |
|
|
|
|
| 82 |
15. K4 mean-by-type tangent consensus + typed no-op prior 0.025-0.035: 35.25%
|
| 83 |
16. K4 mean-by-type tangent consensus + scale-grid typed no-op prior: 35.42%
|
| 84 |
17. K4 masked composed type-consensus tangent transport: 35.30%; with exact near-miss/no-op compatibility mask only: 35.48%; with typed no-op prior: 35.54%; with exact compatibility mask + typed no-op prior: 35.59%; adding singleton near-miss prior 0.01/0.02 ties 35.59%; with source-score prior on that exact chart: 35.48%; with typed no-op + source-score: 35.54%
|
| 85 |
+
18. K4 compatible tangent unique-action candidate-oracle prefix K=8 diagnostic: 43.07%; branch trace shows mean best branch rank 2.85, unique count 8.00, and real but conditional selector headroom
|
| 86 |
+
19. K4 compatible tangent near-miss challenger gate 0.02: 36.00%; current best clean deployment row from trace-motivated two-stage selector calibration
|
| 87 |
+
20. K4 compatible tangent abstention margin 0.10 diagnostic: 34.67%; naive lower abstention hurts despite candidate-prefix oracle headroom
|
| 88 |
20. K4 masked composed type-consensus tangent transport + component-wise no-op prior: 35.36%
|
| 89 |
21. K4 masked composed type-consensus tangent transport + composite L2 trust penalty: 35.54% at 0.02; 35.48% at 0.05; 35.54% with exact compatibility mask + L2 0.02
|
| 90 |
22. K4 mean-by-type tangent consensus + upper/wide scale diagnostics: 35.36% for 0.40/0.45/0.50; 35.13% for 0.35/0.45/0.55
|
|
|
|
| 141 |
|
| 142 |
## Job Status
|
| 143 |
|
| 144 |
+
Last checked: `2026-06-30 03:45 UTC`. The K4 masked composed type-consensus
|
| 145 |
+
sweep produced the previous clean best, 35.59%, when the exact
|
| 146 |
`residual_near_miss+residual_no_op` composite is additionally masked. Pure
|
| 147 |
masked composition reached 35.30%, exact compatibility masking without typed
|
| 148 |
or source-score priors reached 35.48%, masked composition with the exact typed
|
|
|
|
| 154 |
improve success: standalone L2 0.02 ties 35.54%, L2 0.05 reaches 35.48%, and
|
| 155 |
exact compatibility + L2 0.02 reaches 35.54% with the lowest MSE.
|
| 156 |
On the same exact compatibility chart, a train-measured source-score prior
|
| 157 |
+
without the typed no-op prior completed at 35.48%, below the previous 35.59% compatibility row;
|
| 158 |
adding source-score to the typed no-op exact-mask row completed at 35.54%, also
|
| 159 |
below the top row.
|
| 160 |
The singleton near-miss revival diagnostic completed: exact `residual_near_miss`
|
| 161 |
+
bonuses 0.01 and 0.02 both tie the previous 35.59% compatibility row and slightly raise mean
|
| 162 |
progress to 57.10%, but do not improve success.
|
| 163 |
+
The original candidate-oracle prefix diagnostic (`14935471`) is treated as
|
| 164 |
+
non-unique because the prefix contained duplicate policy/zero residual actions
|
| 165 |
+
across residual-scale slots; its summaries were archived with the `_nonunique`
|
| 166 |
+
suffix. The evaluator now deduplicates candidate actions before oracle selection
|
| 167 |
+
and blocks invalid padding branches from winning. The unique-action rerun
|
| 168 |
+
`14953513`/`14953522` completed at 43.07% candidate-oracle success, and the
|
| 169 |
+
branch-trace rerun `14953960`/`14953961` confirmed unique count 8.00 and mean
|
| 170 |
+
best branch rank 2.85. A trace-motivated clean near-miss challenger gate
|
| 171 |
+
(`14954280`/`14954281`) completed at 36.00%, the current clean best. The clean
|
| 172 |
+
compatible-chart margin `0.10` test completed at 34.67%, so simple lower
|
| 173 |
+
abstention is not the answer.
|
| 174 |
|
| 175 |
- `14858328`-`14858333`: completed train-split `field_selected_noexpert_bc5`;
|
| 176 |
direct rollout is 26.84%, field-guided best is 27.65%.
|
|
|
|
| 338 |
array, summary, and rebuild. Dropping only the weak
|
| 339 |
`near_miss+no_op` composite raises the clean row to 35.59% with seeds
|
| 340 |
35.30%/34.78%/36.70%, progress 57.07%, and action MSE 0.4064. This is the
|
| 341 |
+
previous best clean deployment diagnostic before the near-miss challenger gate.
|
| 342 |
- `14915146`: completed CPU smoke for the exact compatibility mask combined
|
| 343 |
with composite-only L2 trust penalty 0.02.
|
| 344 |
- `14915213`/`14915217`/`14915251`: completed the exact compatibility + L2 0.02
|
|
|
|
| 428 |
|
| 429 |
- Promote same-state no-expert lattice (56.99%) as the conservative mechanism
|
| 430 |
result.
|
| 431 |
+
- Use K4 compatible residual transport with advantage abstention, a small typed
|
| 432 |
+
no-op prior, field-gated tangent length calibration over `0.35/0.40/0.45`, an
|
| 433 |
+
exact near-miss/no-op compatibility mask, and the trace-motivated singleton
|
| 434 |
+
near-miss challenger gate as the current best clean deployment diagnostic,
|
| 435 |
+
36.00%, not as a SOTA claim. The previous mean-by-type scale-grid no-op row
|
| 436 |
+
remains 35.42%; pure masked composition is 35.30%, the exact typed-prior
|
| 437 |
+
composition row is 35.54%, exact compatibility reaches 35.59%, and
|
| 438 |
component-wise propagation of the no-op prior reaches 35.36%, so composition
|
| 439 |
helps only when sparse exact priors are preserved and incompatible/anti-goal
|
| 440 |
composite parts are masked. Composite-only L2 trust penalties lower action
|
scripts/build_paper_analysis.py
CHANGED
|
@@ -401,6 +401,30 @@ METHODS = [
|
|
| 401 |
"k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p02_summary.json"
|
| 402 |
),
|
| 403 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 404 |
MethodSpec(
|
| 405 |
key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
|
| 406 |
label="K4 composed compatible tangents, no-op bonus 0.03, singleton near-miss bonus 0.01",
|
|
|
|
| 401 |
"k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p02_summary.json"
|
| 402 |
),
|
| 403 |
),
|
| 404 |
+
MethodSpec(
|
| 405 |
+
key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger001",
|
| 406 |
+
label="K4 compatible tangents, no-op bonus 0.03, near-miss challenger gate 0.01",
|
| 407 |
+
summary_path=(
|
| 408 |
+
"h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_"
|
| 409 |
+
"k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p01_summary.json"
|
| 410 |
+
),
|
| 411 |
+
),
|
| 412 |
+
MethodSpec(
|
| 413 |
+
key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger003",
|
| 414 |
+
label="K4 compatible tangents, no-op bonus 0.03, near-miss challenger gate 0.03",
|
| 415 |
+
summary_path=(
|
| 416 |
+
"h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_"
|
| 417 |
+
"k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p03_summary.json"
|
| 418 |
+
),
|
| 419 |
+
),
|
| 420 |
+
MethodSpec(
|
| 421 |
+
key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmwgchallenger001",
|
| 422 |
+
label="K4 compatible tangents, no-op bonus 0.03, near-miss/wrong-gripper challenger gate 0.01",
|
| 423 |
+
summary_path=(
|
| 424 |
+
"h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_"
|
| 425 |
+
"k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmwgchallenger0p01_summary.json"
|
| 426 |
+
),
|
| 427 |
+
),
|
| 428 |
MethodSpec(
|
| 429 |
key="residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
|
| 430 |
label="K4 composed compatible tangents, no-op bonus 0.03, singleton near-miss bonus 0.01",
|
scripts/build_paper_table_status.py
CHANGED
|
@@ -755,6 +755,36 @@ SPECS = [
|
|
| 755 |
story_role="trace-motivated two-stage selector calibration over the compatible local tangent chart",
|
| 756 |
pending_job="14954280/14954281",
|
| 757 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 758 |
ResultSpec(
|
| 759 |
key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
|
| 760 |
label="K4 composed compatible residual retrieval, no-op bonus 0.03, singleton near-miss bonus 0.01",
|
|
|
|
| 755 |
story_role="trace-motivated two-stage selector calibration over the compatible local tangent chart",
|
| 756 |
pending_job="14954280/14954281",
|
| 757 |
),
|
| 758 |
+
ResultSpec(
|
| 759 |
+
key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger001",
|
| 760 |
+
label="K4 compatible residual retrieval, near-miss challenger gate 0.01",
|
| 761 |
+
path="h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p01_summary.json",
|
| 762 |
+
clean_deployment="yes",
|
| 763 |
+
same_state_proposals="no",
|
| 764 |
+
expert_proposal="no",
|
| 765 |
+
story_role="lower-margin sensitivity for trace-motivated singleton near-miss challenger calibration",
|
| 766 |
+
pending_job="14954526/14954527",
|
| 767 |
+
),
|
| 768 |
+
ResultSpec(
|
| 769 |
+
key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmchallenger003",
|
| 770 |
+
label="K4 compatible residual retrieval, near-miss challenger gate 0.03",
|
| 771 |
+
path="h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmchallenger0p03_summary.json",
|
| 772 |
+
clean_deployment="yes",
|
| 773 |
+
same_state_proposals="no",
|
| 774 |
+
expert_proposal="no",
|
| 775 |
+
story_role="upper-margin sensitivity for trace-motivated singleton near-miss challenger calibration",
|
| 776 |
+
pending_job="14954528/14954529",
|
| 777 |
+
),
|
| 778 |
+
ResultSpec(
|
| 779 |
+
key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmwgchallenger001",
|
| 780 |
+
label="K4 compatible residual retrieval, near-miss/wrong-gripper challenger gate 0.01",
|
| 781 |
+
path="h16_policy_ckpt_near_miss_policy_bc5_bestpt_retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_safe_margin0p20_noopbonus0p03_nmwgchallenger0p01_summary.json",
|
| 782 |
+
clean_deployment="yes",
|
| 783 |
+
same_state_proposals="no",
|
| 784 |
+
expert_proposal="no",
|
| 785 |
+
story_role="trace-motivated challenger calibration test for whether wrong-gripper residuals carry conditional selector headroom",
|
| 786 |
+
pending_job="14954530/14954531",
|
| 787 |
+
),
|
| 788 |
ResultSpec(
|
| 789 |
key="retrieval_residual_k4_composemasked_dropnmnoop_grid035040045_noopbonus003_nmbonus001",
|
| 790 |
label="K4 composed compatible residual retrieval, no-op bonus 0.03, singleton near-miss bonus 0.01",
|