Thibaut Forest Claude Opus 5 (1M context) commited on
Commit
b6d4319
·
1 Parent(s): 2a6a86a

Phase 58: definitive v5-vs-public coherence - zero false adoptions under the full protocol; faithful config reproduction matters (alpha stack reads n.s., approximations overstated misalignment)

Browse files
Files changed (1) hide show
  1. EXPERIMENTS.md +32 -0
EXPERIMENTS.md CHANGED
@@ -3149,3 +3149,35 @@ fleet-composition facts (uncatchable public tail, train-absent ghosts) are
3149
  invisible to ANY local split. The structural shipping bar (flat swaps per
3150
  cell, no label-process exploits) remains the only local defense. One
3151
  coherent cross-fleet signal: vrel -13.4+/-7.9, 104W/61L, swaps flat.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3149
  invisible to ANY local split. The structural shipping bar (flat swaps per
3150
  cell, no label-process exploits) remains the only local defense. One
3151
  coherent cross-fleet signal: vrel -13.4+/-7.9, 104W/61L, swaps flat.
3152
+
3153
+ ## Phase 58 - definitive local-vs-public coherence check on the v5 harness (deployment-faithful GroupKFold)
3154
+
3155
+ All five publicly-measured configs reproduced AS SHIPPED (the alpha point as
3156
+ the true ALPHA_STATIC=1.5 + CLIFF_SCALE=2.2 stack driving the count through
3157
+ p42_sum; cert stacked on rank_score; pooling with time-masked own pairs;
3158
+ deployment-exact planner state incl. the gated floor; v5 baseline = 352.0,
3159
+ byte-identical to v3, floor confirmed inert):
3160
+
3161
+ | config | local delta +/- SE | swaps | public delta | decision at the 2-sigma bar |
3162
+ |---|---|---|---|---|
3163
+ | alpha15+cliff22 | -15.4 +/- 16.8 | 4.18 | +527.3 | NOT adopted (0.9 sigma) - correct |
3164
+ | cert-first | -7.3 +/- 4.3 | 1.90 | +43.2 | NOT adopted (1.7 sigma) - correct |
3165
+ | pooling | +10.2 +/- 3.6 | 1.92 | +4.4 | rejected - SIGN AGREES |
3166
+ | frontier | -5.3 +/- 7.4 | 1.79 | +215.6 | NOT adopted (0.7 sigma) - correct |
3167
+ | (alpha15 solo, no public point) | -28.8 +/- 13.6 | 3.15 | - | 2.1 sigma BUT count-coupled (swaps +1.25) -> quarantine rule catches it |
3168
+
3169
+ VERDICT: the v5 CV cannot reproduce public MAGNITUDES (ghosts and the
3170
+ uncatchable tail are absent from every local fleet - unfixable), but as a
3171
+ DECISION INSTRUMENT the full protocol - v5 harness + 2-sigma adoption bar +
3172
+ count-flat rule + no-label-process rule - makes the correct call on all five
3173
+ public measurements: zero false adoptions, one correct rejection with sign
3174
+ agreement. Notably the faithful reproduction itself mattered: alpha-solo
3175
+ reads a confident-looking -28.8 (2.1 sigma) while the actually-shipped
3176
+ alpha+cliff stack reads a statistical zero - earlier alignment checks that
3177
+ approximated configs overstated the misalignment. Residual sharpest risk:
3178
+ cert-class (1.7 sigma local win, public +43) - the label-process quarantine
3179
+ is what protects against it, not the harness.
3180
+
3181
+ THE CV going forward: v5 building-GroupKFold (deployment-faithful held-out
3182
+ simulation, one instrument for public AND private - both are ~2x held-out
3183
+ fleets), used for ordering/model questions under the quarantine rules.