Thibaut Forest Claude Opus 5 (1M context) commited on
Commit ·
b6d4319
1
Parent(s): 2a6a86a
Phase 58: definitive v5-vs-public coherence - zero false adoptions under the full protocol; faithful config reproduction matters (alpha stack reads n.s., approximations overstated misalignment)
Browse files- EXPERIMENTS.md +32 -0
EXPERIMENTS.md
CHANGED
|
@@ -3149,3 +3149,35 @@ fleet-composition facts (uncatchable public tail, train-absent ghosts) are
|
|
| 3149 |
invisible to ANY local split. The structural shipping bar (flat swaps per
|
| 3150 |
cell, no label-process exploits) remains the only local defense. One
|
| 3151 |
coherent cross-fleet signal: vrel -13.4+/-7.9, 104W/61L, swaps flat.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3149 |
invisible to ANY local split. The structural shipping bar (flat swaps per
|
| 3150 |
cell, no label-process exploits) remains the only local defense. One
|
| 3151 |
coherent cross-fleet signal: vrel -13.4+/-7.9, 104W/61L, swaps flat.
|
| 3152 |
+
|
| 3153 |
+
## Phase 58 - definitive local-vs-public coherence check on the v5 harness (deployment-faithful GroupKFold)
|
| 3154 |
+
|
| 3155 |
+
All five publicly-measured configs reproduced AS SHIPPED (the alpha point as
|
| 3156 |
+
the true ALPHA_STATIC=1.5 + CLIFF_SCALE=2.2 stack driving the count through
|
| 3157 |
+
p42_sum; cert stacked on rank_score; pooling with time-masked own pairs;
|
| 3158 |
+
deployment-exact planner state incl. the gated floor; v5 baseline = 352.0,
|
| 3159 |
+
byte-identical to v3, floor confirmed inert):
|
| 3160 |
+
|
| 3161 |
+
| config | local delta +/- SE | swaps | public delta | decision at the 2-sigma bar |
|
| 3162 |
+
|---|---|---|---|---|
|
| 3163 |
+
| alpha15+cliff22 | -15.4 +/- 16.8 | 4.18 | +527.3 | NOT adopted (0.9 sigma) - correct |
|
| 3164 |
+
| cert-first | -7.3 +/- 4.3 | 1.90 | +43.2 | NOT adopted (1.7 sigma) - correct |
|
| 3165 |
+
| pooling | +10.2 +/- 3.6 | 1.92 | +4.4 | rejected - SIGN AGREES |
|
| 3166 |
+
| frontier | -5.3 +/- 7.4 | 1.79 | +215.6 | NOT adopted (0.7 sigma) - correct |
|
| 3167 |
+
| (alpha15 solo, no public point) | -28.8 +/- 13.6 | 3.15 | - | 2.1 sigma BUT count-coupled (swaps +1.25) -> quarantine rule catches it |
|
| 3168 |
+
|
| 3169 |
+
VERDICT: the v5 CV cannot reproduce public MAGNITUDES (ghosts and the
|
| 3170 |
+
uncatchable tail are absent from every local fleet - unfixable), but as a
|
| 3171 |
+
DECISION INSTRUMENT the full protocol - v5 harness + 2-sigma adoption bar +
|
| 3172 |
+
count-flat rule + no-label-process rule - makes the correct call on all five
|
| 3173 |
+
public measurements: zero false adoptions, one correct rejection with sign
|
| 3174 |
+
agreement. Notably the faithful reproduction itself mattered: alpha-solo
|
| 3175 |
+
reads a confident-looking -28.8 (2.1 sigma) while the actually-shipped
|
| 3176 |
+
alpha+cliff stack reads a statistical zero - earlier alignment checks that
|
| 3177 |
+
approximated configs overstated the misalignment. Residual sharpest risk:
|
| 3178 |
+
cert-class (1.7 sigma local win, public +43) - the label-process quarantine
|
| 3179 |
+
is what protects against it, not the harness.
|
| 3180 |
+
|
| 3181 |
+
THE CV going forward: v5 building-GroupKFold (deployment-faithful held-out
|
| 3182 |
+
simulation, one instrument for public AND private - both are ~2x held-out
|
| 3183 |
+
fleets), used for ordering/model questions under the quarantine rules.
|