Thibaut Forest Claude Opus 5 (1M context) commited on
Commit ·
08031d9
1
Parent(s): 4fb1dc8
Phase 12: spatial peer features rejected (panel +106, 0% selection); pm40 sweep closed
Browse files- EXPERIMENTS.md +39 -0
EXPERIMENTS.md
CHANGED
|
@@ -397,6 +397,45 @@ runtime splits live. Expected effect is largest on public/private scenarios
|
|
| 397 |
near their data end; the panel contains only ~3 such origins, so the local
|
| 398 |
delta (-14) understates the deployable one.
|
| 399 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 400 |
## Test-look ledger
|
| 401 |
|
| 402 |
8 looks total (now historical - see protocol change above); look 9 = the
|
|
|
|
| 397 |
near their data end; the panel contains only ~3 such origins, so the local
|
| 398 |
delta (-14) understates the deployable one.
|
| 399 |
|
| 400 |
+
Follow-up sweeps, both rejected by the panel: pm40 (1705.1, s_38's -721 does
|
| 401 |
+
not survive the fresh-only restriction) and pm40/m1.0 (1715.1). pm25/m0.8 is
|
| 402 |
+
the local optimum of the family. Docker all-48 verification of the adopted
|
| 403 |
+
champion: **2175.4** (was 2202); late 1460 / early 451.
|
| 404 |
+
|
| 405 |
+
## Phase 12 - spatial peer features (rejected: panel +106, 0% selection)
|
| 406 |
+
|
| 407 |
+
User idea: cross-room/building features, age-scaled ("what would my peers
|
| 408 |
+
have measured at my age?"), to catch fast plungers relatively. Attractive on
|
| 409 |
+
paper because it fixes phase-6's fatal flaw: batch death-clocks needed peers'
|
| 410 |
+
DEATHS (14-54% coverage), while peer VOLTAGE at matched age has ~100%
|
| 411 |
+
coverage. Recon: rooms too thin (median 4 peers); buildings workable (median
|
| 412 |
+
9, max 117); install-week batches age-match by construction (median 8).
|
| 413 |
+
Building death rates genuinely cluster (4% to 100% across 24 buildings).
|
| 414 |
+
|
| 415 |
+
Screen on the blind zone (11,599 healthy-voltage pairs, 24 silent deaths):
|
| 416 |
+
|
| 417 |
+
| feature | blind AUC | all-pairs AUC |
|
| 418 |
+
|---|---|---|
|
| 419 |
+
| deficit vs building peers (age-regressed) | 0.761 | 0.869 |
|
| 420 |
+
| deficit vs install-batch peers | 0.663 | 0.808 |
|
| 421 |
+
| building death-rate prior (smoothed, leak-free) | 0.646 | 0.721 |
|
| 422 |
+
| rng_30 (phase-8 incumbent) | 0.832 | 0.764 |
|
| 423 |
+
| v_corr_last (already in FEATURES) | 0.867 | 0.945 |
|
| 424 |
+
|
| 425 |
+
Additive test: blind logistic rng_30 alone 0.843; +peer features 0.825-0.837
|
| 426 |
+
(worse). Panel adjudication anyway (logbag + deficit_bldg + bldg_rate on the
|
| 427 |
+
phase-11 planner, 34 origins): **1807.6 vs 1702.0**, champion selected in
|
| 428 |
+
100% of random and blocked splits. Per-origin AUC is unchanged (s_41: 0.967
|
| 429 |
+
vs 0.968) - the features do not add ranking signal, only calibration noise,
|
| 430 |
+
which the count rule converts into worse inclusion counts.
|
| 431 |
+
|
| 432 |
+
Why it cannot work here: the master curve already performs fleet-level
|
| 433 |
+
age-matching (tau_projected IS "voltage vs expected-at-this-point"); local
|
| 434 |
+
peer medians re-estimate the same quantity from 5-20 samples instead of the
|
| 435 |
+
fleet. The building death-rate signal is real but weak exactly where the
|
| 436 |
+
misses live, and largely confounded with install batch (~90% of lifetime
|
| 437 |
+
variance, phase 6).
|
| 438 |
+
|
| 439 |
## Test-look ledger
|
| 440 |
|
| 441 |
8 looks total (now historical - see protocol change above); look 9 = the
|