Thibaut Forest Claude Opus 5 (1M context) commited on
Commit
08031d9
·
1 Parent(s): 4fb1dc8

Phase 12: spatial peer features rejected (panel +106, 0% selection); pm40 sweep closed

Browse files
Files changed (1) hide show
  1. EXPERIMENTS.md +39 -0
EXPERIMENTS.md CHANGED
@@ -397,6 +397,45 @@ runtime splits live. Expected effect is largest on public/private scenarios
397
  near their data end; the panel contains only ~3 such origins, so the local
398
  delta (-14) understates the deployable one.
399
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
400
  ## Test-look ledger
401
 
402
  8 looks total (now historical - see protocol change above); look 9 = the
 
397
  near their data end; the panel contains only ~3 such origins, so the local
398
  delta (-14) understates the deployable one.
399
 
400
+ Follow-up sweeps, both rejected by the panel: pm40 (1705.1, s_38's -721 does
401
+ not survive the fresh-only restriction) and pm40/m1.0 (1715.1). pm25/m0.8 is
402
+ the local optimum of the family. Docker all-48 verification of the adopted
403
+ champion: **2175.4** (was 2202); late 1460 / early 451.
404
+
405
+ ## Phase 12 - spatial peer features (rejected: panel +106, 0% selection)
406
+
407
+ User idea: cross-room/building features, age-scaled ("what would my peers
408
+ have measured at my age?"), to catch fast plungers relatively. Attractive on
409
+ paper because it fixes phase-6's fatal flaw: batch death-clocks needed peers'
410
+ DEATHS (14-54% coverage), while peer VOLTAGE at matched age has ~100%
411
+ coverage. Recon: rooms too thin (median 4 peers); buildings workable (median
412
+ 9, max 117); install-week batches age-match by construction (median 8).
413
+ Building death rates genuinely cluster (4% to 100% across 24 buildings).
414
+
415
+ Screen on the blind zone (11,599 healthy-voltage pairs, 24 silent deaths):
416
+
417
+ | feature | blind AUC | all-pairs AUC |
418
+ |---|---|---|
419
+ | deficit vs building peers (age-regressed) | 0.761 | 0.869 |
420
+ | deficit vs install-batch peers | 0.663 | 0.808 |
421
+ | building death-rate prior (smoothed, leak-free) | 0.646 | 0.721 |
422
+ | rng_30 (phase-8 incumbent) | 0.832 | 0.764 |
423
+ | v_corr_last (already in FEATURES) | 0.867 | 0.945 |
424
+
425
+ Additive test: blind logistic rng_30 alone 0.843; +peer features 0.825-0.837
426
+ (worse). Panel adjudication anyway (logbag + deficit_bldg + bldg_rate on the
427
+ phase-11 planner, 34 origins): **1807.6 vs 1702.0**, champion selected in
428
+ 100% of random and blocked splits. Per-origin AUC is unchanged (s_41: 0.967
429
+ vs 0.968) - the features do not add ranking signal, only calibration noise,
430
+ which the count rule converts into worse inclusion counts.
431
+
432
+ Why it cannot work here: the master curve already performs fleet-level
433
+ age-matching (tau_projected IS "voltage vs expected-at-this-point"); local
434
+ peer medians re-estimate the same quantity from 5-20 samples instead of the
435
+ fleet. The building death-rate signal is real but weak exactly where the
436
+ misses live, and largely confounded with install batch (~90% of lifetime
437
+ variance, phase 6).
438
+
439
  ## Test-look ledger
440
 
441
  8 looks total (now historical - see protocol change above); look 9 = the