Thibaut Forest Claude Opus 5 (1M context) commited on
Commit
2de326a
·
1 Parent(s): e3fa964

Phase 16: dataset semantics verified (2451-device fleet, end_time = admin snapshot; no model change)

Browse files
Files changed (1) hide show
  1. EXPERIMENTS.md +24 -0
EXPERIMENTS.md CHANGED
@@ -512,6 +512,30 @@ for that objective. detect_swaps() and the GHOST_ZERO/QUARANTINE knobs remain
512
  in the code, inert by default, with the production table and runtime paths
513
  unchanged. Champion unchanged: panel 1679.2, all-48 Docker 2146.9.
514
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
515
  ## Test-look ledger
516
 
517
  8 looks total (now historical - see protocol change above); look 9 = the
 
512
  in the code, inert by default, with the production table and runtime paths
513
  unchanged. Champion unchanged: panel 1679.2, all-48 Docker 2146.9.
514
 
515
+ ## Phase 16 - dataset semantics (two community observations verified; no model change needed)
516
+
517
+ **Categorical metadata leak**: the train parquets' device_id dtype carries
518
+ **2,451 categories** while train uses 461 - the held-out fleets total ~1,990
519
+ devices (~4.3x train). Implications: (a) per-split adaptive refits are the
520
+ right architecture (each split has its own large fleet); (b) worker limits
521
+ (fixed 100/day) bind much harder at 4x fleet size - the public leader's
522
+ daily_limit penalties (108/scenario) are rational there; our optimize_days
523
+ prices limits exactly, but local CV never exercises limit-saturated regimes.
524
+
525
+ **end_time semantics**: locations.end_time is the ADMIN snapshot end, not
526
+ per-device last data - 445/461 devices share 2026-08-01. All 82 dead slots
527
+ keep reporting past EOL (mean 107d, max 305d): replacement batteries under
528
+ the same device_id, invisible to scoring. Verified harmless everywhere:
529
+ master_curve filters tau >= 0 (replacement data never enters), dead devices
530
+ are excluded from planning, and the 15 admin-removed unscoreable devices
531
+ (early end_time, no EOL; e.g. 11 removed 2025-12-26) are scored p42 ~ 0.01
532
+ by the model - the "stale + frozen voltage -> survives" pattern learned from
533
+ exactly these rows (phase 15's load-bearing negatives) keeps them out of
534
+ every plan. The e_far imputation pricing in cost_curves makes them
535
+ prohibitively expensive to the break-even top-up as well (two-sided fix,
536
+ phase 11). Example decoded: d_57e99ff00802 - EOL 2026-02-02 (scored death),
537
+ replacement battery reports until 2026-05-13, admin end_time 2026-08-01.
538
+
539
  ## Test-look ledger
540
 
541
  8 looks total (now historical - see protocol change above); look 9 = the