Thibaut Forest Claude Opus 5 (1M context) commited on
Commit ·
2de326a
1
Parent(s): e3fa964
Phase 16: dataset semantics verified (2451-device fleet, end_time = admin snapshot; no model change)
Browse files- EXPERIMENTS.md +24 -0
EXPERIMENTS.md
CHANGED
|
@@ -512,6 +512,30 @@ for that objective. detect_swaps() and the GHOST_ZERO/QUARANTINE knobs remain
|
|
| 512 |
in the code, inert by default, with the production table and runtime paths
|
| 513 |
unchanged. Champion unchanged: panel 1679.2, all-48 Docker 2146.9.
|
| 514 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 515 |
## Test-look ledger
|
| 516 |
|
| 517 |
8 looks total (now historical - see protocol change above); look 9 = the
|
|
|
|
| 512 |
in the code, inert by default, with the production table and runtime paths
|
| 513 |
unchanged. Champion unchanged: panel 1679.2, all-48 Docker 2146.9.
|
| 514 |
|
| 515 |
+
## Phase 16 - dataset semantics (two community observations verified; no model change needed)
|
| 516 |
+
|
| 517 |
+
**Categorical metadata leak**: the train parquets' device_id dtype carries
|
| 518 |
+
**2,451 categories** while train uses 461 - the held-out fleets total ~1,990
|
| 519 |
+
devices (~4.3x train). Implications: (a) per-split adaptive refits are the
|
| 520 |
+
right architecture (each split has its own large fleet); (b) worker limits
|
| 521 |
+
(fixed 100/day) bind much harder at 4x fleet size - the public leader's
|
| 522 |
+
daily_limit penalties (108/scenario) are rational there; our optimize_days
|
| 523 |
+
prices limits exactly, but local CV never exercises limit-saturated regimes.
|
| 524 |
+
|
| 525 |
+
**end_time semantics**: locations.end_time is the ADMIN snapshot end, not
|
| 526 |
+
per-device last data - 445/461 devices share 2026-08-01. All 82 dead slots
|
| 527 |
+
keep reporting past EOL (mean 107d, max 305d): replacement batteries under
|
| 528 |
+
the same device_id, invisible to scoring. Verified harmless everywhere:
|
| 529 |
+
master_curve filters tau >= 0 (replacement data never enters), dead devices
|
| 530 |
+
are excluded from planning, and the 15 admin-removed unscoreable devices
|
| 531 |
+
(early end_time, no EOL; e.g. 11 removed 2025-12-26) are scored p42 ~ 0.01
|
| 532 |
+
by the model - the "stale + frozen voltage -> survives" pattern learned from
|
| 533 |
+
exactly these rows (phase 15's load-bearing negatives) keeps them out of
|
| 534 |
+
every plan. The e_far imputation pricing in cost_curves makes them
|
| 535 |
+
prohibitively expensive to the break-even top-up as well (two-sided fix,
|
| 536 |
+
phase 11). Example decoded: d_57e99ff00802 - EOL 2026-02-02 (scored death),
|
| 537 |
+
replacement battery reports until 2026-05-13, admin end_time 2026-08-01.
|
| 538 |
+
|
| 539 |
## Test-look ledger
|
| 540 |
|
| 541 |
8 looks total (now historical - see protocol change above); look 9 = the
|