Thibaut Forest Claude Fable 5 commited on
Commit
4f5f060
·
1 Parent(s): 2000281

cv: temporal purge is the ONLY protocol (user spec) - all48oof fits per scenario on ALL buildings' pairs whose 42d label windows lie fully before/after the scenario window (no building holdout, full-fleet plans); all48/static/screen/lobo/ablate and the lobo/wf/self variants REMOVED (git history keeps them; exp_x29/x30 need the old cv.py); SCORE 1877.3 -> 1972.0 (harsher: warmup 3560.3 vs 2875.3 loses the calendar-local window; panel 1839.6 vs 1857.7)

Browse files
Files changed (1) hide show
  1. batteryswap_example/cv.py +114 -731
batteryswap_example/cv.py CHANGED
@@ -1,52 +1,50 @@
1
- """Repo-resident CV protocols (ported from the blind research loop, 2026-08-21).
2
-
3
- This replaces the ad-hoc `/tmp/all48*.py` scripts that every prior phase had to
4
- re-derive, and folds in the three measurement disciplines the blind loop paid
5
- real leaderboard points to learn.
6
-
7
- **1. Report ranking precision at the operating N, not AUC.**
8
- A model can hold AUC 0.947 and recall 0.000 on the subgroup that drives the
9
- late cost (the voltage-silent deaths: median rank 60 of ~450). Mean cost
10
- hides whether a change reordered the list or just cut it deeper. Every
11
- protocol here prints rec12/rec20, the operating-point precision, and the
12
- median rank of missed deaths beside the cost. Implemented in
13
- `models_cv.rank_stats`, so `dev`/`panel`/`test` get it too.
14
-
15
- **2. Group by BUILDING > battery > scenario.**
16
- Each battery contributes one near-duplicate row per closed origin (~3600
17
- rows from ~460 batteries), so holding out a SCENARIO leaves the same
18
- batteries in train and test and tree models score their own memorised
19
- signatures. Measured in the blind loop: leave-scenario-out put HGB ahead of
20
- linear (AUC .9907 vs .9843); leave-battery-out REVERSED it (.9769 vs .9845)
21
- and the cost harness agreed with the clean split. The effective sample size
22
- is ~460, not ~3600. And battery grouping is still not the deployment split:
23
- the scored public/private fleets hold FOREIGN BUILDINGS (organizer design;
24
- also how `static` splits), and siblings in one building share install batch
25
- and thermal environment, so a battery fold keeps a held-out device's
26
- building-mates in train. `screen` runs all three groupings side by side;
27
- selection follows the BUILDING column (2026-08-22 protocol change).
28
-
29
- **3. The deployment gate: would this fire in the static branch?**
30
- The scored splits are fleets the model has never seen; on them `submit.py`
31
- fits ONE model on the foreign train pool and never refits - no per-origin
32
- refit, no accumulating own history, no cold start (the pool always holds 82
33
- deaths, so every `train_deaths < 20` gate is permanently shut). The walk-
34
- forward harness does the opposite. A mechanism can be worth tens of points
35
- at `all48` and exactly zero where it is scored; that has already happened
36
- once (a validated -49.8 warm-up mechanism, structurally unreachable in
37
- deployment). `static` runs the deployment protocol locally on a foreign
38
- half-fleet so the gate is measurable instead of argued.
39
-
40
- Validity boundary, learned twice the hard way: the half-fleet construction
41
- manufactures a sum(p42) undercount that does NOT exist on the real held-out
42
- fleet (phase 20 refuted a foreign floor with it; phase 26 adopted
43
- ALPHA_STATIC=1.5 from it and public went 1661 -> 2189). Use `static` for
44
- ORDERING questions only. Never read an appetite/count decision off it.
45
 
46
  Usage
47
- venv/bin/python -m batteryswap_example.cv all48 [variant]
48
- venv/bin/python -m batteryswap_example.cv static [variant]
49
- venv/bin/python -m batteryswap_example.cv screen
50
  """
51
  import sys
52
  import time
@@ -79,7 +77,7 @@ BLOCKS = (('warmup s_0-7', lambda o: o <= 7),
79
 
80
 
81
  def load_table(dataset_path=Path('dataset')):
82
- """The one table build. Every protocol starts here so a variant can never
83
  silently differ from the champion by a feature-prep line."""
84
  t = mc.load_features(dataset_path)
85
  t['v_proj'] = t['v_last'] + t['slope_lastk'] * t['staleness']
@@ -129,222 +127,7 @@ def scorecard(df, label):
129
 
130
 
131
  # --------------------------------------------------------------------------
132
- # protocol 1: full-fleet walk-forward, s_0..s_47
133
- # --------------------------------------------------------------------------
134
-
135
- def run_all48(variant='base', origins=range(48), dataset_path=Path('dataset')):
136
- t = load_table(dataset_path)
137
- mc.FEATURES = list(VARIANTS[variant](t))
138
- ctx = mc.load_context(dataset_path)
139
- df = mc.walk_forward(t, ctx, origins, [dict(CHAMPION)])
140
- df['variant'] = variant
141
- return df
142
-
143
-
144
- # --------------------------------------------------------------------------
145
- # protocol 2: the deployment gate - one static fit on a foreign fleet
146
- # --------------------------------------------------------------------------
147
-
148
- def fleet_halves(locations, seed=0):
149
- """Split the 24 buildings into two disjoint fleets.
150
-
151
- Split by BUILDING, not by device: rooms and buildings share install
152
- batches and thermal environment, so a device-level split leaves the
153
- target fleet's own siblings in the training pool and the fit is no longer
154
- foreign in the way deployment's is.
155
- """
156
- b = np.sort(locations['building'].unique())
157
- rng = np.random.RandomState(seed)
158
- perm = rng.permutation(len(b))
159
- a = set(b[perm[:len(b) // 2]])
160
- return a, set(b[perm[len(b) // 2:]])
161
-
162
-
163
- def run_static(variant='base', seed=0, origins=range(48),
164
- dataset_path=Path('dataset')):
165
- """Deployment protocol: fit once on fleet A's fully-closed pairs, then plan
166
- every one of fleet B's origins with that single model. No refits, no
167
- accumulating own history, no cold-start gate - exactly submit.py's
168
- held-out branch."""
169
- t = load_table(dataset_path)
170
- mc.FEATURES = list(VARIANTS[variant](t))
171
- locations, timeseries, eol_times, scenarios = mc.load_dataset(
172
- dataset_path / 'train')
173
- fleet_a, fleet_b = fleet_halves(locations, seed)
174
- dev_bldg = locations.drop_duplicates('battery').set_index('battery')['building']
175
- t['bldg'] = t['device_id'].map(dev_bldg)
176
-
177
- # the foreign pool: every fleet-A pair whose 42d window has closed by the
178
- # end of the data. Deployment's pool is likewise fully adjudicated at
179
- # build time and independent of the target origin.
180
- closed = t['origin'] <= t['origin'].max() - pd.Timedelta(days=WINDOW_DAYS)
181
- foreign = t[closed & t['bldg'].isin(fleet_a)]
182
- model = mc.SklearnRUL(CHAMPION['model']).fit(foreign)
183
- uniq = int(foreign.loc[foreign['dies42'] == 1, 'device_id'].nunique())
184
- print(f'static foreign fit: {len(foreign)} pairs, {uniq} unique deaths, '
185
- f'{len(fleet_a)} buildings | target fleet {len(fleet_b)} buildings',
186
- file=sys.stderr)
187
-
188
- p = mc.plan_tsroute
189
- p.alpha, p.p_be, p.floor_day, p.p_conf = CHAMPION['p_star'], 0.0, 0, 0.5
190
- p.ec_margin, p.ec_pmax = CHAMPION['ec_margin'], CHAMPION['ec_pmax']
191
- p.split_days = CHAMPION['split_days']
192
- # the cold-start floor is permanently shut in deployment (82 foreign
193
- # deaths >= gate 20). Leaving it live here would measure a mechanism the
194
- # scored path can never reach.
195
- p.floor_scale, p.recent_deaths = 0.0, 0.0
196
-
197
- recs = []
198
- for idx, (scenario, locs, cut, eol) in enumerate(mc.iterate_scenarios(
199
- locations, timeseries, eol_times, scenarios)):
200
- if idx not in origins:
201
- continue
202
- T = pd.Timestamp(scenario['start_time'])
203
- locs_b = locs[locs['building'].isin(fleet_b)]
204
- rows = (t[t['origin_idx'] == idx].set_index('device_id')
205
- .reindex(locs_b['battery']).dropna(subset=['v_last']))
206
- if rows.empty:
207
- continue
208
- locs_b = locs_b[locs_b['battery'].isin(rows.index)]
209
- # the evaluator prices unobserved EOLs off `locations`, so the eol
210
- # series has to be cut to the target fleet too
211
- eol_b = eol.reindex(locs_b['battery'])
212
- pred = model.predict(rows)
213
- plan = mc.plan_tsroute(pred, None, locs_b, T, CHAMPION['day_q'],
214
- travel_costs=scenario['travel_costs'],
215
- settings=scenario['settings'])
216
- _, _, overall = evaluate_plan(plan, locs_b, scenario['travel_costs'],
217
- scenario['settings'], eol_times=eol_b,
218
- start_time=T, verbose=0)
219
- rs = mc.rank_stats(pred, plan, eol_b, T)
220
- recs.append(dict(overall, **rs, origin=idx, variant=variant, seed=seed))
221
- print(f"s_{idx}: total {overall['total_cost']:7.1f} "
222
- f"late {overall['late_swap']:6.1f} deaths {rs.get('n_deaths', 0):2d} "
223
- f"caught {rs.get('caught', 0):2d}/{rs.get('n_sched', 0):2d} "
224
- f"rec12 {rs.get('rec12', float('nan')):.2f}", flush=True,
225
- file=sys.stderr)
226
- return pd.DataFrame(recs)
227
-
228
-
229
- # --------------------------------------------------------------------------
230
- # protocol 3: offline model screen, grouped by building > battery
231
- # --------------------------------------------------------------------------
232
-
233
- def building_folds(closed, dev_bldg, n_folds=5, seed=0):
234
- """Assign whole BUILDINGS to folds, balancing observed deaths.
235
-
236
- Deaths are lumpy across the 24 buildings (top building holds 15/82, nine
237
- hold exactly one), so a random building assignment routinely starves a
238
- fold of positives. Greedy balance instead: buildings in descending
239
- unique-death order, each to the death-lightest fold so far (ties broken
240
- by device load, then by the seed's shuffle).
241
- """
242
- per_dev = (closed[['device_id', 'dies42']]
243
- .assign(bldg=closed['device_id'].map(dev_bldg).values)
244
- .groupby('device_id').agg(bldg=('bldg', 'first'),
245
- died=('dies42', 'max')))
246
- per_b = per_dev.groupby('bldg').agg(devices=('died', 'size'),
247
- deaths=('died', 'sum'))
248
- order = (per_b.sample(frac=1, random_state=np.random.RandomState(seed))
249
- .sort_values(['deaths', 'devices'], ascending=False, kind='stable'))
250
- deaths_in, devs_in = np.zeros(n_folds), np.zeros(n_folds)
251
- fold_of = {}
252
- for b, row in order.iterrows():
253
- f = int(np.lexsort((devs_in, deaths_in))[0])
254
- fold_of[b] = f
255
- deaths_in[f] += row['deaths']
256
- devs_in[f] += row['devices']
257
- return pd.Series(fold_of, name='fold')
258
-
259
-
260
- def screen(kinds=('logistic', 'logbag', 'rf', 'gbm'), n_folds=5, seed=0,
261
- dataset_path=Path('dataset')):
262
- """Leave-BUILDING-out vs leave-BATTERY-out vs leave-SCENARIO-out on the
263
- same pairs and the same models. The gaps between columns are contamination;
264
- trust the BUILDING column - the scored public/private fleets hold foreign
265
- buildings, and building-mates share install batch and thermal environment,
266
- so battery folds still leak sibling identity. Headline is recall at the
267
- operating N, AUC secondary."""
268
- from sklearn.metrics import roc_auc_score
269
-
270
- t = load_table(dataset_path)
271
- mc.FEATURES = list(BASE_FEATURES)
272
- closed = t[t['origin'] <= t['origin'].max() - pd.Timedelta(days=WINDOW_DAYS)]
273
- devs = np.sort(closed['device_id'].unique())
274
- rng = np.random.RandomState(seed)
275
- dev_fold = pd.Series(rng.permutation(len(devs)) % n_folds, index=devs)
276
- org = np.sort(closed['origin_idx'].unique())
277
- org_fold = pd.Series(np.arange(len(org)) % n_folds, index=org)
278
- dev_bldg = (pd.read_csv(dataset_path / 'train' / 'devices.csv')
279
- .drop_duplicates('device_id')
280
- .set_index('device_id')['building_id'])
281
- bldg_fold = building_folds(closed, dev_bldg, n_folds, seed)
282
-
283
- n_dev_pos = closed.loc[closed['dies42'] == 1, 'device_id'].nunique()
284
- print(f'pairs {len(closed)} batteries {len(devs)} origins {len(org)} '
285
- f'buildings {bldg_fold.size} '
286
- f'positives {int(closed["dies42"].sum())} rows / {n_dev_pos} batteries')
287
- print('effective n is the BATTERY count, not the row count - regularisation '
288
- 'tuned against rows is ~8x too weak here.\n')
289
-
290
- schemes = {'building': closed['device_id'].map(dev_bldg).map(bldg_fold),
291
- 'battery': closed['device_id'].map(dev_fold),
292
- 'scenario': closed['origin_idx'].map(org_fold)}
293
- out = {}
294
- for scheme, fold in schemes.items():
295
- for kind in kinds:
296
- aucs, r12, r20 = [], [], []
297
- for f in range(n_folds):
298
- tr, te = closed[fold != f], closed[fold == f]
299
- if te['dies42'].sum() < 3:
300
- continue
301
- m = mc.SklearnRUL(kind).fit(tr)
302
- p = m.predict(te)['p42']
303
- aucs.append(roc_auc_score(te['dies42'], p))
304
- # recall at the operating depth, per origin, scaled to the
305
- # held-out fleet's size (12/461 of the fleet)
306
- for oi, g in te.groupby('origin_idx'):
307
- y = g['dies42'].values
308
- if y.sum() == 0:
309
- continue
310
- n = max(1, int(round(12 * len(g) / 461)))
311
- order = np.argsort(-p.loc[g.index].values)
312
- r12.append(y[order[:n]].sum() / y.sum())
313
- r20.append(y[order[:max(1, int(round(20 * len(g) / 461)))]].sum()
314
- / y.sum())
315
- out[(scheme, kind)] = (np.mean(aucs), np.mean(r12), np.mean(r20))
316
-
317
- # LEVELS ARE NOT COMPARABLE ACROSS COLUMNS. A battery fold holds ~1/5 of
318
- # the fleet at every origin (recall at depth 12/461 means top-2 of ~92);
319
- # a scenario fold holds the whole fleet at 1/5 of the origins (top-11 of
320
- # ~430). Same depth fraction, very different discreteness. What IS
321
- # comparable is each model's margin over the linear reference WITHIN a
322
- # column - that margin is the thing a model-selection decision reads.
323
- ref = kinds[0]
324
- print(f"{'model':10s} " + ' '.join(f'{s + " " + m:>16s}' for s in schemes
325
- for m in ('AUC', 'rec12')))
326
- for kind in kinds:
327
- cells = []
328
- for s in schemes:
329
- a, r, _ = out[(s, kind)]
330
- cells += [f'{a:16.4f}', f'{r:16.3f}']
331
- print(f'{kind:10s} ' + ' '.join(cells))
332
- print(f'\nmargin over {ref} (the decision-relevant quantity):')
333
- for kind in kinds[1:]:
334
- cells = []
335
- for s in schemes:
336
- a, r, _ = out[(s, kind)]
337
- a0, r0, _ = out[(s, ref)]
338
- cells += [f'{a - a0:+16.4f}', f'{r - r0:+16.3f}']
339
- print(f'{kind:10s} ' + ' '.join(cells))
340
- print('\nA model whose margin shrinks or reverses as the grouping goes '
341
- 'scenario -> battery -> building was reading memorised identity at '
342
- 'that level (battery signatures, then building/batch siblings). '
343
- 'Selection follows the BUILDING column; it is the only one shaped '
344
- 'like the scored splits, which hold foreign buildings.')
345
- return out
346
-
347
-
348
  # --------------------------------------------------------------------------
349
 
350
  def _base(t):
@@ -411,11 +194,9 @@ def _ranks(t):
411
 
412
 
413
  def _vrel_bldg(t):
414
- # The untested arm of the vrel family: BUILDING-relative voltage. The
415
- # phase-44 refutation covers only a fleet-UNIFORM offset (cannot reorder
416
- # a linear model); a per-building offset does reorder across buildings,
417
- # which is exactly what the building-OOF fit can no longer absorb.
418
- # Deployment-legal: computed from the target fleet's own telemetry.
419
  if 'vrel_bldg' not in t.columns:
420
  dev_b = (pd.read_csv(Path('dataset') / 'train' / 'devices.csv')
421
  .drop_duplicates('device_id')
@@ -431,202 +212,22 @@ VARIANTS = {'base': _base, 'zone': _zone, 'vcorr': _vcorr, 'vrel': _vrel,
431
  'v_last_only': _v_last_only, 'blind4': _blind4, 'blind3': _blind3,
432
  'ranks': _ranks, 'vrel_bldg': _vrel_bldg}
433
 
434
- ABLATION = ('base', 'vrel_only', 'vmin7_only', 'vrel', 'v_last_only')
435
-
436
-
437
- # --------------------------------------------------------------------------
438
- # protocol 4: fixed-N ablation - ORDERING, with the count held out of it
439
- # --------------------------------------------------------------------------
440
-
441
- def _order_stats(p42, deaths, ns):
442
- order = p42.sort_values(ascending=False)
443
- ranks = {b: r for r, b in enumerate(order.index, start=1)}
444
- dr = sorted(ranks[b] for b in deaths if b in ranks)
445
- out = {'n_deaths': len(dr), 'mean_rank': float(np.mean(dr))}
446
- # COUNTS, not ratios: pooling caught/deaths across origins is the only
447
- # correct denominator (a 1-death origin must not outvote a 12-death one)
448
- for n in ns:
449
- out[f'c{n}'] = sum(1 for r in dr if r <= n)
450
- tail = [r for r in dr if r > max(ns)]
451
- out['miss_rank'] = float(np.median(tail)) if tail else np.nan
452
- return out
453
-
454
-
455
- # Voltage-LEVEL features. A fleet whose sensors sit at a different calibration
456
- # offset shifts all of these together; every other feature in FEATURES is a
457
- # difference, a rate, a count or a self-relative fraction (below_frac is scored
458
- # against the device's OWN baseline) and is therefore offset-invariant. vrel
459
- # and vmin7_rank are invariant BY CONSTRUCTION - that is what they are for.
460
- LEVEL_FEATURES = ('v_last', 'v_min_30', 'v_min_7', 'v_corr_last', 'thresh_dist')
461
-
462
-
463
- def apply_fleet_shift(rows, delta, dtau_dv=None):
464
- """Simulate a differently-calibrated target fleet.
465
-
466
- The half-fleet gate splits ONE fleet, so both halves share sensor
467
- calibration, battery batches and the maintenance era - it has no
468
- fleet-level offset by construction and therefore CANNOT test transfer
469
- under one. This injects the offset a real held-out fleet may carry.
470
-
471
- vrel and vmin7_rank are recomputed from the shifted values and come out
472
- numerically identical; that is the whole point of the comparison.
473
- """
474
- r = rows.copy()
475
- for c in LEVEL_FEATURES:
476
- if c in r.columns:
477
- r[c] = r[c] + delta
478
- if dtau_dv and 'tau_projected' in r.columns:
479
- # tau is days-to-threshold read off the temp-corrected curve, so an
480
- # offset moves it along the local slope of that curve
481
- r['tau_projected'] = r['tau_projected'] + delta * dtau_dv
482
- if 'vrel' in r.columns:
483
- r['vrel'] = r['v_last'] - r['v_last'].median()
484
- if 'vmin7_rank' in r.columns:
485
- r['vmin7_rank'] = r['v_min_7'].rank(pct=True)
486
- return r
487
-
488
-
489
- def estimate_dtau_dv(t):
490
- """Empirical days-of-tau per volt, from the fleet's own corrected curve."""
491
- d = t[['tau_projected', 'v_corr_last']].dropna()
492
- d = d[d['tau_projected'].abs() < 400]
493
- if len(d) < 100:
494
- return 0.0
495
- return float(np.polyfit(d['v_corr_last'], d['tau_projected'], 1)[0])
496
-
497
-
498
- def ablate(variants=ABLATION, protocol='all48', origins=range(48),
499
- ns=(8, 12, 16, 20, 30), seeds=(0, 1, 2, 3, 4),
500
- dataset_path=Path('dataset'), shift=0.0, shift_tau=True):
501
- """Phase-39's standing directive, made runnable.
502
-
503
- A feature ablation scored on COST cannot answer whether a feature helped:
504
- under the topsum rule the feature moves p42, p42 moves sum(p42), and
505
- sum(p42) moves N - so depth and order change together and the cost delta
506
- is their sum. That is precisely how the blind lineage concluded a
507
- 4-feature set beat its own champion. This protocol holds N FIXED and
508
- varies only the score, so the only thing being measured is the ordering.
509
-
510
- No planner and no evaluator are involved: recall at fixed N and the rank
511
- distribution of missed deaths are functions of the score alone.
512
- """
513
- t = load_table(dataset_path)
514
- recs = []
515
- if protocol == 'all48':
516
- ctx = mc.load_context(dataset_path)
517
- for idx in origins:
518
- c = ctx[idx]
519
- T = c['start_time']
520
- tr = mc.training_pairs(t, T)
521
- if tr['dies42'].sum() < 3:
522
- continue
523
- rows = (t[t['origin_idx'] == idx].set_index('device_id')
524
- .reindex(c['locs']['battery']).dropna(subset=['v_last']))
525
- eol = c['eol'].reindex(rows.index)
526
- deaths = set(eol.index[(eol > T)
527
- & (eol <= T + pd.Timedelta(days=WINDOW_DAYS))])
528
- if not deaths:
529
- continue
530
- for v in variants:
531
- mc.FEATURES = list(VARIANTS[v](t))
532
- p = mc.SklearnRUL(CHAMPION['model']).fit(tr).predict(rows)['p42']
533
- recs.append(dict(_order_stats(p, deaths, ns),
534
- origin=idx, variant=v, seed=0))
535
- print(f's_{idx}: {len(deaths)} deaths', flush=True, file=sys.stderr)
536
- else:
537
- locations, timeseries, eol_times, scenarios = mc.load_dataset(
538
- dataset_path / 'train')
539
- dtau_dv = estimate_dtau_dv(t)
540
- if shift:
541
- print(f'TARGET-fleet shift {shift:+.3f} V, dtau/dv {dtau_dv:.0f} d/V',
542
- file=sys.stderr)
543
- dev_b = locations.drop_duplicates('battery').set_index('battery')['building']
544
- t['bldg'] = t['device_id'].map(dev_b)
545
- closed = t['origin'] <= t['origin'].max() - pd.Timedelta(days=WINDOW_DAYS)
546
- allsc = list(mc.iterate_scenarios(locations, timeseries, eol_times,
547
- scenarios))
548
- for seed in seeds:
549
- fleet_a, fleet_b = fleet_halves(locations, seed)
550
- foreign = t[closed & t['bldg'].isin(fleet_a)]
551
- models = {}
552
- for v in variants:
553
- mc.FEATURES = list(VARIANTS[v](t))
554
- models[v] = mc.SklearnRUL(CHAMPION['model']).fit(foreign)
555
- for idx in origins:
556
- scenario, locs, cut, eol = allsc[idx]
557
- T = pd.Timestamp(scenario['start_time'])
558
- locs_b = locs[locs['building'].isin(fleet_b)]
559
- rows = (t[t['origin_idx'] == idx].set_index('device_id')
560
- .reindex(locs_b['battery']).dropna(subset=['v_last']))
561
- e = eol.reindex(rows.index)
562
- deaths = set(e.index[(e > T)
563
- & (e <= T + pd.Timedelta(days=WINDOW_DAYS))])
564
- if not deaths:
565
- continue
566
- if shift:
567
- rows = apply_fleet_shift(
568
- rows, shift, dtau_dv if shift_tau else None)
569
- for v in variants:
570
- mc.FEATURES = list(VARIANTS[v](t))
571
- p = models[v].predict(rows)['p42']
572
- recs.append(dict(_order_stats(p, deaths, ns),
573
- origin=idx, variant=v, seed=seed))
574
- print(f'seed {seed} done', flush=True, file=sys.stderr)
575
- return pd.DataFrame(recs), ns
576
-
577
-
578
- def ablation_report(df, ns, label, ref='base'):
579
- print(f'\n=== {label} ===')
580
- g = df.groupby('variant')
581
- deaths = g['n_deaths'].sum()
582
- print(f"{'variant':13s} {'deaths':>6s} " +
583
- ' '.join(f'rec@{n:<3d}' for n in ns) +
584
- f" {'mean_rk':>7s} {'miss_rk':>7s}")
585
- for v in df['variant'].unique():
586
- d = df[df['variant'] == v]
587
- cells = ' '.join(f'{d[f"c{n}"].sum() / deaths[v]:6.3f}' for n in ns)
588
- print(f'{v:13s} {int(deaths[v]):6d} {cells} '
589
- f'{d["mean_rank"].mean():7.1f} {d["miss_rank"].median():7.1f}')
590
-
591
- # paired against the reference, per (seed, origin): the only comparison
592
- # that survives the origin-to-origin variance in how many deaths there are
593
- b = df[df['variant'] == ref].set_index(['seed', 'origin'])
594
- print(f'\npaired vs {ref} (per origin, deaths caught at fixed N):')
595
- for v in df['variant'].unique():
596
- if v == ref:
597
- continue
598
- o = df[df['variant'] == v].set_index(['seed', 'origin'])
599
- cells = []
600
- for n in ns:
601
- d = (o[f'c{n}'] - b[f'c{n}']).dropna()
602
- w, l = int((d > 0).sum()), int((d < 0).sum())
603
- cells.append(f'{d.sum():+4.0f} ({w}W/{l}L)')
604
- print(f' {v:13s} ' + ' '.join(cells))
605
-
606
 
607
  # --------------------------------------------------------------------------
608
- # protocol 5: all48 twin with building-OOF predictions (phase 52 follow-up)
609
  # --------------------------------------------------------------------------
610
 
611
- def run_all48_oof(n_folds=5, seed=0, origins=range(48),
612
- dataset_path=Path('dataset'), kind=None, variant='base',
613
- arms=('self', 'oof'), blend=None, fit='static'):
614
- """The walk-forward cost harness with BUILDING-CLEAN predictions.
615
-
616
- all48 itself is temporal, not fold-based, and its PLANNING must stay
617
- full-fleet: the count rule scales with fleet size, so holding buildings
618
- out of the plan would manufacture the same count bias the static gate
619
- carries (phase 26). What CAN be building-grouped is the prediction: here
620
- device d's p42 at every origin comes from a model fitted on training
621
- pairs from every building EXCEPT d's (OOF stitching), then the FULL
622
- fleet is planned exactly as usual. Deployment analogy: the scored fleets
623
- are foreign buildings, so this measures the walk-forward cost when the
624
- model can never read its own building's identity.
625
 
626
- Both arms (oof and self-fit) run through THIS function's machinery with
627
- the cold-start floor off, so the paired per-origin delta isolates the
628
- self-fleet fit effect. Levels may differ slightly from canonical all48
629
- (floor off, no warm-start state); read the DELTA, not the level.
 
 
 
630
 
631
  Candidate re-tests (phase 52 addendum sweep): `kind` swaps the model
632
  family, `variant` swaps the feature set (VARIANTS key), `blend`=(kind2,
@@ -637,32 +238,12 @@ def run_all48_oof(n_folds=5, seed=0, origins=range(48),
637
  a post-hoc blend whose beliefs disagree with the count machinery fails
638
  for that reason alone).
639
  """
640
- # fit='static' (default; user-shaped 2026-08-22): each fold's model is
641
- # fitted ONCE on the OTHER buildings' FULL closed history and scores all
642
- # 48 origins - exactly deployment's shape (the shipped static model is
643
- # one fit on the train fleet's full history through data end, applied at
644
- # public origins that are mostly in its past; the competition itself
645
- # hands the model future-of-origin FOREIGN-building knowledge, so
646
- # respecting a temporal cutoff for other buildings is stricter than the
647
- # thing being estimated and it starves warm-up in a way deployment never
648
- # does). Calendar co-movement across buildings is therefore IN scope by
649
- # design. fit='walkforward' keeps the old per-origin refits.
650
  kind = kind or CHAMPION['model']
651
  t = load_table(dataset_path)
652
  mc.FEATURES = list(VARIANTS[variant](t))
653
  ctx = mc.load_context(dataset_path)
654
- dev_bldg = (pd.read_csv(dataset_path / 'train' / 'devices.csv')
655
- .drop_duplicates('device_id')
656
- .set_index('device_id')['building_id'])
657
- closed = t[t['origin'] <= t['origin'].max() - pd.Timedelta(days=WINDOW_DAYS)]
658
- if fit == 'static':
659
- # EXACT leave-one-building-out predictions (user spec 2026-08-22):
660
- # every device's model excludes precisely its own building - 24
661
- # models on 23 buildings each, not 5 fold models on ~19. Folds are
662
- # only a cost-saving for the per-origin walkforward variant.
663
- bldg_fold = pd.Series({b: b for b in dev_bldg.unique()})
664
- else:
665
- bldg_fold = building_folds(closed, dev_bldg, n_folds, seed)
666
 
667
  p = mc.plan_tsroute
668
  p.alpha, p.p_be, p.floor_day, p.p_conf = CHAMPION['p_star'], 0.0, 0, 0.5
@@ -670,50 +251,18 @@ def run_all48_oof(n_folds=5, seed=0, origins=range(48),
670
  p.split_days = CHAMPION['split_days']
671
  p.floor_scale, p.recent_deaths = 0.0, 0.0
672
 
673
- static_models = {}
674
- if fit == 'static':
675
- # one fit per fold on the full closed foreign history (+ one all-
676
- # buildings fit for the self arm) - the deployment-shaped protocol
677
- hazard = kind in ('aft', 'dthaz', 'dthaz_aux', 'logbag_open',
678
- 'blend_dh', 'blend_dhaux')
679
- pool = (mc.censored_pairs(t, t['origin'].max()) if hazard
680
- else t[t['origin'] <= t['origin'].max()
681
- - pd.Timedelta(days=WINDOW_DAYS)])
682
- pool_fold = pool['device_id'].map(dev_bldg).map(bldg_fold)
683
- make = ((lambda: mc.MODELS[kind]()) if kind in mc.MODELS
684
- else (lambda: mc.SklearnRUL(kind)))
685
- for f in sorted(bldg_fold.dropna().unique()):
686
- static_models[f] = make().fit(pool[(pool_fold != f).values])
687
- if 'self' in arms:
688
- static_models['SELF'] = make().fit(pool)
689
- print(f'static foreign fits: {len(static_models)} on '
690
- f'{len(pool)} closed pairs', flush=True, file=sys.stderr)
691
-
692
  recs = []
693
  for idx in origins:
694
  c = ctx[idx]
695
  T = c['start_time']
696
- tr = mc.training_pairs(t, T)
697
- if fit != 'static' and tr['dies42'].sum() < 3:
698
- continue
699
  rows = (t[t['origin_idx'] == idx].set_index('device_id')
700
  .reindex(c['locs']['battery']).dropna(subset=['v_last']))
701
  if rows.empty:
702
  continue
703
  locs = c['locs'][c['locs']['battery'].isin(rows.index)]
704
  eol = c['eol'].reindex(rows.index)
705
- tr_fold = tr['device_id'].map(dev_bldg).map(bldg_fold)
706
- row_fold = rows.index.to_series().map(dev_bldg).map(bldg_fold)
707
 
708
- def _pred(k, arm):
709
- if fit == 'static' and k == kind:
710
- if arm == 'self':
711
- return static_models['SELF'].predict(rows)
712
- parts = []
713
- for f in sorted(row_fold.dropna().unique()):
714
- parts.append(static_models[f].predict(
715
- rows[(row_fold == f).values]))
716
- return pd.concat(parts).reindex(rows.index)
717
  if k == 'tau':
718
  # fit-free pseudo-model: the physics timer as a rank source
719
  # (only meaningful inside a value-preserving blend)
@@ -722,156 +271,43 @@ def run_all48_oof(n_folds=5, seed=0, origins=range(48),
722
  index=rows.index)
723
  hazard = k in ('aft', 'dthaz', 'dthaz_aux', 'logbag_open',
724
  'blend_dh', 'blend_dhaux')
725
- trk = mc.censored_pairs(t, T) if hazard else tr
726
- fold_k = (trk['device_id'].map(dev_bldg).map(bldg_fold)
727
- if hazard else tr_fold)
 
 
 
728
  make = ((lambda: mc.MODELS[k]()) if k in mc.MODELS
729
  else (lambda: mc.SklearnRUL(k)))
730
- if arm == 'self':
731
- return make().fit(trk).predict(rows)
732
- parts = []
733
- for f in sorted(row_fold.dropna().unique()):
734
- m = make().fit(trk[fold_k != f])
735
- parts.append(m.predict(rows[(row_fold == f).values]))
736
- return pd.concat(parts).reindex(rows.index)
737
-
738
- for arm in arms:
739
- pred = _pred(kind, arm)
740
- if blend is not None:
741
- k2, w2 = blend
742
- r = ((1 - w2) * pred['p42'].rank(pct=True)
743
- + w2 * _pred(k2, arm)['p42'].rank(pct=True))
744
- vals = np.sort(pred['p42'].values)[::-1]
745
- pred = pred.copy()
746
- pred['p42'] = pd.Series(
747
- vals, index=r.sort_values(ascending=False).index
748
- ).reindex(pred.index)
749
- plan = mc.plan_tsroute(pred, None, locs, T, CHAMPION['day_q'],
750
- travel_costs=c['travel_costs'],
751
- settings=c['settings'])
752
- _, _, overall = evaluate_plan(plan, locs, c['travel_costs'],
753
- c['settings'], eol_times=eol,
754
- start_time=T, verbose=0)
755
- rs = mc.rank_stats(pred, plan, eol, T)
756
- recs.append(dict(overall, **rs, origin=idx, arm=arm, seed=seed))
757
- if len(arms) == 2:
758
- a, b = recs[-2], recs[-1]
759
- print(f"s_{idx}: self {a['total_cost']:7.1f} oof {b['total_cost']:7.1f} "
760
- f"delta {b['total_cost'] - a['total_cost']:+7.1f}",
761
- flush=True, file=sys.stderr)
762
- else:
763
- print(f"s_{idx}: {arms[0]} {recs[-1]['total_cost']:7.1f}",
764
- flush=True, file=sys.stderr)
765
- return pd.DataFrame(recs)
766
-
767
-
768
- def run_lobo(origins=range(48), dataset_path=Path('dataset'), kind=None,
769
- variant='base'):
770
- """Leave-One-Building-Out CV (user-specified 2026-08-22): for each
771
- scenario, for each building, train on ALL OTHER buildings' full closed
772
- history (static, one fit per building - 24 fits total, reused across
773
- scenarios) and plan/score THAT building alone in that scenario.
774
-
775
- CV-first doctrine: this evaluates the current model as-is, including
776
- where its count rule (N = 0.8 * sum p42) breaks at building scale -
777
- those breaks are the model-adaptation to-do list, not instrument noise.
778
- Per-building planning removes cross-building travel/limit coupling, so
779
- this instrument governs prediction+inclusion (late/early, ~85% of cost);
780
- routing questions keep the full-fleet harness as the secondary check.
781
- """
782
- kind = kind or CHAMPION['model']
783
- t = load_table(dataset_path)
784
- mc.FEATURES = list(VARIANTS[variant](t))
785
- ctx = mc.load_context(dataset_path)
786
- dev_bldg = (pd.read_csv(dataset_path / 'train' / 'devices.csv')
787
- .drop_duplicates('device_id')
788
- .set_index('device_id')['building_id'])
789
- pool = t[t['origin'] <= t['origin'].max() - pd.Timedelta(days=WINDOW_DAYS)]
790
- pool_b = pool['device_id'].map(dev_bldg)
791
- make = ((lambda: mc.MODELS[kind]()) if kind in mc.MODELS
792
- else (lambda: mc.SklearnRUL(kind)))
793
- fits = {}
794
- for b in sorted(dev_bldg.unique()):
795
- fits[b] = make().fit(pool[(pool_b != b).values])
796
- print(f'lobo: {len(fits)} leave-one-building-out static fits on '
797
- f'{len(pool)} closed pairs', file=sys.stderr, flush=True)
798
-
799
- p = mc.plan_tsroute
800
- p.alpha, p.p_be, p.floor_day, p.p_conf = CHAMPION['p_star'], 0.0, 0, 0.5
801
- p.ec_margin, p.ec_pmax = CHAMPION['ec_margin'], CHAMPION['ec_pmax']
802
- p.split_days = CHAMPION['split_days']
803
- p.floor_scale, p.recent_deaths = 0.0, 0.0
804
-
805
- recs = []
806
- for idx in origins:
807
- c = ctx[idx]
808
- T = c['start_time']
809
- rows_all = (t[t['origin_idx'] == idx].set_index('device_id')
810
- .reindex(c['locs']['battery']).dropna(subset=['v_last']))
811
- for b, locs_b in c['locs'].groupby(c['locs']['battery'].map(dev_bldg)):
812
- rows = rows_all.reindex(locs_b['battery']).dropna(subset=['v_last'])
813
- if rows.empty:
814
- continue
815
- locs_b = locs_b[locs_b['battery'].isin(rows.index)]
816
- eol_b = c['eol'].reindex(rows.index)
817
- pred = fits[b].predict(rows)
818
- plan = mc.plan_tsroute(pred, None, locs_b, T, CHAMPION['day_q'],
819
- travel_costs=c['travel_costs'],
820
- settings=c['settings'])
821
- _, _, ov = evaluate_plan(plan, locs_b, c['travel_costs'],
822
- c['settings'], eol_times=eol_b,
823
- start_time=T, verbose=0)
824
- WE = T + pd.Timedelta(days=WINDOW_DAYS)
825
- deaths = set(eol_b.index[(eol_b > T) & (eol_b <= WE)])
826
- sched = set(plan.loc[plan['day'] <= WE, 'battery'])
827
- recs.append(dict(origin=idx, building=b, n_dev=len(rows),
828
- total=ov['total_cost'], late=ov['late_swap'],
829
- early=ov['early_swap'],
830
- n_deaths=len(deaths), caught=len(sched & deaths),
831
- n_sched=len(sched)))
832
- d = pd.DataFrame([r for r in recs if r['origin'] == idx])
833
- if not d.empty:
834
- starved = int(((d['n_sched'] == 0) & (d['n_deaths'] > 0)).sum())
835
- print(f"s_{idx}: mean/bldg {d['total'].mean():7.1f} "
836
- f"sum {d['total'].sum():8.1f} "
837
- f"deaths {int(d['n_deaths'].sum()):2d} caught "
838
- f"{int(d['caught'].sum()):2d} zero-swap-with-death bldgs "
839
- f"{starved}", flush=True, file=sys.stderr)
840
  return pd.DataFrame(recs)
841
 
842
 
843
- def lobo_report(df):
844
- per = df.groupby('origin').agg(mean_bldg=('total', 'mean'),
845
- sum_=('total', 'sum'),
846
- deaths=('n_deaths', 'sum'),
847
- caught=('caught', 'sum'),
848
- sched=('n_sched', 'sum'))
849
- print('\n=== lobo / leave-one-building-out, per scenario ===')
850
- print(f"{'scen':>5s} {'mean/bldg':>9s} {'sum':>9s} {'deaths':>6s} "
851
- f"{'caught':>6s} {'sched':>5s}")
852
- for i, r in per.iterrows():
853
- print(f"s_{i:<3d} {r['mean_bldg']:9.1f} {r['sum_']:9.1f} "
854
- f"{int(r['deaths']):6d} {int(r['caught']):6d} {int(r['sched']):5d}")
855
- for name, sel in BLOCKS:
856
- m = per.index.to_series().apply(sel)
857
- d = per[m.values]
858
- if d.empty:
859
- continue
860
- dfb = df[df['origin'].isin(d.index)]
861
- starved = int(((dfb['n_sched'] == 0) & (dfb['n_deaths'] > 0)).sum())
862
- print(f"{name:16s} mean/bldg {d['mean_bldg'].mean():7.1f} "
863
- f"sum/scen {d['sum_'].mean():8.1f} recall "
864
- f"{d['caught'].sum() / max(d['deaths'].sum(), 1):.3f} "
865
- f"zero-swap-death building-scenarios {starved}")
866
- print('\nHEADLINE = mean/bldg on the panel. zero-swap-with-death = the '
867
- 'count-granularity break the model adaptation must fix.')
868
-
869
 
870
- # The re-test grid for the phase 52 addendum sweep: every prediction-side
871
- # candidate the campaign rejected on a self-fit or scenario-grouped surface,
872
- # re-asked on the building-OOF cost harness. Planner/appetite candidates are
873
- # deliberately absent - that axis is closed and this harness would not
874
- # change its answer.
875
  OOF_CANDIDATES = {
876
  'logistic': dict(kind='logistic'),
877
  'rf': dict(kind='rf'),
@@ -908,7 +344,7 @@ def oof_candidate_report(base_csv='/tmp/cv_all48oof.csv',
908
  import glob
909
  base = pd.read_csv(base_csv)
910
  b = base[base['arm'] == 'oof'].set_index('origin')
911
- print('paired vs champion oof arm (logbag/base, floor off, fold seed 0).')
912
  print('sched/rec12 shown because a cost delta can be appetite in '
913
  'disguise (phase 39/51): trust a win only if sched is ~flat.\n')
914
  hdr = (f"{'candidate':14s} {'ALL48':>7s} {'delta':>7s} {'W/L':>7s} "
@@ -933,90 +369,37 @@ def oof_candidate_report(base_csv='/tmp/cv_all48oof.csv',
933
 
934
 
935
  def oof_report(df):
936
- arms = [a for a in ('oof', 'self') if (df['arm'] == a).any()]
937
- for arm in arms:
938
- label = 'building-clean' if arm == 'oof' else 'validity-check arm'
939
- scorecard(df[df['arm'] == arm], f'all48oof / arm={arm} / {label} / floor off')
940
- if 'oof' in arms:
941
- # THE SCORE (user, 2026-08-22): one number - the mean cost per
942
- # scenario over all 48, full-fleet plans, building-clean predictions.
943
- # Blocks above are diagnostics.
944
- o = df[df['arm'] == 'oof']
945
- print(f"\nSCORE (building-split CV, single value): "
946
- f"{o['total_cost'].mean():.1f}")
947
- if 'self' not in arms:
948
- return
949
- # the paired flattery term - only shown when the self arm was requested
950
- s = df[df['arm'] == 'self'].set_index('origin')['total_cost']
951
- o = df[df['arm'] == 'oof'].set_index('origin')['total_cost']
952
- d = (o - s).dropna()
953
- for name, sel in BLOCKS:
954
- dd = d[sel(pd.Series(d.index, index=d.index))]
955
- if dd.empty:
956
- continue
957
- w, l = int((dd < 0).sum()), int((dd > 0).sum())
958
- print(f'{name:16s} oof-minus-self {dd.mean():+7.1f}/origin '
959
- f'(sum {dd.sum():+8.1f}, oof-better {w}W / self-better {l}L)')
960
- print('\nPositive delta = self-fleet fitting was flattering the '
961
- 'walk-forward by that much; the scored fleets are foreign '
962
- 'buildings, so the oof arm is the transfer-shaped cost estimate.')
963
 
964
 
965
  def main():
966
- proto = sys.argv[1] if len(sys.argv) > 1 else 'all48'
967
- variant = sys.argv[2] if len(sys.argv) > 2 else 'base'
968
  t0 = time.time()
969
- if proto == 'all48':
970
- df = run_all48(variant)
971
- df.to_csv(f'/tmp/cv_all48_{variant}.csv', index=False)
972
- scorecard(df, f'all48 / {variant} / full-fleet walk-forward')
973
- elif proto == 'static':
974
- # one building split is one draw; a variant comparison needs several.
975
- # 11s a seed, so there is no excuse for reading a single one.
976
- seeds = range(int(sys.argv[3]) if len(sys.argv) > 3 else 5)
977
- df = pd.concat([run_static(variant, seed=s) for s in seeds],
978
- ignore_index=True)
979
- df.to_csv(f'/tmp/cv_static_{variant}.csv', index=False)
980
- for s in seeds:
981
- scorecard(df[df['seed'] == s], f'static / {variant} / seed {s}')
982
- scorecard(df, f'static / {variant} / DEPLOYMENT GATE, {len(seeds)} '
983
- f'building splits pooled')
984
- print('Levels are NOT comparable to all48 (half the fleet, half the '
985
- 'deaths, and the count rule scales with the fleet). Read this '
986
- 'protocol only as variant-vs-variant, ORDERING only - never for '
987
- 'an appetite/count decision (see module docstring).')
988
- elif proto == 'screen':
989
- screen()
990
- elif proto == 'all48oof':
991
- # THE SCORE (user directives 2026-08-22): oof arm, STATIC fit
992
- # protocol (one fit per fold on the other buildings' full closed
993
- # history - deployment's shape). `cv all48oof self` adds the self
994
- # arm (flattery readout); `cv all48oof wf` uses the old per-origin
995
- # walk-forward refits (whose self arm reproduces panel 1679.2).
996
- arms = ('self', 'oof') if variant == 'self' else ('oof',)
997
- fitp = 'walkforward' if variant == 'wf' else 'static'
998
- df = run_all48_oof(arms=arms, fit=fitp)
999
  df.to_csv('/tmp/cv_all48oof.csv', index=False)
1000
  oof_report(df)
1001
  elif proto == 'oofcand':
1002
  label = sys.argv[2]
1003
- df = run_all48_oof(arms=('oof',), **OOF_CANDIDATES[label])
1004
  df['cand'] = label
1005
  df.to_csv(f'/tmp/cv_oofcand_{label}.csv', index=False)
1006
- scorecard(df, f'all48oof / cand={label} / oof arm / floor off')
1007
- elif proto == 'lobo':
1008
- df = run_lobo()
1009
- df.to_csv('/tmp/cv_lobo.csv', index=False)
1010
- lobo_report(df)
1011
  elif proto == 'oofreport':
1012
  oof_candidate_report()
1013
- elif proto == 'ablate':
1014
- which = sys.argv[2] if len(sys.argv) > 2 else 'all48'
1015
- df, ns = ablate(protocol=which)
1016
- df.to_csv(f'/tmp/cv_ablate_{which}.csv', index=False)
1017
- ablation_report(df, ns, f'fixed-N ablation / {which}')
1018
  else:
1019
- raise SystemExit(f'unknown protocol {proto}')
 
 
 
1020
  print(f'\n[{time.time() - t0:.0f}s]')
1021
 
1022
 
 
1
+ """THE CV (user directive 2026-08-22): the temporal-purge cost harness.
2
+
3
+ ONE protocol lives here, deliberately. For each of the 48 scenarios, fit ONE
4
+ model on ALL buildings' training pairs whose 42d label windows lie entirely
5
+ before or after the scenario's own window, predict every device in the
6
+ scenario, plan the FULL fleet, evaluate the cost. THE SCORE is the mean total
7
+ cost per scenario over all 48.
8
+
9
+ Design notes, paid for in prior phases:
10
+
11
+ * The purge is on the LABEL window, not just the origin: a pair originating
12
+ the day before the scenario start carries the scenario's own deaths in its
13
+ label, so a bare origin-not-in-window cut would leak the events being
14
+ scored. Non-overlap is `origin + W <= T or origin >= T + W`.
15
+
16
+ * PLANNING stays full-fleet: the count rule (N ~ 0.8 * sum p42) scales with
17
+ fleet size, so holding devices out of the plan manufactures a count bias
18
+ (phase 26).
19
+
20
+ * Report ranking precision at the operating N beside the cost (rec12/rec20/
21
+ recall/miss_rank, via `models_cv.rank_stats`): a model can hold AUC 0.947
22
+ and recall 0.000 on the subgroup that drives the late cost; mean cost hides
23
+ whether a change reordered the list or just cut it deeper.
24
+
25
+ * Known optimism channels, accepted by user decision 2026-08-22 with the
26
+ numbers in view: devices are one-shot events, so same-device pairs from
27
+ after the scenario can reveal a device's fate through the purge, and
28
+ after-the-scenario same-fleet data does not exist for the deployed static
29
+ model. Reference points the day the protocol landed (champion logbag/base):
30
+ temporal purge 1972.0 vs exact leave-one-building-out 1877.3 vs
31
+ fit-on-everything 1661.7 - overall the purge read HARSHER than LOBO
32
+ (losing the calendar-local +/-42d window costs more than building-holdout
33
+ does; warmup 3560.3 vs 2875.3), slightly softer on the panel block
34
+ (1839.6 vs 1857.7, recall 0.468 vs 0.461, sched 8.9 vs 9.2).
35
+
36
+ Superseded protocols (all48 walk-forward self-fit, static half-fleet gate,
37
+ screen fold comparison, LOBO/walk-forward all48oof variants, cv lobo,
38
+ fixed-N ablation) were REMOVED 2026-08-22 per the same directive - one
39
+ instrument, so nothing else can be read by accident. They live in git
40
+ history (last full version: commit 227c214 and its successors that day).
41
+ exp_x29/exp_x30 referenced the removed ablation helpers and need that
42
+ older cv.py to run.
 
 
43
 
44
  Usage
45
+ venv/bin/python -m batteryswap_example.cv all48oof # THE SCORE
46
+ venv/bin/python -m batteryswap_example.cv oofcand <label>
47
+ venv/bin/python -m batteryswap_example.cv oofreport
48
  """
49
  import sys
50
  import time
 
77
 
78
 
79
  def load_table(dataset_path=Path('dataset')):
80
+ """The one table build. Every run starts here so a variant can never
81
  silently differ from the champion by a feature-prep line."""
82
  t = mc.load_features(dataset_path)
83
  t['v_proj'] = t['v_last'] + t['slope_lastk'] * t['staleness']
 
127
 
128
 
129
  # --------------------------------------------------------------------------
130
+ # feature variants (candidate re-tests run them through the one harness)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
131
  # --------------------------------------------------------------------------
132
 
133
  def _base(t):
 
194
 
195
 
196
  def _vrel_bldg(t):
197
+ # BUILDING-relative voltage: a per-building offset reorders across
198
+ # buildings. Deployment-legal: computed from the target fleet's own
199
+ # telemetry.
 
 
200
  if 'vrel_bldg' not in t.columns:
201
  dev_b = (pd.read_csv(Path('dataset') / 'train' / 'devices.csv')
202
  .drop_duplicates('device_id')
 
212
  'v_last_only': _v_last_only, 'blind4': _blind4, 'blind3': _blind3,
213
  'ranks': _ranks, 'vrel_bldg': _vrel_bldg}
214
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
215
 
216
  # --------------------------------------------------------------------------
217
+ # THE protocol: full-fleet cost with scenario-clean (temporal-purge) fits
218
  # --------------------------------------------------------------------------
219
 
220
+ def run_all48_oof(origins=range(48), dataset_path=Path('dataset'), kind=None,
221
+ variant='base', blend=None):
222
+ """The cost harness with SCENARIO-CLEAN predictions (temporal holdout).
 
 
 
 
 
 
 
 
 
 
 
223
 
224
+ For each scenario, ONE model is fitted on ALL buildings' training pairs
225
+ whose 42d label windows lie entirely before or after the scenario's own
226
+ window (origin + W <= T or origin >= T + W), then the FULL fleet is
227
+ planned and scored exactly as deployment plans it. Buildings are never
228
+ held out - every device's own history at other times is in train. Early
229
+ scenarios train mostly on the future side, near-end scenarios mostly on
230
+ the past side. Cold-start floor off (no warm-start state to gate on).
231
 
232
  Candidate re-tests (phase 52 addendum sweep): `kind` swaps the model
233
  family, `variant` swaps the feature set (VARIANTS key), `blend`=(kind2,
 
238
  a post-hoc blend whose beliefs disagree with the count machinery fails
239
  for that reason alone).
240
  """
 
 
 
 
 
 
 
 
 
 
241
  kind = kind or CHAMPION['model']
242
  t = load_table(dataset_path)
243
  mc.FEATURES = list(VARIANTS[variant](t))
244
  ctx = mc.load_context(dataset_path)
245
+ W = pd.Timedelta(days=WINDOW_DAYS)
246
+ closed = t[t['origin'] <= t['origin'].max() - W]
 
 
 
 
 
 
 
 
 
 
247
 
248
  p = mc.plan_tsroute
249
  p.alpha, p.p_be, p.floor_day, p.p_conf = CHAMPION['p_star'], 0.0, 0, 0.5
 
251
  p.split_days = CHAMPION['split_days']
252
  p.floor_scale, p.recent_deaths = 0.0, 0.0
253
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
254
  recs = []
255
  for idx in origins:
256
  c = ctx[idx]
257
  T = c['start_time']
 
 
 
258
  rows = (t[t['origin_idx'] == idx].set_index('device_id')
259
  .reindex(c['locs']['battery']).dropna(subset=['v_last']))
260
  if rows.empty:
261
  continue
262
  locs = c['locs'][c['locs']['battery'].isin(rows.index)]
263
  eol = c['eol'].reindex(rows.index)
 
 
264
 
265
+ def _pred(k):
 
 
 
 
 
 
 
 
266
  if k == 'tau':
267
  # fit-free pseudo-model: the physics timer as a rank source
268
  # (only meaningful inside a value-preserving blend)
 
271
  index=rows.index)
272
  hazard = k in ('aft', 'dthaz', 'dthaz_aux', 'logbag_open',
273
  'blend_dh', 'blend_dhaux')
274
+ # ONE fit on ALL buildings, training-pair label windows disjoint
275
+ # from the scenario's (T, T+W] window (the purge is approximate
276
+ # for hazard kinds, whose censored observations can span it)
277
+ pool = (mc.censored_pairs(t, t['origin'].max()) if hazard
278
+ else closed)
279
+ pool = pool[(pool['origin'] + W <= T) | (pool['origin'] >= T + W)]
280
  make = ((lambda: mc.MODELS[k]()) if k in mc.MODELS
281
  else (lambda: mc.SklearnRUL(k)))
282
+ return make().fit(pool).predict(rows)
283
+
284
+ pred = _pred(kind)
285
+ if blend is not None:
286
+ k2, w2 = blend
287
+ r = ((1 - w2) * pred['p42'].rank(pct=True)
288
+ + w2 * _pred(k2)['p42'].rank(pct=True))
289
+ vals = np.sort(pred['p42'].values)[::-1]
290
+ pred = pred.copy()
291
+ pred['p42'] = pd.Series(
292
+ vals, index=r.sort_values(ascending=False).index
293
+ ).reindex(pred.index)
294
+ plan = mc.plan_tsroute(pred, None, locs, T, CHAMPION['day_q'],
295
+ travel_costs=c['travel_costs'],
296
+ settings=c['settings'])
297
+ _, _, overall = evaluate_plan(plan, locs, c['travel_costs'],
298
+ c['settings'], eol_times=eol,
299
+ start_time=T, verbose=0)
300
+ rs = mc.rank_stats(pred, plan, eol, T)
301
+ recs.append(dict(overall, **rs, origin=idx, arm='oof'))
302
+ print(f"s_{idx}: oof {recs[-1]['total_cost']:7.1f}",
303
+ flush=True, file=sys.stderr)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
304
  return pd.DataFrame(recs)
305
 
306
 
307
+ # --------------------------------------------------------------------------
308
+ # candidate sweep on the one harness
309
+ # --------------------------------------------------------------------------
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
310
 
 
 
 
 
 
311
  OOF_CANDIDATES = {
312
  'logistic': dict(kind='logistic'),
313
  'rf': dict(kind='rf'),
 
344
  import glob
345
  base = pd.read_csv(base_csv)
346
  b = base[base['arm'] == 'oof'].set_index('origin')
347
+ print('paired vs champion (logbag/base, temporal purge, floor off).')
348
  print('sched/rec12 shown because a cost delta can be appetite in '
349
  'disguise (phase 39/51): trust a win only if sched is ~flat.\n')
350
  hdr = (f"{'candidate':14s} {'ALL48':>7s} {'delta':>7s} {'W/L':>7s} "
 
369
 
370
 
371
  def oof_report(df):
372
+ scorecard(df, 'all48oof / scenario-clean (temporal purge) / floor off')
373
+ # THE SCORE (user, 2026-08-22): one number - the mean cost per scenario
374
+ # over all 48, full-fleet plans, scenario-clean predictions. Blocks
375
+ # above are diagnostics.
376
+ print(f"\nSCORE (scenario-clean temporal purge, mean cost/scenario): "
377
+ f"{df['total_cost'].mean():.1f}")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
378
 
379
 
380
  def main():
381
+ proto = sys.argv[1] if len(sys.argv) > 1 else 'all48oof'
 
382
  t0 = time.time()
383
+ if proto == 'all48oof':
384
+ # THE SCORE (user directive 2026-08-22 v2): per scenario, one fit on
385
+ # ALL buildings' pairs purged of the scenario's 42d window; no
386
+ # building holdout; full-fleet plan; mean cost over the 48.
387
+ df = run_all48_oof()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
388
  df.to_csv('/tmp/cv_all48oof.csv', index=False)
389
  oof_report(df)
390
  elif proto == 'oofcand':
391
  label = sys.argv[2]
392
+ df = run_all48_oof(**OOF_CANDIDATES[label])
393
  df['cand'] = label
394
  df.to_csv(f'/tmp/cv_oofcand_{label}.csv', index=False)
395
+ scorecard(df, f'all48oof / cand={label} / temporal purge / floor off')
 
 
 
 
396
  elif proto == 'oofreport':
397
  oof_candidate_report()
 
 
 
 
 
398
  else:
399
+ raise SystemExit(
400
+ f'unknown protocol {proto!r} - the temporal-purge harness is the '
401
+ f'only CV (user directive 2026-08-22). all48/static/screen/lobo/'
402
+ f'ablate were removed; recover from git history if ever needed.')
403
  print(f'\n[{time.time() - t0:.0f}s]')
404
 
405