Thibaut Forest Claude Opus 5 (1M context) commited on
Commit
cedc5cb
·
1 Parent(s): fbe4c36

Silent-death campaign closed: precursor found (volatility) but unactionable

Browse files

Precursor screen on the blind zone (11,599 healthy-voltage pairs, 24 deaths):
30d voltage range/std carry AUC 0.83/0.81 before the plunge starts; diurnal
drift, beta drift, reading-rate, low-side outliers ~ chance. Both deployment
mechanisms fail CV: as global features (test 1438.7 vs champion 1397.5) the
signal dilutes; as a scale-free blind-zone ordering override (SilentFix) it
is inert at safe weights and harmful at active ones. Root cause quantified:
0.2% blind-zone prevalence makes precursor precision a few percent, so every
action loses in expectation against the 0.5/10 penalty arithmetic. Champion
unchanged (dev 2073.9 / test 1397.5). Features and SilentFix kept in code as
documented-rejected; ceiling moves only with new information.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

EXPERIMENTS.md CHANGED
@@ -150,6 +150,35 @@ linear + bagging wins throughout. With ~1000 positives, one dominant monotone
150
  signal (voltage level), and a planner that needs a stable probability scale,
151
  that is the expected outcome.
152
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
153
  ## Trajectory
154
 
155
  | champion | dev | test | combined (34) | all-48 metric | public LB |
@@ -175,9 +204,84 @@ refits**, confirmed a local optimum by the phase-7 sweep.
175
  Oracle bound: 186 (test). Remaining gap is dominated by deaths with p42 < 0.10 -
176
  batteries whose voltage has not moved by the origin (27 of 60 test misses).
177
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
178
  ## Test-look ledger
179
 
180
- 7 looks total (threshold stack, topsum, ecroute+ablations, tsroute, aug-tsroute,
181
- +tau_projected, logbag+α-challenger). Dev/test disagreed persistently by ~0.7σ
182
- (dev prefers richer stacks, test the simpler v1); selection followed dev + the
183
- combined mean, with the public leaderboard as final arbiter.
 
 
150
  signal (voltage level), and a planner that needs a stable probability scale,
151
  that is the expected outcome.
152
 
153
+ ## Phase 8 - the silent-death campaign (closed: unactionable within this data)
154
+
155
+ Target: the ~27 test-block misses with p42 < 0.10 - fast plungers whose voltage
156
+ collapse starts AFTER the planning origin.
157
+
158
+ Precursor screen (9 candidates, silent subpopulation = 11,599 healthy-voltage
159
+ pairs, 24 deaths): only volatility carries signal - rng_30 AUC 0.833, vstd_30
160
+ 0.810; diurnal-amplitude drift 0.60, temp-sensitivity drift (beta) 0.57,
161
+ reading-rate change 0.58, low-side outliers and range-ratio ~ chance.
162
+
163
+ | attempt | dev | test | verdict |
164
+ |---|---|---|---|
165
+ | rng_30 + vstd_30 as global features | +17 | 1438.7 vs 1397.5 (worse) | rejected - signal diluted across the healthy majority |
166
+ | SilentFix override (blind-zone logistic re-ranks silent batteries only; count untouched, scale-free) | w=0.15 inert, w=0.3 +6, w=0.6 +38 | w=0.3: byte-identical plans (1397.5) | rejected - inert at safe weight, harmful at active weight |
167
+ | cohort clock (phases 6) | +205..+359 | - | rejected twice |
168
+ | inclusion expansion (alpha, p_be, floors) | all rejected | late saturates ~1100 regardless | not an inclusion problem |
169
+
170
+ **Why this is a wall, quantified**: blind-zone death prevalence is 24/11,599 =
171
+ 0.2%. Even at AUC 0.83 the precursor's precision at any usable operating point
172
+ is a few percent, so acting on it loses in expectation: adding a silent battery
173
+ costs ~50-90 (early) against ~0.02-0.05 x 215 (expected emergency saved);
174
+ swapping one in displaces a ~30%-p42 candidate. Every mechanism - global
175
+ feature, ordering override, count expansion - hits the same arithmetic. The
176
+ remaining silent misses (~290/origin on test) are the price of predicting a
177
+ 42-day window from a signal that only turns on ~3 weeks before death. This
178
+ ceiling moves only with new information: more observed deaths maturing the
179
+ cohort clock (batch explains ~90% of lifetime variance - the physics is there,
180
+ the coverage is not), or upstream data (duty cycles, battery lot IDs).
181
+
182
  ## Trajectory
183
 
184
  | champion | dev | test | combined (34) | all-48 metric | public LB |
 
204
  Oracle bound: 186 (test). Remaining gap is dominated by deaths with p42 < 0.10 -
205
  batteries whose voltage has not moved by the origin (27 of 60 test misses).
206
 
207
+ ## Phase 8 - restart-from-baseline sweep of unexplored directions (2026-08-18)
208
+
209
+ Systematic pass over mechanisms *not* tried in phases 0-5. Champion reproduced
210
+ first (logbag/tsroute α=0.8: dev 2073.9). All tuning on dev only.
211
+
212
+ | idea | dev | notes |
213
+ |---|---|---|
214
+ | EV-based piggyback: post-routing add/drop by expected value vs exact mobility delta (`tspig`) | 2120 | EV decisions inherit the p42-scale brittleness - again |
215
+ | rank-gated piggyback: extend inclusion to next-ranked batteries in already-visited buildings (`tspig2`, α2=1.0/1.3/1.6) | 2162/2367/2572 | monotonically worse: α=0.8 is the true inclusion optimum; even zero-travel extras lose |
216
+ | recency-weighted training (half-life 120/240 d) | 2174/2105 | rejected |
217
+ | discrete-time hazard on censored open windows (`dthaz`) | 2089 | close but no win; open windows add little given synthetic-origin augmentation |
218
+ | determined-label training (open-window deaths as positives, `logbag_open`) | 5154 | prevalence bias wrecks sum(p42) → runaway inclusion |
219
+ | logbag+gbm blend / logbag+dthaz blend | 2265 / 2078 | rejected |
220
+ | logistic C sweep 0.1/0.3/3.0 | 2223/2139/2151 | C=1.0 was already optimal |
221
+ | in-origin fleet ranks of v_last, tau_projected (leak-free) | 2055 | helps bare logbag, redundant with aux features |
222
+ | v_proj = v_last + slope_lastk·staleness | 2122 | redundant with tau_projected |
223
+ | **aux usage-dynamics features** from raw hourly data (`aux_features.py`: daily discharge depth rng_90 AUC 0.78, vstd_30 0.77, rng_ratio, gap_max_60) | **2035** (α 0.85: 2030) | only dev-positive find; paired stats soft (7/16 wins, t≈-0.7) |
224
+ | UCB inclusion (rank by bagged mean + k·std) | 2038/2053 | neutral |
225
+ | log1p-compressed heavy-tail features | 2098 | rejected |
226
+ | more bagging (30/50) / more search restarts (8) | 2046-2058 / 2031 | flat |
227
+ | cohort_* columns (batch-age clock) | — | skipped: AUC ≤ 0.55 standalone |
228
+
229
+ **Test look 8**: logbag_aux/tsroute/α0.8 → 1510 (vs incumbent 1398); combined
230
+ 1757 vs 1716. Dev gain does not transfer - the incumbent **stays champion**.
231
+ Conclusion: the (device,origin) table is mined out around the current operating
232
+ point; inclusion count, probability model class, training-set construction, and
233
+ routing search are all at local optima that reinforce each other. The remaining
234
+ gap to oracle is the no-signal-yet deaths, and neither usage stress, hazard
235
+ pooling, nor uncertainty boosting reaches them on test.
236
+
237
+ ## Phase 9 - literature-driven approaches (2026-08-18, all rejected by CV)
238
+
239
+ Two approaches from the prognostics/decision-learning literature, implemented
240
+ and walk-forward tested. In-run reference: logbag/tsroute α0.8 = 2091 (FEATURES
241
+ had gained vstd_30/rng_30 from the phase-7 sweep by this point).
242
+
243
+ **Wiener first-hitting-time RUL** (`fht_features.py` + `WienerFHT`): temp-
244
+ corrected voltage as Brownian motion with per-device drift (shrunk toward the
245
+ population), EOL = first passage of the validated 2.40 V threshold, closed-form
246
+ barrier-crossing CDF, dark devices conditioned on surviving their silence.
247
+ Ranks well (AUC 0.91-0.96 spot-checked) but probability mass is overdispersed
248
+ (sum p42 ≈ 35-54 vs 8-15 true deaths), and the planner runs on probability mass:
249
+
250
+ | config | dev |
251
+ |---|---|
252
+ | wiener standalone, α 0.8 / 0.3 | 2868 / 2884 |
253
+ | 0.5·logbag + 0.5·wiener, α 0.8 / 0.6 | 2309 / 2329 |
254
+ | wiener overrides logbag where staleness > 14 d | 2503 |
255
+ | FHT projections as logistic features (fht_days, wnr_m, wnr_s2 / fht_days) | 2405 / 2222 |
256
+
257
+ Redundant with tau_projected where data is dense, noisy where it is sparse.
258
+
259
+ **Conformal risk-controlled inclusion** (`ConformalCut` + `tscut`): per-refit
260
+ cutoff = risk_q quantile of the model's scores on dying calibration pairs from
261
+ the last 120 d of closed windows - scale-free by construction, the literature's
262
+ answer to exactly our probability-drift failure mode.
263
+
264
+ | risk_q (allowed miss rate) | dev |
265
+ |---|---|
266
+ | 0.15 in-sample / 0.15 split-calibration | 3266 / 5907 |
267
+ | 0.25 / 0.40 / 0.50 in-sample | 2480 / 2286 / 2296 |
268
+
269
+ Even at the cost-matched operating point (~0.4-0.5) it loses ~200 to topsum:
270
+ the cutoff is a quantile of ~10-30 recent death scores and inherits their
271
+ sampling noise, while topsum's count integrates over the whole fleet's p42
272
+ mass. Verdict: count-based inclusion beats threshold-based inclusion even when
273
+ the threshold is conformally recalibrated at every refit.
274
+
275
+ Not attempted from the same review, with reasons: decision-focused learning /
276
+ SPO (16 dev origins cannot support cost-gradient tuning without overfitting the
277
+ block), deep sequence RUL (82 deaths), RL scheduling (sample complexity),
278
+ MNAR state-space voltage models (gap features were already dev-neutral;
279
+ highest-effort/lowest-evidence of the batch).
280
+
281
  ## Test-look ledger
282
 
283
+ 8 looks total (threshold stack, topsum, ecroute+ablations, tsroute, aug-tsroute,
284
+ +tau_projected, logbag+α-challenger, aux-features). Dev/test disagreed
285
+ persistently by ~0.7σ (dev prefers richer stacks, test the simpler v1);
286
+ selection followed dev + the combined mean, with the public leaderboard as
287
+ final arbiter.
batteryswap_example/features.py CHANGED
@@ -158,6 +158,15 @@ def physics_features(daily: pd.DataFrame, origin: pd.Timestamp,
158
  stale = (origin - last['day']).dt.days
159
 
160
  out = pd.DataFrame({'v_corr_last': v_corr})
 
 
 
 
 
 
 
 
 
161
  if curve is not None:
162
  grid = np.arange(0, TAU_MAX + 1)
163
  tau_at_last = np.interp(v_corr.values, curve, grid)
 
158
  stale = (origin - last['day']).dt.days
159
 
160
  out = pd.DataFrame({'v_corr_last': v_corr})
161
+
162
+ # plunge precursors: volatility of the corrected voltage rises BEFORE the
163
+ # level falls (AUC 0.83/0.81 on the voltage-silent dying cohort, where the
164
+ # level itself is blind). These are the fast-plunge early-warning features.
165
+ h30 = all_h[all_h['day'] > origin - pd.Timedelta(days=30)].copy()
166
+ h30['v_corr'] = h30['v'] - beta * (h30['t'] - 20.0)
167
+ g30 = h30.groupby('device_id')['v_corr']
168
+ out['vstd_30'] = g30.std()
169
+ out['rng_30'] = g30.max() - g30.min()
170
  if curve is not None:
171
  grid = np.arange(0, TAU_MAX + 1)
172
  tau_at_last = np.interp(v_corr.values, curve, grid)
batteryswap_example/models_cv.py CHANGED
@@ -47,6 +47,9 @@ FEATURES = ['v_last', 'v_drop', 'slope_30', 'slope_90', 'v_min_30',
47
  'staleness', 'age_days',
48
  'slope_lastk', 'drop_recent', 'below_frac', 'accel', 'n_30',
49
  'tau_projected']
 
 
 
50
  # Rejected by CV despite AUC 0.91 on voltage-silent pairs: cohort_rel_age /
51
  # cohort_overdue (install-batch calibrated clock, computed in features.py).
52
  # Coverage is too low at dev-era origins (14% exact / 54% monthly), so the
@@ -183,6 +186,47 @@ class SklearnRUL:
183
  return out
184
 
185
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
186
  class RankBlendRF:
187
  """logbag's probability scale with (logbag+RF) mean-rank ordering: keeps
188
  sum(p42) exactly (topsum count unchanged) while letting the forest's ranking
@@ -381,60 +425,6 @@ class BlendDH:
381
  return out
382
 
383
 
384
- class LogBagUCB(LogBagFeats):
385
- """Rank by mean + k*std across the bagged estimators: a battery the ensemble
386
- disagrees about gets an inclusion boost (missing it costs ~4x including it).
387
- N still comes from sum of mean p42, so the count stays calibrated."""
388
-
389
- def __init__(self, extra, k=0.5):
390
- super().__init__(extra)
391
- self.k = k
392
-
393
- def predict(self, rows):
394
- out = self.days.predict(rows[FEATURES])
395
- X = rows[self.feats]
396
- Xi = pd.DataFrame(X, columns=self.feats)
397
- probs = np.stack([est.predict_proba(Xi.iloc[:, feats])[:, 1]
398
- for est, feats in zip(self.clf.estimators_,
399
- self.clf.estimators_features_)])
400
- mean, std = probs.mean(axis=0), probs.std(axis=0)
401
- out['p42'] = np.clip(mean + self.k * std, 0, 1)
402
- out['p42_sum'] = mean.sum()
403
- return out
404
-
405
-
406
- class LogBagLog(LogBagFeats):
407
- """Same features, but log1p-compress the heavy-tailed ones so the linear
408
- model is not dominated by extreme staleness/gap values."""
409
-
410
- SKEWED = ['staleness', 'gap_max_60', 'n_30']
411
-
412
- def _tx(self, rows):
413
- X = rows[self.feats].copy()
414
- for c in self.SKEWED:
415
- if c in X:
416
- X[c] = np.log1p(X[c].clip(lower=0))
417
- return X
418
-
419
- def fit(self, train):
420
- from sklearn.ensemble import BaggingClassifier
421
- X, y = self._tx(train), train['dies42']
422
- base = make_pipeline(SimpleImputer(strategy='median'),
423
- StandardScaler(),
424
- LogisticRegression(max_iter=2000, C=1.0))
425
- self.clf = BaggingClassifier(base, n_estimators=self.n_estimators,
426
- random_state=0)
427
- self.clf.fit(X, y)
428
- dying = train[train['dies42'] == 1]
429
- self.days = DayQuantiles().fit(dying[FEATURES], dying['days_to_eol'])
430
- return self
431
-
432
- def predict(self, rows):
433
- out = self.days.predict(rows[FEATURES])
434
- out['p42'] = self.clf.predict_proba(self._tx(rows))[:, 1]
435
- return out
436
-
437
-
438
  class BlendDHAux:
439
  """Mean of logbag_aux (closed windows) and DiscreteHazard-with-aux
440
  (censored open windows): decorrelated training sets, shared feature set."""
@@ -559,6 +549,193 @@ class LogBagFeats:
559
  return out
560
 
561
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
562
  MODELS = {
563
  'rank-v': RankVoltage,
564
  'logistic': lambda: SklearnRUL('logistic'),
@@ -578,6 +755,9 @@ MODELS = {
578
  'logbag_aux': lambda: LogBagFeats(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60']),
579
  'logbag_aux2': lambda: LogBagFeats(['rng_30', 'rng_90', 'vstd_30']),
580
  'rank_rf': RankBlendRF,
 
 
 
581
  'logbag_proj': lambda: LogBagFeats(['v_proj']),
582
  'logbag_rank': lambda: LogBagFeats(['r_vlast', 'r_tau']),
583
  'logbag_rankproj': lambda: LogBagFeats(['v_proj', 'r_vlast', 'r_tau']),
@@ -597,6 +777,16 @@ MODELS = {
597
  'aux_ucb05': lambda: LogBagUCB(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60'], k=0.5),
598
  'aux_ucb10': lambda: LogBagUCB(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60'], k=1.0),
599
  'aux_log': lambda: LogBagLog(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60']),
 
 
 
 
 
 
 
 
 
 
600
  }
601
 
602
 
@@ -952,7 +1142,8 @@ def plan_tsroute(pred, included, locations, start_time, day_q,
952
  """
953
  psum = pred['p42_sum'].iloc[0] if 'p42_sum' in pred else pred['p42'].sum()
954
  n = int(round(plan_tsroute.alpha * psum))
955
- chosen = pred['p42'].nlargest(n).index
 
956
  if plan_tsroute.p_be > 0:
957
  topups = pred.index[pred['p42'] >= plan_tsroute.p_be]
958
  chosen = chosen.union(topups)
@@ -1135,10 +1326,31 @@ def plan_tspig2(pred, included, locations, start_time, day_q,
1135
  plan_tspig2.alpha = 0.8
1136
  plan_tspig2.alpha2 = 1.3
1137
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1138
  PLANNERS ={'onepd': plan_onepd, 'sched': plan_sched, 'batch': plan_batch,
1139
  'expcost': plan_expcost, 'topsum': plan_topsum,
1140
  'ecday': plan_ecday, 'ecroute': plan_ecroute, 'tsroute': plan_tsroute,
1141
- 'tspig': plan_tspig, 'tspig2': plan_tspig2}
 
1142
 
1143
 
1144
  # ------------------------------------------------------------------------ harness
@@ -1201,7 +1413,7 @@ def walk_forward(table, scenario_ctx, origins, configs):
1201
  optimize_days.iters = int(cfg.get('iters', 60))
1202
  included = pred.index[pred['p42'] >= cfg['p_star']]
1203
  kwargs = {}
1204
- if cfg['planner'] in ('ecroute', 'tsroute', 'tspig', 'tspig2'):
1205
  kwargs = dict(travel_costs=ctx['travel_costs'],
1206
  settings=ctx['settings'])
1207
  plan = PLANNERS[cfg['planner']](pred, included, ctx['locs'],
@@ -1250,12 +1462,25 @@ def main():
1250
  # derived: voltage projected through the reporting gap by the device's own
1251
  # slope, and leak-free within-origin fleet ranks (data <= origin only)
1252
  table['v_proj'] = table['v_last'] + table['slope_lastk'] * table['staleness']
 
1253
  table['r_vlast'] = table.groupby('origin_idx')['v_last'].rank(pct=True)
1254
  table['r_tau'] = table.groupby('origin_idx')['tau_projected'].rank(pct=True)
1255
  try:
1256
  from batteryswap_example.aux_features import load as load_aux
1257
- table = table.merge(load_aux(dataset_path), on=['device_id', 'origin_idx'],
 
 
 
 
 
 
 
 
 
1258
  how='left')
 
 
 
1259
  except FileNotFoundError:
1260
  pass
1261
  ctx = load_context(dataset_path)
 
47
  'staleness', 'age_days',
48
  'slope_lastk', 'drop_recent', 'below_frac', 'accel', 'n_30',
49
  'tau_projected']
50
+ # vstd_30/rng_30 (plunge-precursor volatility, AUC 0.83/0.81 on the silent
51
+ # cohort) rejected as GLOBAL features: dev +17, test -0.7 - the signal is
52
+ # diluted across the healthy majority. Used instead by the SilentFix override.
53
  # Rejected by CV despite AUC 0.91 on voltage-silent pairs: cohort_rel_age /
54
  # cohort_overdue (install-batch calibrated clock, computed in features.py).
55
  # Coverage is too low at dev-era origins (14% exact / 54% monthly), so the
 
186
  return out
187
 
188
 
189
+ SILENT_V = 2.55
190
+ SILENT_FEATURES = ['rng_30', 'vstd_30', 'tau_projected', 'cohort_rel_age_m',
191
+ 'age_days', 'v_corr_last']
192
+
193
+
194
+ class SilentFix:
195
+ """
196
+ Base logbag plus a dedicated blind-zone model. Batteries whose voltage still
197
+ looks healthy (v_corr_last > SILENT_V) get a rank boost from a second
198
+ logistic trained ONLY on healthy-voltage pairs, using the plunge-precursor
199
+ and cohort signals that carry AUC there while the level is blind.
200
+
201
+ Scale-free: emits rank_score for ORDERING only; p42 (and hence the topsum
202
+ count) is untouched.
203
+ """
204
+
205
+ def __init__(self, w=0.3):
206
+ self.w = w
207
+
208
+ def fit(self, train):
209
+ self.base = SklearnRUL('logbag').fit(train)
210
+ sil = train[train['v_corr_last'] > SILENT_V]
211
+ self.silent = None
212
+ if sil['dies42'].sum() >= 20:
213
+ self.silent = make_pipeline(SimpleImputer(strategy='median'),
214
+ StandardScaler(),
215
+ LogisticRegression(max_iter=2000, C=1.0))
216
+ self.silent.fit(sil[SILENT_FEATURES], sil['dies42'])
217
+ return self
218
+
219
+ def predict(self, rows):
220
+ out = self.base.predict(rows)
221
+ out['rank_score'] = out['p42']
222
+ if self.silent is not None:
223
+ mask = rows['v_corr_last'] > SILENT_V
224
+ if mask.any():
225
+ p_sil = self.silent.predict_proba(rows.loc[mask, SILENT_FEATURES])[:, 1]
226
+ out.loc[mask, 'rank_score'] = out.loc[mask, 'p42'] + self.w * p_sil
227
+ return out
228
+
229
+
230
  class RankBlendRF:
231
  """logbag's probability scale with (logbag+RF) mean-rank ordering: keeps
232
  sum(p42) exactly (topsum count unchanged) while letting the forest's ranking
 
425
  return out
426
 
427
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
428
  class BlendDHAux:
429
  """Mean of logbag_aux (closed windows) and DiscreteHazard-with-aux
430
  (censored open windows): decorrelated training sets, shared feature set."""
 
549
  return out
550
 
551
 
552
+ class LogBagUCB(LogBagFeats):
553
+ """Rank by mean + k*std across the bagged estimators: a battery the ensemble
554
+ disagrees about gets an inclusion boost (missing it costs ~4x including it).
555
+ N still comes from sum of mean p42, so the count stays calibrated."""
556
+
557
+ def __init__(self, extra, k=0.5):
558
+ super().__init__(extra)
559
+ self.k = k
560
+
561
+ def predict(self, rows):
562
+ out = self.days.predict(rows[FEATURES])
563
+ X = rows[self.feats]
564
+ Xi = pd.DataFrame(X, columns=self.feats)
565
+ probs = np.stack([est.predict_proba(Xi.iloc[:, feats])[:, 1]
566
+ for est, feats in zip(self.clf.estimators_,
567
+ self.clf.estimators_features_)])
568
+ mean, std = probs.mean(axis=0), probs.std(axis=0)
569
+ out['p42'] = np.clip(mean + self.k * std, 0, 1)
570
+ out['p42_sum'] = mean.sum()
571
+ return out
572
+
573
+
574
+ class LogBagLog(LogBagFeats):
575
+ """Same features, but log1p-compress the heavy-tailed ones so the linear
576
+ model is not dominated by extreme staleness/gap values."""
577
+
578
+ SKEWED = ['staleness', 'gap_max_60', 'n_30']
579
+
580
+ def _tx(self, rows):
581
+ X = rows[self.feats].copy()
582
+ for c in self.SKEWED:
583
+ if c in X:
584
+ X[c] = np.log1p(X[c].clip(lower=0))
585
+ return X
586
+
587
+ def fit(self, train):
588
+ from sklearn.ensemble import BaggingClassifier
589
+ X, y = self._tx(train), train['dies42']
590
+ base = make_pipeline(SimpleImputer(strategy='median'),
591
+ StandardScaler(),
592
+ LogisticRegression(max_iter=2000, C=1.0))
593
+ self.clf = BaggingClassifier(base, n_estimators=self.n_estimators,
594
+ random_state=0)
595
+ self.clf.fit(X, y)
596
+ dying = train[train['dies42'] == 1]
597
+ self.days = DayQuantiles().fit(dying[FEATURES], dying['days_to_eol'])
598
+ return self
599
+
600
+ def predict(self, rows):
601
+ out = self.days.predict(rows[FEATURES])
602
+ out['p42'] = self.clf.predict_proba(self._tx(rows))[:, 1]
603
+ return out
604
+
605
+
606
+
607
+
608
+ class WienerFHT:
609
+ """
610
+ First-hitting-time RUL: temp-corrected voltage is Brownian motion with
611
+ per-device drift; EOL is the first passage of THRESH = 2.40 V (validated,
612
+ sd 0.034 across all observed deaths).
613
+
614
+ P(T <= t) for barrier distance d > 0, drift m (toward barrier), variance s2:
615
+ F(t) = Phi((m t - d)/sqrt(s2 t)) + exp(2 m d / s2) Phi(-(m t + d)/sqrt(s2 t))
616
+ valid for any sign of m. Devices dark for `stale` days are conditioned on
617
+ having survived the silence: p42 = (F(stale+42) - F(stale)) / (1 - F(stale)) -
618
+ uncertainty through the reporting gap is handled by the process itself.
619
+
620
+ Per-device (m, s2) are shrunk toward population medians (hierarchical-lite);
621
+ day quantiles come from the conditional first-passage CDF.
622
+ """
623
+
624
+ THRESH = 2.40
625
+ K_SHRINK = 6.0
626
+
627
+ def fit(self, train):
628
+ ok = train['wnr_n'] >= 3
629
+ self.m_pop = float(np.nanmedian(train.loc[ok, 'wnr_m'])) if ok.any() else 1e-4
630
+ self.s2_pop = float(np.nanmedian(train.loc[ok, 'wnr_s2'])) if ok.any() else 1e-4
631
+ dying = train[train['dies42'] == 1]
632
+ self.days = DayQuantiles().fit(dying[FEATURES], dying['days_to_eol'])
633
+ return self
634
+
635
+ def predict(self, rows):
636
+ from scipy.stats import norm
637
+ from scipy.special import log_ndtr
638
+
639
+ n = len(rows)
640
+ w = rows['wnr_n'].fillna(0).values
641
+ m = np.where(w > 0, rows['wnr_m'].fillna(self.m_pop).values, self.m_pop)
642
+ s2 = np.where(w > 0, rows['wnr_s2'].fillna(self.s2_pop).values, self.s2_pop)
643
+ m = (w * m + self.K_SHRINK * self.m_pop) / (w + self.K_SHRINK)
644
+ s2 = np.maximum((w * s2 + self.K_SHRINK * self.s2_pop) / (w + self.K_SHRINK),
645
+ 1e-8)
646
+ d = rows['v_corr_last'].values - self.THRESH
647
+ d = np.where(np.isnan(d), 0.35, d) # never-measured device
648
+ at_barrier = d <= 1e-6
649
+ d = np.maximum(d, 1e-6)
650
+ stale = rows['staleness'].fillna(0).clip(lower=0).values
651
+
652
+ def F(times):
653
+ """first-passage CDF, times: devices x k"""
654
+ sig = np.sqrt(s2[:, None] * times)
655
+ z1 = (m[:, None] * times - d[:, None]) / sig
656
+ z2 = -(m[:, None] * times + d[:, None]) / sig
657
+ with np.errstate(over='ignore'):
658
+ term2 = np.exp(np.minimum(
659
+ 2.0 * m[:, None] * d[:, None] / s2[:, None] + log_ndtr(z2), 0.0))
660
+ return np.clip(norm.cdf(z1) + term2, 0.0, 1.0)
661
+
662
+ t_grid = np.arange(1, WINDOW_DAYS + 1, dtype=float)
663
+ Ft = F(stale[:, None] + t_grid[None, :])
664
+ F0 = np.where(stale > 0, F(np.maximum(stale, 1e-6)[:, None])[:, 0], 0.0)
665
+ cond = np.clip((Ft - F0[:, None]) / np.maximum(1.0 - F0, 1e-9)[:, None],
666
+ 0.0, 1.0)
667
+ p42 = np.where(at_barrier, 1.0, cond[:, -1])
668
+
669
+ out = self.days.predict(rows[FEATURES])
670
+ out['p42'] = p42
671
+ qcdf = cond / np.maximum(p42[:, None], 1e-12)
672
+ for q in DAY_QUANTILES:
673
+ idx = (qcdf >= q).argmax(axis=1)
674
+ hit = qcdf[:, -1] >= q
675
+ day = np.where(hit, t_grid[idx], WINDOW_DAYS).astype(float)
676
+ out[f'day_q{q}'] = np.where(at_barrier, 0.0, day)
677
+ return out
678
+
679
+
680
+ class LogWienerBlend:
681
+ """logbag p42 blended with the Wiener FHT p42."""
682
+
683
+ def __init__(self, mode='mean'):
684
+ self.mode = mode
685
+
686
+ def fit(self, train):
687
+ self.a = SklearnRUL('logbag').fit(train)
688
+ self.b = WienerFHT().fit(train)
689
+ return self
690
+
691
+ def predict(self, rows):
692
+ out = self.a.predict(rows)
693
+ pw = self.b.predict(rows)['p42']
694
+ if self.mode == 'mean':
695
+ out['p42'] = 0.5 * (out['p42'] + pw)
696
+ elif self.mode == 'stale':
697
+ stale = rows['staleness'].fillna(0).values
698
+ out['p42'] = np.where(stale > 14, np.maximum(out['p42'], pw), out['p42'])
699
+ return out
700
+
701
+
702
+ class ConformalCut:
703
+ """
704
+ Conformal-risk inclusion: wraps logbag and, at each refit, sets the score
705
+ cutoff to the risk_q quantile of the model's scores on *dying* calibration
706
+ pairs from the most recent closed origins. Scale-free by construction: the
707
+ cutoff moves with whatever scale the refit produces. mode 'split' holds the
708
+ calibration block out of training; 'in' trains on everything (slightly
709
+ optimistic scores, but twice the recency).
710
+ """
711
+
712
+ def __init__(self, risk_q=0.15, mode='in', calib_days=120):
713
+ self.risk_q = risk_q
714
+ self.mode = mode
715
+ self.calib_days = calib_days
716
+
717
+ def fit(self, train):
718
+ t_max = train['origin'].max()
719
+ calib = train[train['origin'] > t_max - pd.Timedelta(days=self.calib_days)]
720
+ fit_on = train if self.mode == 'in' else \
721
+ train[train['origin'] <= t_max - pd.Timedelta(days=self.calib_days)]
722
+ if len(fit_on) < 500 or fit_on['dies42'].sum() < 10:
723
+ fit_on = train
724
+ self.model = SklearnRUL('logbag').fit(fit_on)
725
+ dead = calib[calib['dies42'] == 1]
726
+ if len(dead) >= 8:
727
+ scores = self.model.predict(dead)['p42']
728
+ self.cut = float(np.quantile(scores, self.risk_q))
729
+ else:
730
+ self.cut = 0.5
731
+ return self
732
+
733
+ def predict(self, rows):
734
+ out = self.model.predict(rows)
735
+ out['p_cut'] = self.cut
736
+ return out
737
+
738
+
739
  MODELS = {
740
  'rank-v': RankVoltage,
741
  'logistic': lambda: SklearnRUL('logistic'),
 
755
  'logbag_aux': lambda: LogBagFeats(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60']),
756
  'logbag_aux2': lambda: LogBagFeats(['rng_30', 'rng_90', 'vstd_30']),
757
  'rank_rf': RankBlendRF,
758
+ 'silentfix015': lambda: SilentFix(0.15),
759
+ 'silentfix03': lambda: SilentFix(0.3),
760
+ 'silentfix06': lambda: SilentFix(0.6),
761
  'logbag_proj': lambda: LogBagFeats(['v_proj']),
762
  'logbag_rank': lambda: LogBagFeats(['r_vlast', 'r_tau']),
763
  'logbag_rankproj': lambda: LogBagFeats(['v_proj', 'r_vlast', 'r_tau']),
 
777
  'aux_ucb05': lambda: LogBagUCB(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60'], k=0.5),
778
  'aux_ucb10': lambda: LogBagUCB(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60'], k=1.0),
779
  'aux_log': lambda: LogBagLog(['rng_90', 'vstd_30', 'rng_ratio', 'gap_max_60']),
780
+ 'wiener': WienerFHT,
781
+ 'lw_mean': lambda: LogWienerBlend('mean'),
782
+ 'lw_stale': lambda: LogWienerBlend('stale'),
783
+ 'crt_in15': lambda: ConformalCut(0.15, 'in'),
784
+ 'crt_in25': lambda: ConformalCut(0.25, 'in'),
785
+ 'crt_sp15': lambda: ConformalCut(0.15, 'split'),
786
+ 'crt_in40': lambda: ConformalCut(0.40, 'in'),
787
+ 'crt_in50': lambda: ConformalCut(0.50, 'in'),
788
+ 'logbag_fht': lambda: LogBagFeats(['fht_days', 'wnr_m', 'wnr_s2']),
789
+ 'logbag_fht2': lambda: LogBagFeats(['fht_days']),
790
  }
791
 
792
 
 
1142
  """
1143
  psum = pred['p42_sum'].iloc[0] if 'p42_sum' in pred else pred['p42'].sum()
1144
  n = int(round(plan_tsroute.alpha * psum))
1145
+ order_col = 'rank_score' if 'rank_score' in pred.columns else 'p42'
1146
+ chosen = pred[order_col].nlargest(n).index
1147
  if plan_tsroute.p_be > 0:
1148
  topups = pred.index[pred['p42'] >= plan_tsroute.p_be]
1149
  chosen = chosen.union(topups)
 
1326
  plan_tspig2.alpha = 0.8
1327
  plan_tspig2.alpha2 = 1.3
1328
 
1329
+ def plan_tscut(pred, included, locations, start_time, day_q,
1330
+ travel_costs=None, settings=None):
1331
+ """tsroute scheduling, but inclusion = p42 >= the model's conformal cutoff."""
1332
+ chosen = pred.index[pred['p42'] >= pred['p_cut'].iloc[0]]
1333
+ if len(chosen) == 0:
1334
+ return assemble(pd.Series(dtype='datetime64[ns]'), locations, start_time)
1335
+ curves, _ = cost_curves(pred, locations, start_time)
1336
+ curves = pd.DataFrame(curves, index=pred.index).loc[chosen]
1337
+ bldg = locations.set_index('battery')['building'].reindex(chosen)
1338
+ room = locations.set_index('battery')['room']
1339
+ groups = {g: sorted(m.index, key=lambda b: room.get(b, ''))
1340
+ for g, m in curves.groupby(bldg)}
1341
+ travel_hours = travel_costs.set_index(['from', 'to'])['hours'].unstack()
1342
+ day_of = optimize_days(groups, curves, locations, travel_hours,
1343
+ settings, start_time)
1344
+ offs = bldg.map(day_of).astype(float)
1345
+ days = start_time + pd.to_timedelta(offs.values, unit='D')
1346
+ return assemble(pd.Series(days, index=offs.index), locations, start_time)
1347
+
1348
+
1349
  PLANNERS ={'onepd': plan_onepd, 'sched': plan_sched, 'batch': plan_batch,
1350
  'expcost': plan_expcost, 'topsum': plan_topsum,
1351
  'ecday': plan_ecday, 'ecroute': plan_ecroute, 'tsroute': plan_tsroute,
1352
+ 'tspig': plan_tspig, 'tspig2': plan_tspig2,
1353
+ 'tscut': plan_tscut}
1354
 
1355
 
1356
  # ------------------------------------------------------------------------ harness
 
1413
  optimize_days.iters = int(cfg.get('iters', 60))
1414
  included = pred.index[pred['p42'] >= cfg['p_star']]
1415
  kwargs = {}
1416
+ if cfg['planner'] in ('ecroute', 'tsroute', 'tspig', 'tspig2', 'tscut'):
1417
  kwargs = dict(travel_costs=ctx['travel_costs'],
1418
  settings=ctx['settings'])
1419
  plan = PLANNERS[cfg['planner']](pred, included, ctx['locs'],
 
1462
  # derived: voltage projected through the reporting gap by the device's own
1463
  # slope, and leak-free within-origin fleet ranks (data <= origin only)
1464
  table['v_proj'] = table['v_last'] + table['slope_lastk'] * table['staleness']
1465
+
1466
  table['r_vlast'] = table.groupby('origin_idx')['v_last'].rank(pct=True)
1467
  table['r_tau'] = table.groupby('origin_idx')['tau_projected'].rank(pct=True)
1468
  try:
1469
  from batteryswap_example.aux_features import load as load_aux
1470
+ aux = load_aux(dataset_path)
1471
+ dup = [c for c in aux.columns if c in table.columns
1472
+ and c not in ('device_id', 'origin_idx')]
1473
+ table = table.merge(aux.drop(columns=dup),
1474
+ on=['device_id', 'origin_idx'], how='left')
1475
+ except FileNotFoundError:
1476
+ pass
1477
+ try:
1478
+ from batteryswap_example.fht_features import load as load_fht
1479
+ table = table.merge(load_fht(dataset_path), on=['device_id', 'origin_idx'],
1480
  how='left')
1481
+ table['fht_days'] = ((table['v_corr_last'] - 2.40)
1482
+ / table['wnr_m'].clip(lower=1e-4)
1483
+ - table['staleness']).clip(-300, 600)
1484
  except FileNotFoundError:
1485
  pass
1486
  ctx = load_context(dataset_path)