Phase 15: swap-truncation/ghost/quarantine all rejected by panel; label 'noise' is the label process

EXPERIMENTS.md

@@ -482,6 +482,36 @@ New champion: **logbag / tsroute alpha=0.8 + break-even top-up (m0.8/pm25,
 fresh-only) + split_days=7 + censoring-aware two-sided cost curves**.
 Panel 1679.2 (from 1715.8 pre-phase-11).
 
+## Phase 15 - undocumented swaps & unlabeled deaths (all three fixes rejected; the "noise" is the label process)
+
+User finding (voltage_all_devices notebook): timeseries extend beyond battery
+lifetimes - unrecorded preventive swaps. Parallel forensics (3 agents)
+established the full scale: **31 devices with undocumented swaps** (32 events,
+clustering on building maintenance days - 2025-07-14 x8 and 2025-07-23 x7 in
+b_afdd8b1a066d, 2026-05-08 x6), detectable by a rule with a wide dead zone
+(consecutive-day corrected-voltage jump from <= 2.85 up to >= 3.00: identical
+33-event set for any jump threshold 0.10-0.25 V; real swaps jump >= 0.28,
+nearest noise landing 2.989). Beyond the 2 suspected unlabeled deaths: **26
+confirmed ghosts** (crossed 2.45, never recovered, stopped reporting, no EOL;
+dropout days cluster on batch-removal dates: 2025-12-07 x13, 2026-02-19 x10)
+plus 32 deaths-in-progress still limping below 2.45 at data end.
+
+Three CV-adjudicated treatments (panel, sd7 champion 1679.2):
+
+| treatment | panel | why it failed |
+|---|---|---|
+| A: truncate history at detected swap (age_days restarts, windows never span the swap) | 1684.5 (+5.3; ties except s_9 +101, s_14 +122, s_32 -43) | the age error only touches age_days (weak, linear); cohort features were already rejected; truncation shortens post-swap history and loses more than it fixes |
+| B: exclude ghosts from plans (p42=0 if below 2.45 > 42d, still < 2.55, no EOL) | **2226.0 (+547)** | WRONG PREMISE: EOL is the ~2.40 crossing and batteries spend >42d in the 2.45->2.40 twilight - the rule zeroed genuinely dying devices |
+| C: quarantine "looks dead, labeled survives" training rows (3,027 rows) | **2519.1 (+840)**, per-origin AUC collapses to ~0.51 | those negatives are LOAD-BEARING: they teach the model that deep-low voltage does not reliably produce a recorded death - which is exactly what the metric scores |
+
+Conclusion: what looks like label noise IS the label-generating process, and
+the incumbent model has already learned it (via staleness and the low-voltage
+negatives). The evaluator scores recorded deaths, not physical deaths; ghosts
+are unscoreable, and their "survives" labels are the correct training signal
+for that objective. detect_swaps() and the GHOST_ZERO/QUARANTINE knobs remain
+in the code, inert by default, with the production table and runtime paths
+unchanged. Champion unchanged: panel 1679.2, all-48 Docker 2146.9.
+
 ## Test-look ledger
 
 8 looks total (now historical - see protocol change above); look 9 = the

batteryswap_example/features.py

@@ -43,10 +43,37 @@ def slope(days: np.ndarray, values: np.ndarray) -> float:
     return float(np.polyfit(x.astype(float), values, 1)[0])
 
 
+def detect_swaps(dall: pd.DataFrame, beta: float) -> pd.Series:
+    """
+    Last UNDOCUMENTED battery-swap day per device, from daily corrected
+    voltage <= whatever the caller masked to (leak-free if dall is masked to
+    the origin). A swap = consecutive daily readings jumping up by >= 0.15 V
+    from a degraded level (<= 2.85) onto the fresh plateau (>= 3.00). The
+    landing gate is load-bearing: the detected set is identical for jump
+    thresholds 0.10-0.25 (real swaps jump >= 0.28 V, nearest noise landing
+    2.989), so the rule sits in a wide dead zone. Devices without a detected
+    swap are absent from the result.
+    """
+    d = dall.sort_values(['device_id', 'day'])
+    vc = d['v'] - beta * (d['t'] - 20.0)
+    same = d['device_id'].eq(d['device_id'].shift())
+    jump = (vc - vc.shift() >= 0.15) & (vc.shift() <= 2.85) & (vc >= 3.00) & same
+    return d.loc[jump].groupby('device_id')['day'].max()
+
+
 def features_at_origin(daily: pd.DataFrame, devices: pd.DataFrame,
-                       origin: pd.Timestamp) -> pd.DataFrame:
-    """Feature rows for every device with any reading at or before origin."""
+                       origin: pd.Timestamp, swaps: pd.Series = None) -> pd.DataFrame:
+    """Feature rows for every device with any reading at or before origin.
+
+    swaps: optional device -> last-swap-day map (detect_swaps). History before
+    a device's swap belongs to the PREVIOUS battery: it is dropped so slopes,
+    baselines and windows describe the battery actually in the slot, and
+    age_days restarts at the swap.
+    """
     h = daily[daily['day'] <= origin]
+    if swaps is not None and len(swaps):
+        cut = h['device_id'].map(swaps)
+        h = h[cut.isna() | (h['day'] >= cut)]
 
     g = h.groupby('device_id')
     last = g.tail(1).set_index('device_id')
@@ -86,7 +113,11 @@ def features_at_origin(daily: pd.DataFrame, devices: pd.DataFrame,
     feats['accel'] = feats['slope_30'] - feats['slope_90']
 
     install = devices.set_index('battery')['start_time'].dt.tz_localize(None)
-    feats['age_days'] = (origin - install.reindex(feats.index)).dt.days
+    eff_install = install.reindex(feats.index)
+    if swaps is not None and len(swaps):
+        sw = swaps.reindex(feats.index)
+        eff_install = eff_install.where(sw.isna() | (eff_install >= sw), sw)
+    feats['age_days'] = (origin - eff_install).dt.days
 
     return feats
 
@@ -141,7 +172,8 @@ def master_curve(daily: pd.DataFrame, dead_eol: pd.Series, beta: float,
 
 
 def physics_features(daily: pd.DataFrame, origin: pd.Timestamp,
-                     beta: float, curve, match: bool = True) -> pd.DataFrame:
+                     beta: float, curve, match: bool = True,
+                     swaps: pd.Series = None) -> pd.DataFrame:
     """
     Physics-model features.
       v_corr_last   - latest temp-corrected voltage (full history, no window)
@@ -153,12 +185,24 @@ def physics_features(daily: pd.DataFrame, origin: pd.Timestamp,
       tau_hat/fit_err - trajectory registration (match=True only; expensive)
     """
     all_h = daily[daily['day'] <= origin]
+    if swaps is not None and len(swaps):
+        cut = all_h['device_id'].map(swaps)
+        all_h = all_h[cut.isna() | (all_h['day'] >= cut)]
     last = all_h.sort_values('day').groupby('device_id').tail(1).set_index('device_id')
     v_corr = last['v'] - beta * (last['t'] - 20.0)
     stale = (origin - last['day']).dt.days
 
     out = pd.DataFrame({'v_corr_last': v_corr})
 
+    # ghost marker: days since the corrected voltage first crossed below 2.45
+    # (post-swap history only). A real recorded death follows the crossing
+    # within days; a crossing >42d old with no EOL on file means the recording
+    # process abandoned this device - it will never be scored as a death.
+    vc_all = all_h['v'] - beta * (all_h['t'] - 20.0)
+    below = all_h.loc[vc_all < 2.45]
+    first_below = below.groupby('device_id')['day'].min()
+    out['days_below_245'] = (origin - first_below.reindex(out.index)).dt.days
+
     # plunge precursors: volatility of the corrected voltage rises BEFORE the
     # level falls (AUC 0.83/0.81 on the voltage-silent dying cohort, where the
     # level itself is blind). These are the fast-plunge early-warning features.
@@ -241,13 +285,18 @@ def build(dataset_path: Path, out_path: Path) -> pd.DataFrame:
         alive = eol_times.isna() | (eol_times > origin)
         alive_ids = eol_times.index[alive]
 
+        # NOTE (phase 15): detect_swaps truncation was tested here and NOT
+        # adopted - panel 1684.5 vs 1679.2. The undocumented-swap age error
+        # touches only age_days (weak, linear); the machinery stays available
+        # via the swaps= parameter but production builds run without it.
+        past = betas[betas['week_end'] <= origin]['beta']
+        beta = float(past.median()) if len(past) >= 20 else BETA_DEFAULT
+
         f = features_at_origin(daily, locations, origin)
         f = f.reindex(alive_ids)  # NaN feature rows kept: planner must handle them
 
         # physics features: beta and curve from information <= origin only.
         # Synthetic origins get the light variant (no trajectory matching).
-        past = betas[betas['week_end'] <= origin]['beta']
-        beta = float(past.median()) if len(past) >= 20 else BETA_DEFAULT
         dead = eol_times[eol_times.notna() & (eol_times <= origin)]
         curve = master_curve(dall[dall['day'] <= origin], dead, beta)
         f = f.join(physics_features(dall, origin, beta, curve, match=not is_synth))

batteryswap_example/models_cv.py

@@ -183,9 +183,21 @@ class SklearnRUL:
         X = rows[FEATURES]
         out = self.days.predict(X)
         out['p42'] = self.clf.predict_proba(X)[:, 1]
+        if GHOST_ZERO and 'days_below_245' in rows.columns:
+            # a device below the 2.45 death threshold for >42d with no EOL on
+            # file will never be scored as a death (26 such ghosts in train) -
+            # swapping it is a pure false-positive cost. Require it to STILL
+            # be in the death zone: days_below_245 is sticky from the first
+            # crossing, and a dip-then-recovery must stay eligible.
+            ghost = ((rows['days_below_245'] > WINDOW_DAYS)
+                     & (rows['v_corr_last'] < 2.55)).values
+            out.loc[ghost, 'p42'] = 0.0
         return out
 
 
+GHOST_ZERO = False   # phase-15 candidate: exclude label-process ghosts from plans
+QUARANTINE = False   # phase-15 candidate: drop looks-dead-labeled-alive training rows
+
 SILENT_V = 2.55
 SILENT_FEATURES = ['rng_30', 'vstd_30', 'tau_projected', 'cohort_rel_age_m',
                    'age_days', 'v_corr_last']
@@ -1484,7 +1496,15 @@ PLANNERS ={'onepd': plan_onepd, 'sched': plan_sched, 'batch': plan_batch,
 
 def training_pairs(table, origin_time):
     closed = table['origin'] <= origin_time - pd.Timedelta(days=WINDOW_DAYS)
-    return table[closed]
+    sub = table[closed]
+    if QUARANTINE and 'v_corr_last' in sub.columns:
+        # drop "looks dead, labeled survives" rows: devices already below the
+        # death threshold whose death never made it into eol_times (26 ghost
+        # trajectories in train). Feeding them as negatives teaches the model
+        # that textbook plunges survive - label noise on the hardest pattern.
+        poison = (sub['dies42'] == 0) & (sub['v_corr_last'] < 2.48)
+        sub = sub[~poison]
+    return sub
 
 
 def censored_pairs(table, origin_time):