Phase 52: building-grouped CV adopted - screen gains leave-building-out column; contamination gradient rf +.194 (scenario) -> ~0 (building); logbag choice confirmed, no reversal

EXPERIMENTS.md

@@ -2644,3 +2644,46 @@ catch. It caught ZERO. The local -37.9 was wobbler catches that the public
 fleet does not offer below the cut; a guard cannot create catches that are
 not there. Cert-first family closed entirely - the label-lag population
 either does not exist on the scored fleets or is already inside the head.
+
+## Phase 52 - building-grouped CV adopted; the contamination gradient quantified
+
+Trigger: a sweep of every public competitor artifact (HF repos, hidden HF
+branches, GitHub repos, and deleted-HF-repo cards recovered via the
+686f6c61/radar-huggingface-dataset archive) established that the scored
+public/private fleets hold FOREIGN BUILDINGS: the Attention-Heads card states
+it outright, and the FAQ transcription in ziadnasif77-debug's repo puts the
+full pool at ~200 buildings vs train's 24. Both serious hidden teams
+(YassY-The-AlchemYst's RobustHazardPlanner lineage, ziad) validate and
+calibrate building-grouped. Organizer-shaped conclusion, user directive:
+the trusted offline fold must be leave-BUILDING-out.
+
+Change (cv.py): `screen` gains a third grouping column, whole buildings
+assigned to 5 folds greedily balanced by unique observed deaths (deaths are
+lumpy - the top building holds 15/82, nine hold exactly one; random building
+assignment starves folds). `static`/`ablate` already split by building
+(fleet_halves), so this closes the last battery-grouped selection surface.
+Selection guidance in the module now reads: building > battery > scenario.
+
+Result, 3 fold-assignment seeds - rec12 margin over logistic (AUC margins
+agree in sign; levels are NOT comparable across columns, margins are the
+decision quantity):
+
+| model | scenario | battery (s0/s1/s2) | building (s0/s1/s2) |
+|---|---|---|---|
+| logbag | +.003 | -.006 / -.004 / +.001 | -.002 / +.000 / +.000 |
+| rf | **+.194** | +.023 / +.031 / -.008 | +.005 / -.005 / -.014 |
+| gbm | **+.148** | -.000 / -.016 / -.026 | -.022 / -.039 / -.030 |
+
+The contamination gradient is monotone: rf's +.194 phantom margin on
+scenario folds collapses to ~+.02 on battery folds and to zero-or-negative
+on building folds; gbm goes outright negative. The residual rf margin in the
+battery column (sign-unstable across seeds) is sibling leakage - a held-out
+device's building/batch mates stay in train. logbag == logistic (margin ~0)
+in every column and every seed: the champion's model choice never rested on
+fold luck, and the deployment-shaped fold CONFIRMS it rather than reversing
+anything. No selection changes; the protocol does.
+
+Standing rule upgrade (supersedes the blind-loop "group by battery" rule):
+**group by BUILDING when the deployment is cross-building.** Battery
+grouping is necessary but not sufficient; it still leaks the environment/
+install-batch channel that phase 10/13 closed on the feature side.

batteryswap_example/cv.py

@@ -12,15 +12,19 @@ real leaderboard points to learn.
    median rank of missed deaths beside the cost. Implemented in
    `models_cv.rank_stats`, so `dev`/`panel`/`test` get it too.
 
-**2. Group by BATTERY, not by scenario or row.**
+**2. Group by BUILDING > battery > scenario.**
    Each battery contributes one near-duplicate row per closed origin (~3600
    rows from ~460 batteries), so holding out a SCENARIO leaves the same
    batteries in train and test and tree models score their own memorised
    signatures. Measured in the blind loop: leave-scenario-out put HGB ahead of
    linear (AUC .9907 vs .9843); leave-battery-out REVERSED it (.9769 vs .9845)
    and the cost harness agreed with the clean split. The effective sample size
-   is ~460, not ~3600. `screen` runs both groupings side by side so the gap
-   stays visible.
+   is ~460, not ~3600. And battery grouping is still not the deployment split:
+   the scored public/private fleets hold FOREIGN BUILDINGS (organizer design;
+   also how `static` splits), and siblings in one building share install batch
+   and thermal environment, so a battery fold keeps a held-out device's
+   building-mates in train. `screen` runs all three groupings side by side;
+   selection follows the BUILDING column (2026-08-22 protocol change).
 
 **3. The deployment gate: would this fire in the static branch?**
    The scored splits are fleets the model has never seen; on them `submit.py`
@@ -205,14 +209,44 @@ def run_static(variant='base', seed=0, origins=range(48),
 
 
 # --------------------------------------------------------------------------
-# protocol 3: offline model screen, grouped by battery
+# protocol 3: offline model screen, grouped by building > battery
 # --------------------------------------------------------------------------
 
+def building_folds(closed, dev_bldg, n_folds=5, seed=0):
+    """Assign whole BUILDINGS to folds, balancing observed deaths.
+
+    Deaths are lumpy across the 24 buildings (top building holds 15/82, nine
+    hold exactly one), so a random building assignment routinely starves a
+    fold of positives. Greedy balance instead: buildings in descending
+    unique-death order, each to the death-lightest fold so far (ties broken
+    by device load, then by the seed's shuffle).
+    """
+    per_dev = (closed[['device_id', 'dies42']]
+               .assign(bldg=closed['device_id'].map(dev_bldg).values)
+               .groupby('device_id').agg(bldg=('bldg', 'first'),
+                                         died=('dies42', 'max')))
+    per_b = per_dev.groupby('bldg').agg(devices=('died', 'size'),
+                                        deaths=('died', 'sum'))
+    order = (per_b.sample(frac=1, random_state=np.random.RandomState(seed))
+             .sort_values(['deaths', 'devices'], ascending=False, kind='stable'))
+    deaths_in, devs_in = np.zeros(n_folds), np.zeros(n_folds)
+    fold_of = {}
+    for b, row in order.iterrows():
+        f = int(np.lexsort((devs_in, deaths_in))[0])
+        fold_of[b] = f
+        deaths_in[f] += row['deaths']
+        devs_in[f] += row['devices']
+    return pd.Series(fold_of, name='fold')
+
+
 def screen(kinds=('logistic', 'logbag', 'rf', 'gbm'), n_folds=5, seed=0,
            dataset_path=Path('dataset')):
-    """Leave-BATTERY-out vs leave-SCENARIO-out on the same pairs and the same
-    models. The gap between the two columns is the contamination; trust the
-    battery column. Headline is recall at the operating N, AUC secondary."""
+    """Leave-BUILDING-out vs leave-BATTERY-out vs leave-SCENARIO-out on the
+    same pairs and the same models. The gaps between columns are contamination;
+    trust the BUILDING column - the scored public/private fleets hold foreign
+    buildings, and building-mates share install batch and thermal environment,
+    so battery folds still leak sibling identity. Headline is recall at the
+    operating N, AUC secondary."""
     from sklearn.metrics import roc_auc_score
 
     t = load_table(dataset_path)
@@ -223,14 +257,20 @@ def screen(kinds=('logistic', 'logbag', 'rf', 'gbm'), n_folds=5, seed=0,
     dev_fold = pd.Series(rng.permutation(len(devs)) % n_folds, index=devs)
     org = np.sort(closed['origin_idx'].unique())
     org_fold = pd.Series(np.arange(len(org)) % n_folds, index=org)
+    dev_bldg = (pd.read_csv(dataset_path / 'train' / 'devices.csv')
+                .drop_duplicates('device_id')
+                .set_index('device_id')['building_id'])
+    bldg_fold = building_folds(closed, dev_bldg, n_folds, seed)
 
     n_dev_pos = closed.loc[closed['dies42'] == 1, 'device_id'].nunique()
     print(f'pairs {len(closed)}  batteries {len(devs)}  origins {len(org)}  '
+          f'buildings {bldg_fold.size}  '
           f'positives {int(closed["dies42"].sum())} rows / {n_dev_pos} batteries')
     print('effective n is the BATTERY count, not the row count - regularisation '
           'tuned against rows is ~8x too weak here.\n')
 
-    schemes = {'battery': closed['device_id'].map(dev_fold),
+    schemes = {'building': closed['device_id'].map(dev_bldg).map(bldg_fold),
+               'battery': closed['device_id'].map(dev_fold),
                'scenario': closed['origin_idx'].map(org_fold)}
     out = {}
     for scheme, fold in schemes.items():
@@ -279,9 +319,11 @@ def screen(kinds=('logistic', 'logbag', 'rf', 'gbm'), n_folds=5, seed=0,
             a0, r0, _ = out[(s, ref)]
             cells += [f'{a - a0:+16.4f}', f'{r - r0:+16.3f}']
         print(f'{kind:10s} ' + ' '.join(cells))
-    print('\nA model whose margin shrinks or reverses when the grouping goes '
-          'from scenario to battery was reading memorised battery identity. '
-          'Selection follows the battery column.')
+    print('\nA model whose margin shrinks or reverses as the grouping goes '
+          'scenario -> battery -> building was reading memorised identity at '
+          'that level (battery signatures, then building/batch siblings). '
+          'Selection follows the BUILDING column; it is the only one shaped '
+          'like the scored splits, which hold foreign buildings.')
     return out