Phase 21-23: unmasked foreign pooling for held-out splits + Docker PATH fix

- submit.py: pool the packaged train table UNMASKED on held-out splits
(66,890 closed-window pairs, 82 dead devices from origin 0); the target
split's own fleet stays under the T-42d mask; byte-inert when the target
IS train (all-48 Docker reproduces 1851.0186805555559 exactly).
Holdout-validated (fleet A=packaged, B=target): warm-up 1108.6 vs floor's
1091.0 (tie) vs starved 1478.9; mid/late spots 1088.4 vs 1141.9. The
cold-start floor gate now never opens on held-out splits (by design);
it remains live for local warm-up regimes.
- Dockerfile: pin /app/env/bin onto PATH - newer base-image entrypoints
shadow the competition env with miniconda at runtime, breaking
`python3 script.py` with ModuleNotFoundError.
- models_cv.py: phase-21 comment (evaluator emergency queue truly opens at
end-of-week, day 48; exact pricing tested and rejected) + inert nn_order
visit-ordering machinery (phase 22, rejected: panel tie 1679.4 vs 1679.2).
- EXPERIMENTS.md: phases 21 (emergency-timing pricing rejected), 22
(sd<7 and NN visit order rejected; ops channel closed), 23 (pooling
change adopted with holdout evidence).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Dockerfile

@@ -19,5 +19,10 @@ RUN pip install --no-cache-dir -r requirements.txt
 COPY batteryswap_example/ ./batteryswap_example
 COPY script.py ./
 
+# The competition env (with the pinned packages) lives at /app/env; newer
+# base-image entrypoints put miniconda first on PATH at runtime, which shadows
+# it and breaks `python3 script.py` with ModuleNotFoundError. Pin it back.
+ENV PATH=/app/env/bin:${PATH}
+
 # Default to making submissions
 CMD python3 script.py

EXPERIMENTS.md

@@ -715,6 +715,138 @@ Champion: logbag / tsroute a0.8 + top-up (m0.8/pm25) + sd7 + censor-aware
 pricing + gated cold-start floor (k=2.5, gate 20). Panel 1679.2 (unchanged),
 warm-up 2380, near-end 604.8, all-48 Docker 1851.0, public 1717.8.
 
+## Phase 21 - forced-emergency timing: exact pricing tested, rejected
+
+Evaluator fact (established by reading evaluate_plan's emergency loop): every
+recorded in-window EOL the plan never swaps gets a FORCED emergency visit -
+late penalty + full unoptimized ops - and the queue does not start at day 42.
+It starts at the **end of the calendar week containing end_time**
+(`end_day = end_time + (6 - weekday)`), then processes one battery per day in
+alphabetical order. All 48 train scenarios start on Mondays, so the first
+emergency is always **day 48**, and the k-th miss lands at day 48+(k-1). Our
+cost model prices parking against `EMERGENCY_DAY = 45` - a systematic ~30
+underprice per expected death, plus an unmodeled queue-position term.
+
+This supersedes the phase-10 disclosure that `emergency_day` is "structurally
+inert": the phase-11 top-up and optimize_days' add/drop passes both consult
+`park`, so the knob is live now.
+
+| candidate | verdict | evidence |
+|---|---|---|
+| emergency_day=48 (exact) | rejected | panel **1682.8 vs 1679.2** (+3.6); 31/34 plans byte-identical - the repricing only flips decisions at near-data-end origins (s_39 -74, s_40 +67, s_41 +130); near-end block s_42-47 **631.9 vs 604.8** (+27), 0 wins / 4 losses / 2 ties |
+| emergency_day=50 / 52 (exact + queue pad) | rejected | +9.4 / +13.1, monotonically worse |
+
+Why the exact price loses: the deaths we park are mostly voltage-silent
+(p42 ~ 0.05), so +30 x p42 adds ~1.5 to their park price - never enough to
+include them. What the pricier park DOES flip are marginal p42 ~ 0.3-0.5
+devices that mostly survive: late improves -3.5 while early pays +6.6
+(panel decomposition). The champion's underpriced park is a de-facto
+regularizer against over-inclusion where FP prices collapse. Same conclusion
+as the phase-14 park_mobility rejection from the other side: emergency-cost
+underestimation is not the binding error - p42 on silent deaths is.
+
+## Phase 22 - optimizer-constraint challenges: sd<7 and NN visit order (both rejected)
+
+The evaluator's "constraints" are soft prices (weekly limit flat 100/blown
+week, daily flat 100, overtime 2x/h past 8h; ONE worker hard-wired in the
+state machine - no worker field exists in the plan schema) and mobility_cost
+replicates its cost machine exactly, so the ~152/origin the champion pays in
+penalties is a purchased trade (phase 18 already adversarially confirmed).
+The two remaining SELF-imposed constraints were challenged:
+
+| candidate | verdict | evidence |
+|---|---|---|
+| split_days=3 / 1 (finer group granularity) | rejected | panel **1685.0 / 1691.1 vs 1679.2**; ops rises 216.6 -> 223.7 -> 228.2 - finer groups fragment the local search's argmin start and it converges to worse optima with more travel; more freedom != better local search |
+| nn=True (visit a day's buildings in NN route order from base instead of name order, priced in the sim AND emitted identically) | rejected | panel **1679.4 vs 1679.2** - tie; 8 wins / 13 losses / 13 ties; per-origin moves up to +-40 are search-trajectory perturbations, not route savings (with a symmetric travel matrix 2-building days are order-invariant; 3+ building days are rare and route deltas are <1 each) |
+
+Machinery kept inert: optimize_days.nn_order (default False, cfg key 'nn'),
+assemble(sort_key=). Ops pot accounting: 217/origin total = 152 penalties
+(correctly purchased) + 54 travel/changes + 11 swap/room time; the recoverable
+slice was bounded by ~5-10/origin and the tests found ~0. The ops channel is
+CLOSED - remaining cost concentrates where it always was: late_swap (p42 on
+silent deaths).
+
+Provenance note: the first nn panel run was invalidated by a parallel-session
+file race (a peer harness's frozen-file guard checked models_cv.py out to HEAD
+mid-experiment; nn configs silently ran as no-ops - detected because knob
+configs came back byte-identical to baseline). Long background runs now print
+the imported module's md5 and assert the knob under test exists.
+
+## Phase 23 - worker-limit saturation campaign (all rejected; non-adaptation is optimal)
+
+The one regime local CV never samples: held-out fleets are ~4.3x train
+(phase 16), where fixed worker limits bind and the public leader pays
+~108/scenario in daily-limit penalties. Harness (`research/run_sat.py`,
+frozen): divide each scenario's worker limits (weekly/daily/overtime_start)
+by 4 on real panel origins - planner and evaluator share the transformed
+settings (the phase-11 property). Champion baseline on the block: **2115.1**
+(vs 1679.2 native) = late 1022 / early 446 / OT 112 / daily 262 / weekly 212 /
+travel 41 - so ~585/scenario of limit-driven penalties, with late/early
+unchanged from native: the champion does not adapt its plan to congestion.
+
+Autoresearch protocol (mutable surface = opt-knob JSON only): s0 = 6-origin
+saturated screen; s1 = all 34 saturated origins + native no-regression leg.
+10 s0 experiments + 1 s1, all rejected:
+
+| knob family | best variant | s0 delta | verdict |
+|---|---|---|---|
+| constructive start (phase 18) | on | +16.2 | worse basin under saturation too |
+| search budget | r6/i120 | -13.4 | one -80 escape in six origins; 5 ties |
+| split_days | 12 / 4 | -4.1 / +19.4 | both directions wash; limits untouched |
+| day_q | 0.2 | 0.0 | identical plans - dead knob under saturation |
+| top-up gate | pm12 | +116.3 | starving the top-up raises late AND limits |
+| alpha | 0.75 / 0.7 | **-59.8** (4/6) / -10.5 | promoted at 0.75+sd12 combo -> s1 |
+
+s1 (alpha 0.75 + sd12): sat **+20.0** (18/34 wins but 4 blow-ups sum +1835:
+s_17 +628, s_8 +459, s_31 +391, s_23 +357 vs 18 wins summing -1317), native
+**+42.0**. Decomposition: late +43.5 / early -27.0 / daily +5.9 / weekly -2.9 -
+inclusion cuts buy ZERO congestion relief; they are a mortality bet with
+fat-tailed downside, made worse by saturation (emergencies consume the same
+scarce worker-hours). Screen lesson: 6-origin s0 is high-variance for
+inclusion knobs (-59.8 flipped to +20.0 at s1); the s1 mean gate caught it.
+
+**Conclusion**: the ~585/scenario limit spend is the shadow price of
+scheduling MASS, not of bad choices - the day-assignment search is converged
+(extra search, day quantiles, both sd directions, constructive start: all
+flat or worse), and paying the flat fines beats the timing cost of leaving
+peak mortality weeks. The champion's non-adaptation to congestion is optimal
+within its plan space; the leader's limit penalties on the LB are likewise a
+purchased trade, not an exploitable gap. The saturated-regime channel is
+CLOSED; ec_margin was left untested as the same condemned inclusion family.
+
+## Phase 23 - unmasked foreign pooling in deployment (ADOPTED; user change, holdout-validated)
+
+User change to submit.py: the packaged train table is pooled UNMASKED on
+held-out splits (all 66,890 closed-window pairs, 82 dead devices, explicit
+window_closed filter), while the target split's own pairs - and any train-table
+rows for devices in the target fleet - stay under the T-42d mask. The old
+uniform mask was CV hygiene misapplied to deployment: it starved a held-out
+split's early origins to 3-20 unique training deaths (the cold-start
+mechanism). Byte-inert when the target split IS train (own-fleet guard), so
+panel / warm-up / all-48 are unchanged by construction.
+
+Validation (pooling-holdout harness /tmp/pool_holdout.py: fleet A = packaged
+table, fleet B = target, B-only scenarios scored):
+
+| regime | warm-up s_0-7 | spots s_16/24/32/40 |
+|---|---|---|
+| old masked pool, no floor | 1478.9 (late 1274) | 1141.9 |
+| old + gated k=2.5 floor (shipped before) | **1091.0** | 1141.9 (gate closed) |
+| **new unmasked pool (ships now)** | 1108.6 | **1088.4** |
+
+- Count calibration transfers: held-out s_0 sum(p42) 0.5 -> 3.4 vs 4 true
+  deaths. The starvation the phase-19 floor patched is gone at its source.
+- Warm-up vs the floor: statistical tie (5W/3L; s_4 -827 for the floor - a
+  local prevalence burst that own-fleet recent-death signals catch and
+  foreign calibration lags). Profiles differ: floor buys insurance early
+  (480 early), new stays lean (177 early / 786 late).
+- The undersold gain is MID/LATE held-out origins: -53.5 (3W/1L, s_16 -438) -
+  the unmasked pool improves every held-out origin's training set, and a real
+  split is mostly mid-history origins.
+- Deployment consequence: on held-out splits the floor gate NEVER opens now
+  (uniq >= 82 from origin 0); the floor remains live only for genuinely
+  history-poor local regimes (train warm-up block, all-48 metric).
+
 ## Test-look ledger
 
 8 looks total (now historical - see protocol change above); look 9 = the

batteryswap_example/models_cv.py

@@ -996,7 +996,7 @@ def resolve_model(name):
 
 # ------------------------------------------------------------------------- planners
 
-def assemble(selected_days, locations, start_time):
+def assemble(selected_days, locations, start_time, sort_key=None):
     parked_day = start_time + pd.Timedelta(days=WINDOW_DAYS + 1)
     scheduled = set(selected_days.index)
     parked = [b for b in locations['battery'] if b not in scheduled]
@@ -1005,10 +1005,15 @@ def assemble(selected_days, locations, start_time):
         pd.DataFrame({'day': parked_day, 'battery': parked}),
     ], ignore_index=True)
     # within a day the evaluator visits in row order: group by building then room,
-    # otherwise interleaved buildings pay repeated building-changes and travel
+    # otherwise interleaved buildings pay repeated building-changes and travel.
+    # sort_key overrides the building order (per-battery rank, e.g. NN route);
+    # parked rows fall past end_time and are cut by the evaluator, so their
+    # order is irrelevant.
     loc = locations.set_index('battery')
     plan['building'] = plan['battery'].map(loc['building'])
     plan['room'] = plan['battery'].map(loc['room'])
+    if sort_key is not None:
+        plan['building'] = plan['battery'].map(sort_key).fillna(10 ** 9)
     plan = (plan.sort_values(['day', 'building', 'room'], kind='stable')
                 .drop(columns=['building', 'room']).reset_index(drop=True))
     check_plan_valid(plan, locations, start_time=start_time)
@@ -1066,7 +1071,12 @@ def _batch_days(offs, locations, start_time, max_groups_per_day=2):
 
 
 EARLY, LATE = 0.5, 10.0
-EMERGENCY_DAY = WINDOW_DAYS + 3  # emergencies run just after the plan ends
+# The evaluator's true emergency queue opens at the END OF THE WEEK containing
+# end_time (day 48 for the all-Monday train scenarios), +1 day per queued miss.
+# Exact pricing (48/50/52) was tested and REJECTED (phase 21): it only flips
+# decisions at near-data-end origins and loses there - the underprice acts as
+# a regularizer against over-including marginal survivors. Keep 45.
+EMERGENCY_DAY = WINDOW_DAYS + 3
 CENSOR_AWARE = True     # price unobservable (post-data-end) deaths via imputation
 PARK_MOBILITY = 0.0     # extra cost per expected emergency (travel/overtime)
 
@@ -1258,6 +1268,19 @@ def mobility_cost(day_batteries, locations, travel_hours, settings, start_time):
 
 
 
+def _nn_building_order(bldgs, travel_hours, base):
+    """Nearest-neighbor route over a day's buildings starting from base:
+    {building: visit_rank}. Deterministic (ties broken by name)."""
+    left = sorted(set(bldgs))
+    order, cur = {}, base
+    for i in range(len(left)):
+        nxt = min(left, key=lambda b: (float(travel_hours.at[cur, b]), b))
+        order[nxt] = i
+        left.remove(nxt)
+        cur = nxt
+    return order
+
+
 def optimize_days(groups, curves, locations, travel_hours, settings, start_time,
                   restarts=None, iters=None):
     restarts = restarts if restarts is not None else optimize_days.restarts
@@ -1267,13 +1290,26 @@ def optimize_days(groups, curves, locations, travel_hours, settings, start_time,
     group onto another group's day), and deterministic random restarts.
     Objective = expected timing cost (model curves) + exact mobility cost.
     """
+    loc_b = locations.set_index('battery')['building']
+    gbldg = {g: loc_b.at[bs[0]] for g, bs in groups.items()}
+
     def timing(day_of):
         return sum(curves.at[b, day_of[g]] for g, bs in groups.items() for b in bs)
 
     def mobility(day_of):
-        day_batteries = {}
+        day_groups = {}
         for g in sorted(day_of, key=lambda g: (day_of[g], g)):
-            day_batteries.setdefault(day_of[g], []).extend(groups[g])
+            day_groups.setdefault(day_of[g], []).append(g)
+        if optimize_days.nn_order:
+            # visit a day's buildings in nearest-neighbor route order instead
+            # of name order (matters only on 3+ building days: with a
+            # symmetric travel matrix a 2-building day is order-invariant)
+            for gs in day_groups.values():
+                rank = _nn_building_order([gbldg[g] for g in gs], travel_hours,
+                                          settings.base_location)
+                gs.sort(key=lambda g: (rank[gbldg[g]], g))
+        day_batteries = {d: [b for g in gs for b in groups[g]]
+                         for d, gs in day_groups.items()}
         return mobility_cost(day_batteries, locations, travel_hours,
                              settings, start_time)
 
@@ -1332,6 +1368,7 @@ def optimize_days(groups, curves, locations, travel_hours, settings, start_time,
 optimize_days.restarts = 2
 optimize_days.iters = 60
 optimize_days.constructive = False
+optimize_days.nn_order = False   # visit a day's buildings in NN route order
 
 
 def plan_ecroute(pred, included, locations, start_time, day_q,
@@ -1448,7 +1485,19 @@ def plan_tsroute(pred, included, locations, start_time, day_q,
 
     offs = key.map(day_of).astype(float)
     days = start_time + pd.to_timedelta(offs.values, unit='D')
-    return assemble(pd.Series(days, index=offs.index), locations, start_time)
+    sort_key = None
+    if optimize_days.nn_order:
+        # emit rows in the same NN building order the mobility sim priced
+        base_b = settings.base_location
+        gb = {g: bldg.loc[bs[0]] for g, bs in groups.items()}
+        ranks = {}
+        for d in set(day_of.values()):
+            gs = [g for g in day_of if day_of[g] == d]
+            ranks[d] = _nn_building_order([gb[g] for g in gs], travel_hours, base_b)
+        sort_key = pd.Series({b: ranks[day_of[g]][gb[g]]
+                              for g, bs in groups.items() for b in bs})
+    return assemble(pd.Series(days, index=offs.index), locations, start_time,
+                    sort_key=sort_key)
 
 
 plan_tsroute.alpha = 0.8
@@ -1739,6 +1788,7 @@ def walk_forward(table, scenario_ctx, origins, configs):
             optimize_days.restarts = int(cfg.get('restarts', 2))
             optimize_days.iters = int(cfg.get('iters', 60))
             optimize_days.constructive = bool(cfg.get('constructive', False))
+            optimize_days.nn_order = bool(cfg.get('nn', False))
             included = pred.index[pred['p42'] >= cfg['p_star']]
             kwargs = {}
             if cfg['planner'] in ('ecroute', 'tsroute', 'tspig', 'tspig2', 'tscut'):
@@ -1777,7 +1827,8 @@ def summarize(df, label):
                      ('alpha2', 'a2'), ('emergency_day', 'e'), ('restarts', 'r'),
                      ('iters', 'i'), ('ec_margin', 'ec'), ('ec_pmax', 'pm'),
                      ('split_days', 'sd'), ('park_mobility', 'pk'), ('floor', 'fl'),
-                     ('floor_gate', 'fg'), ('floor_zone', 'fz')]:
+                     ('floor_gate', 'fg'), ('floor_zone', 'fz'), ('constructive', 'cn'),
+                     ('nn', 'nn')]:
         if key in df.columns and df[key].nunique(dropna=False) > 1:
             df['name'] += '/' + tag + df[key].fillna(0).astype(str)
     mean = df.groupby('name')['total_cost'].agg(['mean', 'std']).sort_values('mean')
@@ -1820,7 +1871,9 @@ def selection_stability(df, n_random=2000, n_blocks=6, seed=0):
     for key, tag in [('blend', 'b'), ('p_be', 'be'), ('floor_day', 'f'), ('p_conf', 'c'),
                      ('alpha2', 'a2'), ('emergency_day', 'e'), ('restarts', 'r'),
                      ('iters', 'i'), ('ec_margin', 'ec'), ('ec_pmax', 'pm'),
-                     ('split_days', 'sd'), ('park_mobility', 'pk'), ('floor', 'fl')]:
+                     ('split_days', 'sd'), ('park_mobility', 'pk'), ('floor', 'fl'),
+                     ('floor_gate', 'fg'), ('floor_zone', 'fz'), ('constructive', 'cn'),
+                     ('nn', 'nn')]:
         if key in df.columns and df[key].nunique(dropna=False) > 1:
             df['name'] += '/' + tag + df[key].fillna(0).astype(str)
     pv = df.pivot_table(index='origin', columns='name', values='total_cost')

batteryswap_example/submit.py

@@ -8,10 +8,14 @@ static train-split model cannot avoid.
 
 Label integrity: the runtime dataset ships each split's eol_times.csv, which
 contains ground truth beyond the scenario origins. This module NEVER uses it
-directly for planning. Training pairs are restricted to origins whose full 42-day
-window CLOSED before the scenario origin, so every label used was observable at
-planning time; an assertion enforces it. (This mirrors iterate_scenarios' own
-legitimate use of eol_times to drop already-dead devices.)
+directly for planning. The split's OWN training pairs are restricted to origins
+whose full 42-day window CLOSED before the scenario origin, so every label used
+was observable at planning time; an assertion enforces it. (This mirrors
+iterate_scenarios' own legitimate use of eol_times to drop already-dead devices.)
+Pairs from the shipped TRAIN table are a different fleet whose labels were all
+known when the submission was built - those are pooled UNMASKED (closed windows
+only), except for devices that also belong to the target split's fleet, which
+stay under the time mask so a train-split run cannot read its own future.
 
 Falls back to the static pickled planner per split if anything goes wrong.
 """
@@ -105,12 +109,24 @@ def adaptive_split_plans(split, split_path, log=print, deadline=None):
     table, daily, dall, betas = build_split_table(
         locations, timeseries, eol_times, scen_starts, log=log)
 
+    table['foreign'] = False
     pooled = [table]
     if TRAIN_TABLE.exists():
         base = pd.read_parquet(TRAIN_TABLE)
-        pooled.append(base[np.intersect1d(base.columns, table.columns)])
-        log(f'  pooled with train table: +{len(base)} pairs')
+        # open-window rows (origins within 42d of the train data end) carry
+        # unverifiable "survives" labels; the old T-mask excluded them only by
+        # coincidence, so filter explicitly - and BEFORE the column intersect,
+        # which drops window_closed (build_split_table does not produce it)
+        base = base[base['window_closed']]
+        base = base[np.intersect1d(base.columns, table.columns)].copy()
+        base['foreign'] = True
+        pooled.append(base)
+        log(f'  pooled with train table: +{len(base)} closed-window pairs')
     pool = pd.concat(pooled, ignore_index=True)
+    # a train-table row for a device in THIS split's fleet is not foreign -
+    # same fleet means same answer key - so it stays under the time mask.
+    # Makes the unmasked pooling inert when the target split IS train.
+    pool['foreign'] &= ~pool['device_id'].isin(set(locations['battery']))
 
     plans = []
     fitted = {}
@@ -124,9 +140,15 @@ def adaptive_split_plans(split, split_path, log=print, deadline=None):
             raise TimeoutError('split deadline exceeded mid-adaptive')
 
         if T not in fitted:
-            # LABEL INTEGRITY: only pairs whose window closed before T
-            train = pool[pool['origin'] <= T - pd.Timedelta(days=WINDOW_DAYS)]
-            assert (train['origin'] + pd.Timedelta(days=WINDOW_DAYS) <= T).all()
+            # LABEL INTEGRITY: own pairs only where the window closed before T
+            # (the split's eol_times is its answer key). Foreign train-fleet
+            # pairs are used in full: masking them was CV hygiene misapplied
+            # to deployment, starving early origins to 3-20 unique training
+            # deaths out of the 82 available (the cold-start mechanism).
+            closed = pool['origin'] <= T - pd.Timedelta(days=WINDOW_DAYS)
+            train = pool[closed | pool['foreign']]
+            own_pairs = train[~train['foreign']]
+            assert (own_pairs['origin'] + pd.Timedelta(days=WINDOW_DAYS) <= T).all()
             pos_dev = train.loc[train['dies42'] == 1, 'device_id']
             uniq = int(pos_dev.nunique())
             # foreign-fleet fraction: share of training positives that belong