Thibaut Forest Claude Opus 5 (1M context) commited on
Commit
86f3f93
·
1 Parent(s): 698401a

Phase 56 addendum: public refutes UNANCHORED frontier (1877.08, swaps 517 vs 580); fix = value-anchor to plain logbag

Browse files

Count collapse, not ordering: logbag_rank's sum(p42) is 0.78-0.80x plain
logbag under deployment past-only fits at late origins (both-sides purge
fits hid it - swaps read flat in CV). FrontierRUL now anchors p42 VALUES to
a plain-logbag donor (count/pricing/timing = the measured 580-swap champion
machinery, byte-equivalent) while rank+gbm supply ONLY the order. Verified
at worst-collapse origins s_35/s_45: donor sums exact, frontier order exact,
top-13/25 identical. BATTERYSWAP_FRONTIER_ANCHOR=0 restores the refuted
unanchored form for forensics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Files changed (2) hide show
  1. EXPERIMENTS.md +31 -0
  2. batteryswap_example/submit.py +22 -10
EXPERIMENTS.md CHANGED
@@ -3048,3 +3048,34 @@ instrument); top-12 slot squatters are label-process ghosts that still die at
3048
  real-but-subsumed; label windows d35/d21 count-starve (+72/+105); building-
3049
  balanced +40; thinning +19..+32; past-only pool +19 (future-side purge pairs
3050
  stabilize fits). Candidate CSVs: /tmp/cv_oofcand_*.csv (~180 full-48 files).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3048
  real-but-subsumed; label windows d35/d21 count-starve (+72/+105); building-
3049
  balanced +40; thinning +19..+32; past-only pool +19 (future-side purge pairs
3050
  stabilize fits). Candidate CSVs: /tmp/cv_oofcand_*.csv (~180 full-48 files).
3051
+
3052
+ ### Phase 56 addendum - PUBLIC REFUTES the unanchored frontier (698401a: 1877.08); mechanism found and fixed same-day
3053
+
3054
+ | | swaps | late-days | early-days | total |
3055
+ |---|---|---|---|---|
3056
+ | champion 3c7b883 | 580 | 5819 | 13197 | 1661.45 |
3057
+ | frontier 698401a | 517 | 7076 | 8011 | 1877.08 |
3058
+ | delta | **-63** | **+1257** | -5186 | +215.63 |
3059
+
3060
+ The failure is a COUNT collapse, not (necessarily) an ordering failure: the
3061
+ value-preserving blend preserves the PRIMARY's (logbag_rank's) p42 values,
3062
+ and under deployment's PAST-ONLY fits that model's calibration collapses as
3063
+ the pool grows - measured sum(p42) ratio vs plain logbag: 1.02 (s_5), 1.01
3064
+ (s_15), 0.95 (s_25), 0.80 (s_35), 0.78 (s_45). Public scenarios live in the
3065
+ late-origin regime -> -11% swaps -> ~+5 extra misses/48. The purge CV could
3066
+ not see it: its fits train on BOTH sides of the window, where the two models
3067
+ calibrate identically (swaps 9.23 vs 9.21). Fourth public refutation of a
3068
+ locally-validated intervention (alpha, cliff, cert, unanchored frontier) -
3069
+ but the first with the failing channel measured and separable.
3070
+
3071
+ FIX (same commit): FrontierRUL now VALUE-ANCHORS to plain logbag
3072
+ (BATTERYSWAP_FRONTIER_ANCHOR=1 default): the donor logbag supplies p42
3073
+ values - count/pricing/timing byte-equivalent to the old champion's measured
3074
+ 580-swap calibration - and the rank+gbm frontier supplies ONLY the order.
3075
+ Ordering-only in deployment by construction, not just in CV. Verified at
3076
+ s_35/s_45 (worst collapse origins): anchored sum(p42) == plain logbag
3077
+ exactly; order == frontier blend exactly (top-13/top-25 sets identical;
3078
+ tie-aware rank comparison zero mismatches). What the anchored head tests
3079
+ publicly, cleanly for the first time: is the frontier ORDER better on the
3080
+ scored fleet? Downside is bounded to ordering (count machinery is the
3081
+ measured champion's).
batteryswap_example/submit.py CHANGED
@@ -107,6 +107,16 @@ MODEL_KIND = 'logbag'
107
  # Confirmed local optimum by 5 lanes (hyperparams, features, weights, neural
108
  # rankers, physics blends all subsumed or worse head-to-head).
109
  FRONTIER = os.environ.get('BATTERYSWAP_FRONTIER', '1') != '0'
 
 
 
 
 
 
 
 
 
 
110
  # Blend weight dial for public A/B: 0.30 = full frontier (purge 1630.3);
111
  # 0 = plain logbag_rank, no gbm (purge 1746.4 - the clean linear component,
112
  # fallback if the gbm increment turns out to be purge flattery on public);
@@ -132,6 +142,8 @@ class FrontierRUL:
132
  t['r_tau'] = t.groupby('origin')['tau_projected'].rank(pct=True)
133
  self.primary = LogBagFeats(['r_vlast', 'r_tau']).fit(t)
134
  self.partner = SklearnRUL('gbm').fit(t)
 
 
135
  return self
136
 
137
  def predict(self, rows):
@@ -141,17 +153,17 @@ class FrontierRUL:
141
  r['r_vlast'] = r['v_last'].rank(pct=True)
142
  r['r_tau'] = r['tau_projected'].rank(pct=True)
143
  pred = self.primary.predict(r)
144
- if FRONTIER_W2 <= 0:
145
- return pred
146
- p2 = self.partner.predict(r)
147
- rk = ((1 - FRONTIER_W2) * pred['p42'].rank(pct=True)
148
- + FRONTIER_W2 * p2['p42'].rank(pct=True))
149
- vals = np.sort(pred['p42'].values)[::-1]
150
- pred = pred.copy()
151
- pred['p42'] = pd.Series(
152
  vals, index=rk.sort_values(ascending=False).index
153
- ).reindex(pred.index)
154
- return pred
155
 
156
 
157
  def _fit_model(train):
 
107
  # Confirmed local optimum by 5 lanes (hyperparams, features, weights, neural
108
  # rankers, physics blends all subsumed or worse head-to-head).
109
  FRONTIER = os.environ.get('BATTERYSWAP_FRONTIER', '1') != '0'
110
+ # PUBLIC REFUTATION of the UNANCHORED frontier (698401a: 1877.08 vs 1661.45,
111
+ # swaps 517 vs 580): under deployment's past-only fits, logbag_rank's
112
+ # sum(p42) collapses vs plain logbag as the pool grows (measured ratio 1.02
113
+ # at s_5 -> 0.78 at s_45; the purge CV's both-sides fits hid it - swaps read
114
+ # flat there). The ordering signal was never publicly tested; the count
115
+ # channel drowned it. ANCHOR fixes this by construction: plain logbag is the
116
+ # VALUE DONOR (count/pricing/timing = the measured-580-swap calibration,
117
+ # byte-identical to the old champion's machinery) and the frontier supplies
118
+ # ONLY the rank order. Ordering-only in deployment, not just in CV.
119
+ FRONTIER_ANCHOR = os.environ.get('BATTERYSWAP_FRONTIER_ANCHOR', '1') != '0'
120
  # Blend weight dial for public A/B: 0.30 = full frontier (purge 1630.3);
121
  # 0 = plain logbag_rank, no gbm (purge 1746.4 - the clean linear component,
122
  # fallback if the gbm increment turns out to be purge flattery on public);
 
142
  t['r_tau'] = t.groupby('origin')['tau_projected'].rank(pct=True)
143
  self.primary = LogBagFeats(['r_vlast', 'r_tau']).fit(t)
144
  self.partner = SklearnRUL('gbm').fit(t)
145
+ # value donor: the old champion's calibration owns the swap count
146
+ self.donor = SklearnRUL(MODEL_KIND).fit(train) if FRONTIER_ANCHOR else None
147
  return self
148
 
149
  def predict(self, rows):
 
153
  r['r_vlast'] = r['v_last'].rank(pct=True)
154
  r['r_tau'] = r['tau_projected'].rank(pct=True)
155
  pred = self.primary.predict(r)
156
+ rk = pred['p42'].rank(pct=True)
157
+ if FRONTIER_W2 > 0:
158
+ p2 = self.partner.predict(r)
159
+ rk = (1 - FRONTIER_W2) * rk + FRONTIER_W2 * p2['p42'].rank(pct=True)
160
+ base = self.donor.predict(rows) if self.donor is not None else pred
161
+ vals = np.sort(base['p42'].values)[::-1]
162
+ out = base.copy()
163
+ out['p42'] = pd.Series(
164
  vals, index=rk.sort_values(ascending=False).index
165
+ ).reindex(out.index)
166
+ return out
167
 
168
 
169
  def _fit_model(train):