phase 70: fleet-composition probe - CV packing win is a low-pressure artifact; selection locks to 4c93883

EXPERIMENTS.md

@@ -3570,3 +3570,29 @@ FINAL SELECTION (updated): private pick 4c93883 (measured 1575.31,
 best public we own) FIRST; 60d65da/71bfff2 (measured 1583.11, CV
 favorite at 1541.3) as the second slot / CV-side hedge if two picks
 exist. Never e4745d2 (1594.89) or 698401a (1877.08).
+
+## Phase 70 - Fleet-composition probe: the CV's lam2sp3 win is a LOW-PRESSURE artifact - pick locks to 4c93883
+
+Post-refutation diagnostic (report agent_reports_bgkf/t5_fleetcomp.md,
+raw /tmp/c35): A = tw2.0/sd3 (60d65da) vs B = tw1.0/sd7 (4c93883),
+identical model, paired, count-flat in all 1536 stage-2 cells.
+- Fold fleets: A wins 5/5 but pays an ops tax in 4/5 - the CV win is
+  entirely late/early timing on small sub-fleets.
+- 32 random subset fleets (3-12 bldgs): 20W/8L/4T, median -0.14; the
+  dials only ACTIVATE under pressure - P(cell differs) 5.5%->85% from
+  <50 to 300-400 devices, corr(delta, n_dev) -0.71.
+- PRESSURE fleets (18 bldgs, 250-384 devices): 3/6 flip to A-loses; the
+  two big losses replicate the public signature almost exactly (+7.90
+  with dOps +10.82; +9.31 with dOps +9.35; public +7.8 with ops +9.45,
+  timing slightly better in all three).
+- CAPSTONE: the full 24-building universe planned as ONE fleet flips A
+  to +0.97 (24W/20L, dLate -4.4, dOps +6.47). The CV's 2.9sig win exists
+  only because bgkf test fleets are five small sub-fleets where
+  daily/weekly limits never bind.
+VERDICT: the -4.1 CV delta is not transferable to large fleets; at
+public-like pressure A-vs-B is a coin flip locally and A measured worse
+publicly. INSTRUMENT FIX (for future work): adjudicate planner/packing
+dials on pressure-matched fleets (whole-universe-as-one-fleet + stress
+fleets at 250-400 devices), not the pooled small-sub-fleet mean.
+FINAL SELECTION locked: 4c93883 (measured 1575.31). 60d65da/71bfff2
+only as a second slot. Never e4745d2 / 698401a.

agent_reports_bgkf/t5_fleetcomp.md

@@ -0,0 +1,128 @@
+# t5_fleetcomp — FLEET-COMPOSITION probe of the A-vs-B planner-dial trade
+
+**Question.** Config A (timing_weight=2.0, split_days=3; lam2sp3 head) vs Config B
+(timing_weight=1.0, split_days=7; old champion 4c93883). Same model, same counts,
+same ordering — only the two packing dials differ. CV says A wins −4.1±1.4;
+public says A loses +7.8 (swaps identical, late/early −1.6 better, ops +9.45
+worse). Is the CV win composition-robust?
+
+**Verdict: A's win is composition-fragile — the public +7.8 is unsurprising.**
+A's CV edge is earned in a low-pressure regime (five ~5-building fold fleets
+where daily/weekly limits almost never bind). Under public-like fleet pressure
+the A−B trade becomes an ops-limit lottery with fleet-level swings of ±10–25,
+and 3 of 7 high-pressure synthetic fleets — including the monolithic
+24-building universe itself — reproduce the public loss with the same
+component signature (timing slightly better, ops ~+7..+11 worse).
+
+Method: reused cv.run_bgkf machinery verbatim (same table build, complement
+fit with the deployed submit.Ft2RUL, deployment-exact planner state, real
+evaluator, floor gate as in run_bgkf); one fit per fleet shared across arms,
+paired per fleet×origin. Runner and raw CSVs in /tmp/c35/. Module hashes
+(cv.py 6b3460e9…, models_cv.py ea340538…) verified unchanged before and after.
+
+## Stage 1 — the existing CV comparison, split by fold fleet (48 origins each)
+
+Canonical pair: /tmp/cv_bgkfcand_ft2a10.csv (A) vs /tmp/cv_bgkfcand_ft2a10_sp7.csv
+(B); reproduces the known ALL delta −4.11±1.41, 25W/4L exactly. (Note:
+cv_bgkfcand_head.csv at 19:25 is a stale model state — 42 cells differ from
+ft2a10.csv; do not pair against it.)
+
+| fold | dTotal | W/L/T (48) | dLate | dEarly | dOps |
+|---|---|---|---|---|---|
+| 0 | −0.60 | 8/2/38 | −0.42 | −0.45 | +0.14 |
+| 1 | −0.74 | 12/3/33 | −0.62 | −0.36 | +0.11 |
+| 2 | −0.95 | 3/0/45 | −0.83 | +0.04 | −0.16 |
+| 3 | −0.02 | 1/1/46 | 0.00 | −0.07 | +0.04 |
+| 4 | −1.80 | 12/8/28 | −1.67 | −0.36 | +0.13 |
+
+No fold-level sign flip, but only 50/240 cells are nonzero, and **A already
+pays a small ops tax in 4 of 5 fold fleets** (ops-worse on 46% of origins) —
+the entire CV win is late/early. The public signature (ops-dominated) is the
+same trade at ~40x the ops magnitude.
+
+## Stage 2 — 32 random building subsets as standalone fleets (sizes 3/5/8/12, all 48 origins, paired)
+
+1536 paired cells; 432 nonzero; 157 A-loss cells; **swap counts identical in
+every single cell** (the dials are count-flat everywhere). Fleet-level
+(mean over 48 origins): **A wins 20 / loses 8 / ties 4**; mean −2.10,
+median −0.14; all 8 losses tiny (max +0.95).
+
+Pressure activates the trade — P(cell differs at all) and mean delta by fleet
+device count:
+
+| fleet devices | cells | P(nonzero) | P(A-loss) | mean dt |
+|---|---|---|---|---|
+| 0–50 | 617 | 5.5% | 1.3% | −0.04 |
+| 50–100 | 180 | 19% | 6.7% | −0.35 |
+| 100–200 | 376 | 24% | 9.3% | −0.62 |
+| 200–300 | 310 | 74% | 28% | −7.01 |
+| 300–400 | 53 | 85% | 30% | −13.7 |
+
+Composition predictors of the fleet-level delta: corr(dt, n_dev) = −0.71,
+n_bldg −0.64, max building size −0.59, n_remote −0.51. So on AVERAGE bigger
+fleets favor A — but the variance explodes with size: cell deltas are late-swap
+quanta of ±10..±80 (a timing lottery on which deaths each packing catches) plus
+rare ±100/±200 daily/weekly-limit quanta (e.g. fleet n12_4 saw dOps −214 at two
+origins: B tripped daily+weekly limits A avoided).
+
+## Stage 3 — pressure stress: 18-building fleets and the 24-building monolith (48 origins)
+
+Six random 18-building fleets (250–384 devices; ≥15/24 buildings shared
+pairwise, yet the sign flips fleet-to-fleet):
+
+| fleet | n_dev | dt | dOps | dLate | per-origin W/L |
+|---|---|---|---|---|---|
+| n18_1 | 352 | **−24.24** | −21.89 | −1.46 | 20/23 |
+| n18_2 | 251 | −5.25 | −4.23 | −0.83 | 15/9 |
+| n18_4 | 353 | −3.73 | −2.94 | −0.21 | 23/16 |
+| n18_5 | 322 | −1.58 | −5.43 | +5.00 | 19/19 |
+| n18_0 | 384 | **+7.90** | **+10.82** | −3.54 | 20/28 |
+| n18_3 | 349 | **+9.31** | **+9.35** | −0.42 | 21/26 |
+
+**3 of 6 flip to A-loses, and the two big losses are carbon copies of the
+public measurement** (+7.9 with ops +10.8; +9.3 with ops +9.4; timing slightly
+better in both — public: +7.8, ops +9.45, late/early −1.6).
+
+**n24_all — the exact CV 24-building universe planned as ONE fleet instead of
+five fold sub-fleets: mean +0.97 (24W/20L), dLate −4.38, dEarly −1.62,
+dOps +6.47 (daily +2.08, weekly +4.17, travel +0.36).** The same buildings that
+give A a 2.9σ win when planned as five small fleets give a coin-flip-to-loss
+with the public's ops signature when planned as one fleet. Worst origins:
+s21 +257 (ops +311), s16 +192 (ops +214) — pure daily/weekly-limit quanta.
+(Caveat: complement is empty for this fleet, so the fit is self-fit — leaky,
+but identical in both arms; the paired planner delta stands.)
+
+Across all 39 synthetic fleets: A loses in 11/39 (28%); among the 15 fleets
+with ≥200 devices, 3 losses, all with the ops signature.
+
+## Which composition features predict the sign
+
+1. **Constraint pressure (total fleet devices) is the master variable** — it
+   predicts whether the dials matter at all (5% → 85% of origins) and blows up
+   the variance. Below ~100 devices A≈B almost everywhere.
+2. At high pressure the sign is a **daily/weekly-limit collision lottery**
+   (±100/±200 quanta), quasi-random in composition; weakly suggestive (n=6,
+   anecdotal): including the two death-densest mid-size buildings
+   (b_9a5a45d6e498 24dev/15deaths, b_a0e6376410cb 28dev/11deaths) correlates
+   with A-worse ops (membership-dOps corr +0.65/+0.74).
+3. Remote buildings and the 117-device giant increase activation but do not
+   fix the sign (fleets containing the giant span dt −19.9 to +8.8 at n≤12,
+   and the giant is absent from both n18 losers).
+
+## Implications
+
+- The CV structurally cannot price this trade: its test fleets are ~5-building
+  sub-fleets where limits don't bind. The public fleet is a single large fleet
+  where they do. The n24 monolith is the cheapest local proxy for that regime,
+  and it sides with the public (A ≈ +1, ops-worse).
+- The dial choice (split3/tw2 vs split7/tw1) is a ~±10–25 ops lottery on any
+  large unseen fleet; the −4.1 CV delta should not be treated as transferable
+  evidence for A on the private split. On pressure-matched local evidence the
+  two configs are statistically indistinguishable (n24: +0.97±8.64).
+
+Raw data: /tmp/c35/stage1_merged.csv, stage2_all48.csv (+ per-fleet
+stage2_perfleet_48orig.csv, cells stage2_cells_48orig.csv), stage3_stress.csv
+(+ cells), stage3_n24.csv (+ cells), building_meta.csv, fleet defs, and the
+runner fleetprobe.py. Stage-2 6-origin pilot (stage2_results.csv) superseded
+by the all-48 rerun. Origins for stress/all48: all 48; fits per fleet on the
+complement buildings' closed pairs (deployment-faithful), shared across arms.