Thibaut Forest Claude Opus 5 (1M context) commited on
Commit ·
a96ad81
1
Parent(s): 1cefc38
Campaign 2: agent_reports_bgkf README + FRONTIER board (cross-lane composition hub)
Browse files- agent_reports_bgkf/FRONTIER.md +22 -0
- agent_reports_bgkf/README.md +28 -0
- agent_reports_bgkf/bl_findings.md +49 -0
- agent_reports_bgkf/ft_findings.md +34 -0
- agent_reports_bgkf/md_findings.md +21 -0
- agent_reports_bgkf/pl_findings.md +35 -0
- agent_reports_bgkf/rk_findings.md +40 -0
- agent_reports_bgkf/tr_findings.md +28 -0
- agent_reports_bgkf/wd_findings.md +50 -0
agent_reports_bgkf/FRONTIER.md
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# FRONTIER BOARD (maintained by the coordinator - agents READ before each new candidate)
|
| 2 |
+
|
| 3 |
+
Updated: 2026-08-22 21:2x. Baseline: champion logbag/base = 1760.2 (swaps 9.5).
|
| 4 |
+
|
| 5 |
+
## Current best composable base
|
| 6 |
+
- **variant='vrel'** (fleet-relative voltage + vmin7_rank): -66.8 +/- 41.4, 32W/15L,
|
| 7 |
+
swaps flat (v3-vintage read, ft lane re-confirming). COMPOSE ON THIS where
|
| 8 |
+
compatible: cv.run_bgkf(variant='vrel', kind=<yours>) or (variant='vrel',
|
| 9 |
+
blend=(<yours>, w)). variant/kind/blend all compose in one call.
|
| 10 |
+
|
| 11 |
+
## Confirmed reads on this instrument (paired vs 1760.2 on 48 origin-sums)
|
| 12 |
+
- logbag_rank -42.0 +/- 37.0 | ranks_add -37.5 | frontier(lbr+gbm30) -26.7 +/- 39.6 n.s.
|
| 13 |
+
- rf -23.6 +/- 63.5 | rankavg25rf -4.5 | gbm +49 | dthaz +23 | logistic +26 | poolown +50.8
|
| 14 |
+
- Quarantined (count-coupled): alpha15solo -144 @ swaps 15.8; alpha+cliff -77 n.s.
|
| 15 |
+
|
| 16 |
+
## Cross-lane rules
|
| 17 |
+
1. Before designing each new candidate, re-read this file.
|
| 18 |
+
2. Prefer composing your idea ON the current best base over the plain champion.
|
| 19 |
+
3. Any result of yours with delta < -1.5*SE and quarantine-clean: ALSO run its
|
| 20 |
+
composition with the frontier base (one extra ~60s run) and note both in
|
| 21 |
+
your findings file.
|
| 22 |
+
4. The coordinator updates this board as lanes report; expect it to change.
|
agent_reports_bgkf/README.md
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# bgkf-CV improvement campaign — live agent reports (2026-08-22 night)
|
| 2 |
+
|
| 3 |
+
Second full campaign, on the settled CV: building-GroupKFold,
|
| 4 |
+
deployment-faithful (cv.py v5, folds summed per scenario).
|
| 5 |
+
|
| 6 |
+
**THE SCORE**: `venv/bin/python -m batteryswap_example.cv bgkf`
|
| 7 |
+
Champion (plain logbag) baseline: **1760.2**
|
| 8 |
+
(warmup 2616.1 / panel 1773.4 / nearend 544.0; swaps 9.5/scenario;
|
| 9 |
+
all-building out-of-fold totals, mean over 48 scenarios).
|
| 10 |
+
|
| 11 |
+
Baseline cells: /tmp/cv_bgkf.csv (fold x origin). Candidates pair on the
|
| 12 |
+
48 per-scenario all-building sums. Survivor bar: delta < 0 beyond ~2*SE
|
| 13 |
+
with swaps ~flat per scenario. Quarantine (public-proven, phase 56-58):
|
| 14 |
+
count-coupled and label-process wins are report-only; split-file leak
|
| 15 |
+
forbidden. This instrument measures CROSS-FLEET transfer (unseen
|
| 16 |
+
buildings) - the deployment path both public and private splits take.
|
| 17 |
+
|
| 18 |
+
| file | theme |
|
| 19 |
+
|---|---|
|
| 20 |
+
| ft_findings.md | feature engineering (relative/rank forms favored here) |
|
| 21 |
+
| md_findings.md | model families & hyperparameters |
|
| 22 |
+
| pl_findings.md | planner: timing, routing (count-coupled = report-only) |
|
| 23 |
+
| bl_findings.md | blends, ensembles, calibration |
|
| 24 |
+
| tr_findings.md | training composition: weights, labels, sampling |
|
| 25 |
+
| rk_findings.md | learning-to-rank objectives |
|
| 26 |
+
| wd_findings.md | RADICAL: drop p42 entirely + registry sweep + wild cards |
|
| 27 |
+
|
| 28 |
+
Runs are ~40-120s each - no triage subsets needed, full runs always.
|
agent_reports_bgkf/bl_findings.md
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# BL: blends / ensembles / calibration (building-GroupKFold)
|
| 2 |
+
|
| 3 |
+
STATUS: Runs A (surface) + C (fold-ensemble, wiener refine) done; Run B (vrel base) running.
|
| 4 |
+
|
| 5 |
+
Driver /tmp/bl2/driver.py = exact replica of cv.run_bgkf, per-fold fits amortized across
|
| 6 |
+
candidates; all blends value-preserving (instrument's own rank-blend mapping -> count
|
| 7 |
+
machinery untouched). Validation: bl_w0 delta 0.0 exactly; bl_rf25 (-4.5+/-5.2) and
|
| 8 |
+
bl_gbm25 (+7.9) reproduce the recorded instrument reads to the decimal.
|
| 9 |
+
|
| 10 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 11 |
+
|---|---|---|---|---|---|
|
| 12 |
+
| bl_wnr20 | 1709.9 | -50.3 | 19.7 | 18W/12L | 9.62 |
|
| 13 |
+
| bl_wnr25 | 1715.4 | -44.8 | 23.1 | 21W/14L | 9.65 |
|
| 14 |
+
| bl_wnr40 | 1720.6 | -39.6 | 33.8 | 20W/23L | 9.69 |
|
| 15 |
+
| bl_wnr30 | 1720.9 | -39.3 | 23.3 | 19W/18L | 9.69 |
|
| 16 |
+
| bl_wnr15 | 1726.8 | -33.4 | 16.9 | 13W/10L | 9.60 |
|
| 17 |
+
| bl_wnr35 | 1738.0 | -22.1 | 31.2 | 19W/23L | 9.69 |
|
| 18 |
+
| bl_wnr10 | 1742.1 | -18.0 | 12.7 | 9W/5L | 9.52 |
|
| 19 |
+
| bl_rf40 | 1754.4 | -5.8 | 25.5 | 19W/20L | 9.56 |
|
| 20 |
+
| bl_rf25 | 1755.6 | -4.5 | 5.2 | 15W/9L | 9.54 |
|
| 21 |
+
| bl_dh40 | 1759.5 | -0.7 | 0.6 | 3W/0L | 9.48 |
|
| 22 |
+
| bl_dh25 / bl_dh10 / bl_bbo25 / bl_w0 | 1760.2 | ~0.0 | 0.0 | - | 9.50 |
|
| 23 |
+
| bl_gbm10 | 1760.3 | +0.2 | 0.4 | 4W/2L | 9.52 |
|
| 24 |
+
| bl_rf10 | 1761.7 | +1.5 | 1.4 | 3W/3L | 9.54 |
|
| 25 |
+
| bl_gbm25 | 1768.1 | +7.9 | 9.1 | 13W/9L | 9.54 |
|
| 26 |
+
| bl_bbjo100 (LOBO jackknife order) | 1770.4 | +10.3 | 9.7 | 1W/3L | 9.50 |
|
| 27 |
+
| bl_gbm40 | 1770.6 | +10.4 | 23.0 | 16W/21L | 9.60 |
|
| 28 |
+
| bl_bbo50 | 1781.7 | +21.5 | 11.4 | 2W/7L | 9.50 |
|
| 29 |
+
| bl_tau10 | 1785.2 | +25.1 | 12.6 | 1W/6L | 9.52 |
|
| 30 |
+
| bl_bbo100 (building-bag order, w=1) | 1801.1 | +41.0 | 14.8 | 6W/16L | 9.54 |
|
| 31 |
+
| bl_bbraw (building-bag raw values) | 1803.9 | +43.8 | 19.9 | 10W/35L | 10.02 |
|
| 32 |
+
| bl_tau25 | 1880.4 | +120.3 | 45.9 | 7W/27L | 9.62 |
|
| 33 |
+
| bl_tau40 | 1984.7 | +224.5 | 57.4 | 11W/30L | 9.73 |
|
| 34 |
+
|
| 35 |
+
One-liners:
|
| 36 |
+
- SURVIVOR: bl_wnr20 = champion logbag + 20% Wiener first-hitting-time ORDER, value-preserving.
|
| 37 |
+
-50.3 +/- 19.7 (2.55 SE), 18W/12L. Count-coupling CLEAN: corr(count delta, cost delta)
|
| 38 |
+
= -0.005; count-flat scenarios (43/48) delta -53.7; wins in warmup (-50.5) AND panel
|
| 39 |
+
(-59.6), flat nearend (+2.8). Physics order corrects the classifier exactly where
|
| 40 |
+
cross-fleet probability calibration drifts. Weight curve is a clean bowl peaked at 0.20.
|
| 41 |
+
- FOLD-ENSEMBLE (novel test) REFUTED: building-bagged logbag hurts in every form -
|
| 42 |
+
raw values +43.8 (and count-coupled, swaps 10.02), pure order +41.0, 50% order +21.5,
|
| 43 |
+
LOBO jackknife order +10.3. One fit on ALL train buildings is strictly better; bagging
|
| 44 |
+
over buildings only adds order noise. Do not revisit.
|
| 45 |
+
- tau (physics rank alone) toxic at any weight; dthaz order ~= logbag order (inert);
|
| 46 |
+
rf/gbm blends flat cross-fleet: campaign-1 rank-blend magic did not transfer.
|
| 47 |
+
|
| 48 |
+
Conclusion: pending Run B (wiener on vrel base), Run D (confidence-weighted local blend),
|
| 49 |
+
Run E (quantile calibration, report-only by construction).
|
agent_reports_bgkf/ft_findings.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# FT (feature engineering) — building-GroupKFold campaign 2
|
| 2 |
+
|
| 3 |
+
STATUS: singles sweep nearly done; combos on the vrel lead next.
|
| 4 |
+
|
| 5 |
+
Baseline (plain logbag, /tmp/cv_bgkf.csv): SCORE 1760.2, swaps 9.5/scenario.
|
| 6 |
+
|
| 7 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 8 |
+
|---|---|---|---|---|---|
|
| 9 |
+
| ft_vrel_ranks | 1685.6 | -74.6 | 43.2 | 31/17 | 8.94 |
|
| 10 |
+
| vrel | 1693.4 | -66.8 | 41.4 | 32/15 | 9.29 |
|
| 11 |
+
| ft_vrel_only | 1697.5 | -62.7 | 43.1 | 33/15 | 9.27 |
|
| 12 |
+
| logbag_rank | 1718.1 | -42.0 | 37.0 | 30/17 | 9.10 |
|
| 13 |
+
| ft_zv | 1719.6 | -40.5 | 38.8 | 27/18 | 9.27 |
|
| 14 |
+
| ranks_add | 1722.7 | -37.5 | 37.7 | 28/19 | 9.04 |
|
| 15 |
+
| ft_vmin7r_only | 1737.0 | -23.2 | 13.9 | 27/12 | 9.50 |
|
| 16 |
+
| ft_rstale | 1742.8 | -17.4 | 19.3 | 32/15 | 9.40 |
|
| 17 |
+
| ft_rslope | 1755.3 | -4.9 | 13.1 | 28/13 | 9.48 |
|
| 18 |
+
| ft_vrel7 | 1763.2 | +3.0 | 31.7 | 22/22 | 9.44 |
|
| 19 |
+
| ft_bldg_vrel | 1771.5 | +11.4 | 9.5 | 25/15 | 9.48 |
|
| 20 |
+
| ft_rdrop | 1774.7 | +14.6 | 14.5 | 21/18 | 9.42 |
|
| 21 |
+
|
| 22 |
+
- ft_vrel_ranks: vrel+vmin7_rank+r_vlast+r_tau. Best point est. but swaps 8.94 (-0.56 count swing — appetite-in-disguise risk, needs scrutiny).
|
| 23 |
+
- vrel: prior read reproduced exactly (-66.8+/-41.4). The lead. Swaps near-flat (-0.21).
|
| 24 |
+
- ft_vrel_only: vrel without vmin7_rank — carries most of the effect, noisy (SE 43).
|
| 25 |
+
- logbag_rank / ranks_add: priors reproduced exactly.
|
| 26 |
+
- ft_zv: per-fleet z-score of v_last — same family, weaker than median-offset vrel.
|
| 27 |
+
- ft_vmin7r_only: vmin7_rank alone — small, VERY clean (SE 13.9, 27W/12L, swaps exactly 9.50 = pure ordering).
|
| 28 |
+
- ft_rstale: fleet rank of staleness — weakly helpful, ns.
|
| 29 |
+
- ft_rslope: fleet rank of slope_lastk — flat.
|
| 30 |
+
- ft_vrel7: v_min_7 median-offset — dead; v_last is the right reference.
|
| 31 |
+
- ft_bldg_vrel: per-building median offset — HARMFUL; buildings too small a group.
|
| 32 |
+
- ft_rdrop: fleet rank of drop_recent — harmful.
|
| 33 |
+
|
| 34 |
+
Conclusion so far: fleet-relative LEVEL (vrel) is the transfer signal; median-offset beats z-score; per-building grouping fails; dynamics ranks add nothing.
|
agent_reports_bgkf/md_findings.md
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# md — Model families & hyperparameters (building-GroupKFold)
|
| 2 |
+
|
| 3 |
+
STATUS: running. Baseline logbag SCORE 1760.2, swaps 9.5. HGB+monotone axis in progress.
|
| 4 |
+
|
| 5 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 6 |
+
|---|---|---|---|---|---|
|
| 7 |
+
| md_hgb_d2 (mono, depth2, l2=1) | 1673.7 | -86.4 | 72.7 | 28/20 | 9.25 |
|
| 8 |
+
| md_hgb_l25 (mono, l2=5) | 1691.8 | -68.3 | 69.2 | 26/22 | 9.40 |
|
| 9 |
+
| md_hgbmono_l2 (mono, l2=1) | 1715.1 | -45.1 | 70.2 | 27/21 | 9.35 |
|
| 10 |
+
| md_hgb_leaf50 (mono, leaf50, l2=1) | 1716.1 | -44.1 | 71.4 | 25/23 | 9.44 |
|
| 11 |
+
| md_hgb_l2s (mono, l2=1, lr.05 x400) | 1726.2 | -34.0 | 69.9 | 23/25 | 9.46 |
|
| 12 |
+
| logbag (baseline) | 1760.2 | 0.0 | - | - | 9.50 |
|
| 13 |
+
| md_hgbmono_slow (mono, lr.05 x400) | 1768.6 | +8.5 | 73.6 | 23/25 | 9.38 |
|
| 14 |
+
| md_hgbmono (mono, gbm defaults) | 1788.6 | +28.4 | 71.0 | 22/26 | 9.54 |
|
| 15 |
+
| md_gbm (unconstrained control) | 1809.5 | +49.3 | 85.7 | 24/24 | 9.94 |
|
| 16 |
+
|
| 17 |
+
- gbm control reproduces prior read exactly (+49). Monotone constraints alone: +49 -> +28.
|
| 18 |
+
- Monotone + regularization is a clean monotone trend: l2=1 -45, l2=5 -68, depth2+l2=1 -86.
|
| 19 |
+
- None past 2*SE yet; swaps drift slightly low (9.25-9.45) — count-coupling under watch.
|
| 20 |
+
|
| 21 |
+
Conclusion: pending — continuing HGB regularization sweep (stumps, depth2+l2=5), then linear-family grid.
|
agent_reports_bgkf/pl_findings.md
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# PL (planner) findings — building-GroupKFold campaign 2
|
| 2 |
+
|
| 3 |
+
STATUS: running — repro exact (1760.2, delta 0.0). Routing knobs converged; timing-shift family in progress.
|
| 4 |
+
|
| 5 |
+
## Code facts (read of models_cv.py / cv.py)
|
| 6 |
+
- CHAMPION['day_q']=0.05 is INERT for tsroute: `plan_tsroute` accepts `day_q` but never reads it.
|
| 7 |
+
Day placement = argmin of `cost_curves` (piecewise-linear CDF through DayQuantiles cols) + `optimize_days` local search with exact mobility sim.
|
| 8 |
+
- Untouched routing knobs: `optimize_days.nn_order` (NN building order within a day, default False), `.constructive` (limit-aware greedy start, default False), `.restarts`=2/`.iters`=60.
|
| 9 |
+
- `cost_curves(blend>0)` flattens low-p42 devices' day-CDF toward uniform (defers marginal swaps) — unused by tsroute (blend=0).
|
| 10 |
+
- ec_margin/ec_pmax gate the break-even top-up (inclusion-adjacent — swaps must stay flat).
|
| 11 |
+
|
| 12 |
+
## Running table
|
| 13 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 14 |
+
|---|---|---|---|---|---|
|
| 15 |
+
| baseline | 1760.2 | - | - | - | 9.5 |
|
| 16 |
+
| pl_shift4 | 1750.4 | -9.7 | 7.5 | 21W/27L | 9.5 |
|
| 17 |
+
| pl_shift2 | 1754.1 | -6.1 | 6.2 | 21W/27L | 9.5 |
|
| 18 |
+
| pl_shift6 | 1756.5 | -3.7 | 6.5 | 15W/33L | 9.5 |
|
| 19 |
+
| pl_split10 | 1759.8 | -0.4 | 0.7 | 11W/13L | 9.5 |
|
| 20 |
+
| pl_nn | 1760.0 | -0.2 | 0.2 | 8W/6L | 9.5 |
|
| 21 |
+
| pl_restarts8 | 1760.1 | -0.1 | 0.1 | 8W/4L | 9.5 |
|
| 22 |
+
| pl_repro | 1760.2 | 0.0 | 0.0 | - | 9.5 |
|
| 23 |
+
| pl_split14 | 1760.6 | +0.4 | 0.8 | 6W/18L | 9.5 |
|
| 24 |
+
| pl_split3 | 1761.0 | +0.9 | 0.7 | 10W/11L | 9.5 |
|
| 25 |
+
| pl_split5 | 1761.1 | +0.9 | 0.8 | 11W/9L | 9.5 |
|
| 26 |
+
| pl_cons | 1761.2 | +1.1 | 0.7 | 5W/8L | 9.5 |
|
| 27 |
+
|
| 28 |
+
## One-liners
|
| 29 |
+
- pl_repro: exact match to shared baseline — in-process runner validated.
|
| 30 |
+
- pl_nn / pl_restarts8 / pl_cons: optimize_days is already converged; routing slack on ~90-device fleets is <1 point despite frictions being relatively larger. Routing axis is DEAD here.
|
| 31 |
+
- pl_split3..14: flat (SE ~0.7); champion split_days=7 fine, knob insensitive.
|
| 32 |
+
- pl_shift2/4/6 (risk-scaled k*p42 days earlier via day-quantile shift): late drops -10..-15, early rises +4..+11, net best -9.7±7.5 at k=4 — SUB-BAR (needs 2*SE=15), same verdict as campaign 1. Swaps flat 9.5 everywhere (shift is inclusion-neutral by construction except ec top-up, unchanged).
|
| 33 |
+
|
| 34 |
+
## Conclusion
|
| 35 |
+
- (pending)
|
agent_reports_bgkf/rk_findings.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# rk: learning-to-rank (Campaign 2, building-GroupKFold)
|
| 2 |
+
|
| 3 |
+
STATUS: running — headline (within-building pairwise logistic) mapped across weights; variants next.
|
| 4 |
+
|
| 5 |
+
Baseline: champion logbag/tsroute, SCORE 1760.2, swaps 9.5/scenario (48 paired all-building scenario sums).
|
| 6 |
+
|
| 7 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 8 |
+
|---|---|---|---|---|---|
|
| 9 |
+
| rk_plb25 | 1767.1 | +7.0 | 9.2 | 14W/9L | 9.46 |
|
| 10 |
+
| rk_plb50 | 1853.3 | +93.2 | 39.1 | 13W/26L | 9.50 |
|
| 11 |
+
| rk_plb100 | 2070.3 | +310.1 | 55.8 | 9W/39L | 9.56 |
|
| 12 |
+
|
| 13 |
+
- rk_plb*: pairwise logistic on standardized feature diffs, pairs within (origin, building) — the deployment-shaped comparison. In-sample AUC 0.965, sane coefs. Ordering is strictly WORSE than champion logbag: monotone dose-response (+310 → +93 → +7 as weight 1.0 → 0.5 → 0.25). At 0.25 near-neutral (14W/9L but heavy right tail).
|
| 14 |
+
|
| 15 |
+
Conclusion: pending — within-origin contrast, days-weighting, vrel/rank-input features, RankNet still to run.
|
| 16 |
+
|
| 17 |
+
## Update 1 (all runs, sorted)
|
| 18 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 19 |
+
|---|---|---|---|---|---|
|
| 20 |
+
| rk_rnod50 | 1699.1 | -61.1 | 33.5 | 24W/15L | 9.58 |
|
| 21 |
+
| rk_rno50 | 1709.8 | -50.4 | 34.5 | 27W/14L | 9.50 |
|
| 22 |
+
| rk_rno40 | 1734.0 | -26.2 | 16.8 | 21W/8L | 9.56 |
|
| 23 |
+
| rk_rno25 | 1741.3 | -18.9 | 12.2 | 14W/5L | 9.54 |
|
| 24 |
+
| rk_plo25 | 1761.2 | +1.0 | 1.1 | 6W/5L | 9.50 |
|
| 25 |
+
| rk_plb25 | 1767.1 | +7.0 | 9.2 | 14W/9L | 9.46 |
|
| 26 |
+
| rk_rno100 | 1786.1 | +25.9 | 45.1 | 23W/23L | 9.50 |
|
| 27 |
+
| rk_plb50 | 1853.3 | +93.2 | 39.1 | 13W/26L | 9.50 |
|
| 28 |
+
| rk_rnb100 | 1868.3 | +108.2 | 54.4 | 16W/32L | 9.48 |
|
| 29 |
+
| rk_plod100 | 1871.4 | +111.2 | 36.1 | 16W/28L | 9.67 |
|
| 30 |
+
| rk_plo100 | 1894.0 | +133.9 | 37.9 | 13W/31L | 9.60 |
|
| 31 |
+
| rk_plov100 | 1924.5 | +164.4 | 33.3 | 11W/33L | 9.62 |
|
| 32 |
+
| rk_plbr100 | 2054.9 | +294.7 | 53.9 | 11W/37L | 9.54 |
|
| 33 |
+
| rk_plb100 | 2070.3 | +310.1 | 55.8 | 9W/39L | 9.56 |
|
| 34 |
+
|
| 35 |
+
- HEADLINE INVERTED: within-BUILDING pairs are the WORST pair scheme (plb100 +310 vs plo100 +134; rnb100 +108 vs rno100 +26). Only ~25k pairs from 80 dying devices — too sparse; cross-building negatives add signal, not noise.
|
| 36 |
+
- Linear pairwise logistic never beats logbag ordering at any weight (it is logbag's own family refit on a noisier objective).
|
| 37 |
+
- RankNet (torch MLP 32-16, 3-seed bag, within-origin pairs) is a NEAR-PEER ordering at full weight (+25.9 +/- 45.1, 23W/23L) that is DECORRELATED from champion -> rank-average hedges win: w=0.5 -50.4 (1.5 SE), days-weighted w=0.5 -61.1 (1.8 SE), swaps flat (9.50-9.58).
|
| 38 |
+
- None past 2*SE yet; variance is RankNet seed noise -> trying bigger seed bag next.
|
| 39 |
+
|
| 40 |
+
Conclusion (interim): promising non-survivor; best rk_rnod50 at 1.8 SE.
|
agent_reports_bgkf/tr_findings.md
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# TR (training-data composition) — building-GroupKFold campaign 2
|
| 2 |
+
|
| 3 |
+
STATUS: running. Baseline verified: SCORE 1760.2, swaps 9.50/scenario (/tmp/cv_bgkf.csv).
|
| 4 |
+
Paired eval: per-origin all-building sums vs baseline, SE = std/sqrt(48).
|
| 5 |
+
|
| 6 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 7 |
+
|---|---|---|---|---|---|
|
| 8 |
+
| tr_d49 | 1761.6 | +1.4 | 28.0 | 12/36 | 10.54 |
|
| 9 |
+
| tr_lbo5 | 1780.1 | +19.9 | 10.3 | 20/20 | 9.71 |
|
| 10 |
+
| tr_rw240 | 1780.1 | +19.9 | 37.9 | 27/16 | 9.06 |
|
| 11 |
+
| tr_invk | 1782.7 | +22.5 | 51.0 | 16/32 | 12.44 |
|
| 12 |
+
| tr_soft21 | 1799.3 | +39.1 | 41.0 | 21/20 | 9.10 |
|
| 13 |
+
| tr_rw120 | 1803.8 | +43.6 | 44.4 | 27/21 | 8.79 |
|
| 14 |
+
| tr_d35 | 1824.2 | +64.0 | 38.4 | 27/21 | 8.29 |
|
| 15 |
+
| tr_bw | 1827.9 | +67.7 | 57.3 | 17/31 | 14.77 |
|
| 16 |
+
| tr_bwsqrt | 1740.0 | -20.2 | 45.0 | 18/30 | 12.12 |
|
| 17 |
+
|
| 18 |
+
One-liners:
|
| 19 |
+
- tr_rw240/tr_rw120: recency half-lives (240/120d) — both harmful; deployment-meaningful recency does NOT help cross-fleet transfer; swaps drift down.
|
| 20 |
+
- tr_bw: building-balanced weights (equal mass/building) — badly count-coupled (+5.3 swaps) and harmful, as on same-fleet CV.
|
| 21 |
+
- tr_bwsqrt: sqrt building balance — only negative mean so far but SE 45 (0.4 sigma) AND +2.6 swaps = quarantined count-coupling; twin pending.
|
| 22 |
+
- tr_invk: 1/k origin-overlap deweighting — count-coupled (+2.9 swaps) and harmful; campaign-1 dedup lesson reproduces.
|
| 23 |
+
- tr_lbo5: leave-building-group-out bagging (5 groups) — tight SE, clearly harmful (+19.9 +/- 10.3); averaging over building-dropped fits blurs, not robustifies.
|
| 24 |
+
- tr_d49/tr_d35: relabel window 49/35d — d35 clearly bad; d49 null-mean but 12/36 W/L with +1 swap (count-driven big catches vs steady losses); value twin pending.
|
| 25 |
+
- tr_soft21: soft positive decay by days-to-death (hl 21, mass-normalized) — harmful.
|
| 26 |
+
|
| 27 |
+
Conclusion so far: no survivor. All count-moving "wins" are on the wrong side anyway.
|
| 28 |
+
Workspace: /tmp/tr2 (tr_models.py, run_one.py); candidate CSVs /tmp/cv_bgkfcand_tr_*.csv.
|
agent_reports_bgkf/wd_findings.md
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# WD (radical: drop p42 + registry sweep + wild cards) — bgkf campaign 2
|
| 2 |
+
|
| 3 |
+
STATUS: registry sweep running in background (17/43 kinds done, ~25s each).
|
| 4 |
+
Radical pipelines coded (/tmp/wd2/): trajectory-sim cache + calibration,
|
| 5 |
+
expected-cost planner, direct death-day planner, rank-blend hybrid. Runs
|
| 6 |
+
start when the sweep finishes (one-run-at-a-time rule).
|
| 7 |
+
|
| 8 |
+
Baseline (plain logbag + tsroute, /tmp/cv_bgkf.csv): SCORE 1760.2
|
| 9 |
+
(late 1221.7 / early 297.0; swaps 9.5/scenario).
|
| 10 |
+
|
| 11 |
+
## Part 2 — registry sweep (interim, from /tmp/wd2/sweep.log)
|
| 12 |
+
|
| 13 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 14 |
+
|---|---|---|---|---|---|
|
| 15 |
+
| wd_logcal | 1841.7 | +81.5 | 45.1 | 22W/26L | 11.0 |
|
| 16 |
+
| wd_logbag_rw120 | 1803.8 | +43.6 | 44.4 | 27W/21L | 8.8 |
|
| 17 |
+
| wd_logbag_rw240 | 1780.1 | +19.9 | 37.9 | 27W/16L | 9.1 |
|
| 18 |
+
| wd_logbag_c03 | 1750.5 | -9.7 | 8.5 | 23W/9L | 9.5 |
|
| 19 |
+
| wd_logbag_c3 | 1768.8 | +8.6 | 10.2 | 7W/9L | 9.5 |
|
| 20 |
+
| wd_logbag_c01 | 1760.6 | +0.4 | 28.4 | 25W/16L | 9.5 |
|
| 21 |
+
| wd_logbag_aux | 1799.7 | +39.5 | ~ | ~ | ~ |
|
| 22 |
+
| (…running; full table at completion) | | | | | |
|
| 23 |
+
|
| 24 |
+
One-liner so far: nothing beats the champion; logbag_c03 (stronger reg.)
|
| 25 |
+
is the only negative delta and is inside 2*SE. Aux features hurt on this
|
| 26 |
+
cross-fleet instrument (they helped nothing on the old one either).
|
| 27 |
+
|
| 28 |
+
## Part 1 — radical pipelines (no p42)
|
| 29 |
+
|
| 30 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 31 |
+
|---|---|---|---|---|---|
|
| 32 |
+
|
| 33 |
+
(runs pending — sim cache found FAST: forecast3 = 0.7s/origin, full-origin
|
| 34 |
+
cache ~2min/seed)
|
| 35 |
+
|
| 36 |
+
Design notes:
|
| 37 |
+
- Raw sim p42 is a CROSSING probability, not a recorded-death probability:
|
| 38 |
+
fleet sum_p42 ~60 vs logbag ~12 at s_20. A standalone raw-sim run through
|
| 39 |
+
the count rule will be count-inflated (will report, honest baseline).
|
| 40 |
+
- wd_simcal*: per-fold 1D logistic calibration logit(sim_p42)->dies42 fit
|
| 41 |
+
ONLY on TRAIN buildings' closed pairs — physics does the ranking+timing,
|
| 42 |
+
labels only set the 2-parameter scale. Headline arm.
|
| 43 |
+
- wd_ec_*: plan_ecroute (expected-cost include + day, NO count rule).
|
| 44 |
+
- wd_dd_*: direct death-day scheduling (theta/q/margin, building-batched).
|
| 45 |
+
- wd_simblend*: order-only rank blend into champion via run_bgkf blend
|
| 46 |
+
machinery (quarantine-safe hybrid).
|
| 47 |
+
|
| 48 |
+
## Conclusion
|
| 49 |
+
|
| 50 |
+
Pending.
|