Thibaut Forest Claude Fable 5 commited on
Commit ·
ba865a1
1
Parent(s): 3f9bf12
Campaign 2 CLOSED: all 7 lanes final; frontier adopted at 3f9bf12; rk_4w shelved (torch, no sign edge); aux_ho raw = value channel (Phase 59)
Browse files- EXPERIMENTS.md +59 -0
- agent_reports_bgkf/FRONTIER.md +27 -4
- agent_reports_bgkf/md_findings.md +57 -34
- agent_reports_bgkf/rk_findings.md +88 -1
EXPERIMENTS.md
CHANGED
|
@@ -3190,3 +3190,62 @@ all-24-building out-of-fold cost), SCORE = mean of the 48 totals.
|
|
| 3190 |
nearend 544.0; swaps 9.5/scenario). Identical information to the fold-mean
|
| 3191 |
form (352.0 x 5), same paired decisions; stale pre-v5 candidate CSVs purged
|
| 3192 |
from /tmp (only the phase-58 faithful runs remain valid).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3190 |
nearend 544.0; swaps 9.5/scenario). Identical information to the fold-mean
|
| 3191 |
form (352.0 x 5), same paired decisions; stale pre-v5 candidate CSVs purged
|
| 3192 |
from /tmp (only the phase-58 faithful runs remain valid).
|
| 3193 |
+
|
| 3194 |
+
## Phase 59 - campaign 2 (bgkf instrument, 7 lanes) closes; frontier ft_vmin7r_wnr20 ADOPTED and deployed (3f9bf12)
|
| 3195 |
+
|
| 3196 |
+
Deadline-day morning. Campaign 2 re-tested every prior method plus radical
|
| 3197 |
+
non-p42 pipelines on THE SCORE (bgkf v5 summed, baseline 1760.2), with a
|
| 3198 |
+
coordinator-maintained FRONTIER board for cross-lane composition
|
| 3199 |
+
(agent_reports_bgkf/). Overnight outage (repo re-cloned, /tmp+venv+dataset
|
| 3200 |
+
lost) fully recovered: dataset copied from rul_mgt_zero, venv symlinked to
|
| 3201 |
+
the blind repo (versions identical), baseline and every headline number
|
| 3202 |
+
reproduced BIT-EXACT (the harness is deterministic end to end).
|
| 3203 |
+
|
| 3204 |
+
**ADOPTED - ft_vmin7r_wnr20 (commit 3f9bf12, BATTERYSWAP_FT2 default ON):**
|
| 3205 |
+
logbag on FEATURES+vmin7_rank, wiener .20 value-preserving order blend.
|
| 3206 |
+
1688.4 vs 1760.2 = -71.8 +/- 23.5 (3.05 sigma, Wilcoxon .037, swaps 9.58
|
| 3207 |
+
flat, count-clean by every control, components additive -50.3 wiener /
|
| 3208 |
+
-23.2 vmin7r, reproduced exactly by ft, md and rk lanes independently).
|
| 3209 |
+
Deployment review that preceded adoption:
|
| 3210 |
+
- Reference-shift worst case (the vrel killer): vmin7_rank recomputed
|
| 3211 |
+
WITHIN each ~5-building test fleet -> still -60.8 +/- 26.3 (2.31 sigma);
|
| 3212 |
+
consistent small degradation vs table-native (+11.0 +/- 17.9, 9W/35L).
|
| 3213 |
+
Deployment ranks over a full ~922-device fleet (features_at_origin
|
| 3214 |
+
own-fleet semantics), closer to the shared-reference case.
|
| 3215 |
+
- Ft2RUL parity: submit.py implementation run through the CV harness
|
| 3216 |
+
reproduces the frontier CELL-EXACT (max diff 0.0, swaps identical).
|
| 3217 |
+
- Deployment data path: wnr_m/s2/n merged into the shipped train table
|
| 3218 |
+
(97% coverage; WienerFHT shrinks through NaNs); split-path wiener params
|
| 3219 |
+
computed past-only from the scored split's own telemetry (1.3s once +
|
| 3220 |
+
0.1s/scenario). Soft degradation: missing wnr -> pure vm7 ordering
|
| 3221 |
+
(values identical by construction); any fit error -> legacy champion.
|
| 3222 |
+
BATTERYSWAP_FT2=0 = the measured-1661.45 config.
|
| 3223 |
+
|
| 3224 |
+
**NOT adopted (the two bigger nominal numbers):**
|
| 3225 |
+
- rk_4w (vm7 values; order .35 logbag + .20 wiener + .20 RankNet + .25
|
| 3226 |
+
aux_ho) = -111.5 +/- 27.7, 4.02 sigma, Wilcoxon .001 - the campaign's
|
| 3227 |
+
strongest fully-anchored number, seed-block replicated. Head-to-head vs
|
| 3228 |
+
the frontier: -39.7 +/- 20.3 but 15W/15L/18T, Wilcoxon .131 - mean-only
|
| 3229 |
+
support; plus a torch dependency (MLP 32-16 x 7 seeds) the Docker is not
|
| 3230 |
+
known to carry. SHELVED (agent_reports_bgkf/artifacts/).
|
| 3231 |
+
- wd raw vm7+aux_ho+wiener.20 = -126.4 +/- 43.8: rk's 4-way control run
|
| 3232 |
+
settled the adoption form - aux_ho as an ORDER source adds +6.1 n.s. on
|
| 3233 |
+
the frontier, so the raw excess is VALUE-CHANNEL (the calibration channel
|
| 3234 |
+
publicly refuted in phase 56). Report-only. aux_ho's anchored ordering
|
| 3235 |
+
core (-59.7 on champion base) is real but subsumed.
|
| 3236 |
+
|
| 3237 |
+
Lane verdicts (all closed, findings files final): pl - ranking is the wall
|
| 3238 |
+
(~99% of late cost = exclusion misses; routing axis +/-1). tr - keep the
|
| 3239 |
+
training pool exactly as-is; value-anchored twin flipped every count-smuggle.
|
| 3240 |
+
bl - wiener .20 the one piece; bagging/local-blend/calibration refuted.
|
| 3241 |
+
md - logbag(C=1,b=15) locally optimal; monotone HGB real (-44.7 anchored)
|
| 3242 |
+
but subsumed (+1.0 as third source); MORE BAGS HURT. ft - vmin7_rank the
|
| 3243 |
+
piece; entire vrel family quarantined (anchored twin +21.4, within-fleet
|
| 3244 |
+
recompute +89). rk - see above. wd/radical - physics loses to the 82
|
| 3245 |
+
labels at matched count (+604); EC planner = count knob; direct death-day
|
| 3246 |
+
rejected both ways.
|
| 3247 |
+
|
| 3248 |
+
Private-selection matrix as of this phase: candidate head = 3f9bf12 (FT2
|
| 3249 |
+
frontier); fallback = 3c7b883 (measured 1661.45); never 698401a (1877.08).
|
| 3250 |
+
The user manually triggers public scoring; a public look at 3f9bf12 prices
|
| 3251 |
+
the residual ghost/composition risk the CV cannot see (phase 58).
|
agent_reports_bgkf/FRONTIER.md
CHANGED
|
@@ -1,10 +1,27 @@
|
|
| 1 |
-
# FRONTIER BOARD
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
-
##
|
| 6 |
- **ft_vmin7r_wnr20 -71.8 +/- 23.5 (3.05 sigma, wilcoxon p=.037, swaps 9.58)
|
| 7 |
-
=
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
Composition: FEATURES + vmin7_rank (fleet pct-rank of 7d-min voltage,
|
| 9 |
already in the table) with the wiener .20 value-preserving order blend.
|
| 10 |
Count-clean by every control (corr +0.048; count-flat scenarios carry the
|
|
@@ -53,6 +70,12 @@ Also dead: z-scores (weaker vrel), building relatives, dynamics ranks.
|
|
| 53 |
hgbd1l25-on-vrel leader row is void - vrel base quarantined.)
|
| 54 |
|
| 55 |
## Closed lanes
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
- MODEL FAMILIES (md, complete 08-23, no survivor): keep champion
|
| 57 |
logbag(C=1, bags=15) - locally optimal (C=0.3 -9.7 n.s.; MORE BAGS HURT
|
| 58 |
+21/+23 at 30/60; scalers/penalties/imputers noise). Monotone HGB is
|
|
|
|
| 1 |
+
# FRONTIER BOARD - CAMPAIGN 2 CLOSED (2026-08-23 morning)
|
| 2 |
|
| 3 |
+
All 7 lanes closed. **ft_vmin7r_wnr20 ADOPTED and DEPLOYED at commit
|
| 4 |
+
3f9bf12** (submit.py Ft2RUL, BATTERYSWAP_FT2 default ON; deployment parity
|
| 5 |
+
vs the CV frontier bit-exact, max cell diff 0; soft degradation to legacy
|
| 6 |
+
champion on any failure; BATTERYSWAP_FT2=0 = the measured-1661.45 config).
|
| 7 |
+
Baseline: champion logbag/base = 1760.2 (swaps 9.5).
|
| 8 |
|
| 9 |
+
## Final standings (paired on 48 origin-sums vs 1760.2)
|
| 10 |
- **ft_vmin7r_wnr20 -71.8 +/- 23.5 (3.05 sigma, wilcoxon p=.037, swaps 9.58)
|
| 11 |
+
= THE FRONTIER, adopted. The campaign's one fully-clean survivor that
|
| 12 |
+
also clears deployment review.** Worst-case reference-shift bound: vmin7
|
| 13 |
+
recomputed WITHIN each ~5-building test fleet still -60.8 +/- 26.3
|
| 14 |
+
(2.31 sigma); scored fleets are full ~922-device populations (closer to
|
| 15 |
+
the shared-reference case).
|
| 16 |
+
- rk_4w -111.5 +/- 27.7 (4.02 sigma, Wilcoxon .001, 29W/13L) = strongest
|
| 17 |
+
anchored number, NOT adopted: no sign-test edge over the frontier
|
| 18 |
+
(head-to-head -39.7 +/- 20.3, 15W/15L, Wilcoxon .131) and RankNet is a
|
| 19 |
+
torch dependency with cross-version determinism risk. SHELVED
|
| 20 |
+
(artifacts/rankers.py + run3w.py).
|
| 21 |
+
- wd raw vm7+aux_ho+wiener.20 -126.4 +/- 43.8 = REPORT-ONLY: rk_4wctl
|
| 22 |
+
settled the adoption form - aux_ho as ORDER adds +6.1 n.s. on the
|
| 23 |
+
frontier, so the raw excess is VALUE-CHANNEL (the calibration channel
|
| 24 |
+
the public refuted in phase 56). Statistical tie with rk_4w (+14.9).
|
| 25 |
Composition: FEATURES + vmin7_rank (fleet pct-rank of 7d-min voltage,
|
| 26 |
already in the table) with the wiener .20 value-preserving order blend.
|
| 27 |
Count-clean by every control (corr +0.048; count-flat scenarios carry the
|
|
|
|
| 70 |
hgbd1l25-on-vrel leader row is void - vrel base quarantined.)
|
| 71 |
|
| 72 |
## Closed lanes
|
| 73 |
+
- LEARNING-TO-RANK (rk, complete 08-23, no adoption): see final standings
|
| 74 |
+
- rk_4w/3w552025h are real, seed-replicated, fully anchored, but no
|
| 75 |
+
sign-supported edge over the frontier + torch dependency. Within-building
|
| 76 |
+
pairs are the WORST pair scheme (+310 at full weight); within-origin
|
| 77 |
+
days-weighted pairs the right one. aux_ho order-slot test settled the
|
| 78 |
+
wd adoption form (order adds nothing; raw = value channel).
|
| 79 |
- MODEL FAMILIES (md, complete 08-23, no survivor): keep champion
|
| 80 |
logbag(C=1, bags=15) - locally optimal (C=0.3 -9.7 n.s.; MORE BAGS HURT
|
| 81 |
+21/+23 at 30/60; scalers/penalties/imputers noise). Monotone HGB is
|
agent_reports_bgkf/md_findings.md
CHANGED
|
@@ -1,11 +1,17 @@
|
|
| 1 |
-
# md — Model families & hyperparameters (building-GroupKFold)
|
| 2 |
|
| 3 |
-
STATUS:
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
|
| 8 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
|
| 10 |
| label | SCORE | delta | SE | W/L | swaps |
|
| 11 |
|---|---|---|---|---|---|
|
|
@@ -19,49 +25,66 @@ twin of bowl winner, 3-way order source on vm7 frontier base.
|
|
| 19 |
| md_bl_hgbd1_w3 (mean blend w=.3) | 1702.0 | -58.1 | 38.4 | 27/18 | 9.52 |
|
| 20 |
| md_hgb_d2s (mono d2, lr.05x400) | 1704.6 | -55.5 | 72.2 | 29/19 | 9.50 |
|
| 21 |
| md_hgb_d2l25 (mono d2, l2=5) | 1710.3 | -49.9 | 66.1 | 27/21 | 9.52 |
|
|
|
|
| 22 |
| md_hgbmono_l2 (mono, depth3, l2=1) | 1715.1 | -45.1 | 70.2 | 27/21 | 9.35 |
|
| 23 |
| md_hgb_leaf50 (mono, leaf50, l2=1) | 1716.1 | -44.1 | 71.4 | 25/23 | 9.44 |
|
| 24 |
-
| md_blcm_hgbd1 (blend, count-matched) | 1719.7 | -40.5 | 31.2 | 23/22 | 9.65 |
|
| 25 |
-
|
|
| 26 |
-
|
|
|
|
|
|
|
|
| 27 |
| md_robust (logbag + RobustScaler) | 1747.8 | -12.4 | 13.8 | 24/8 | 9.46 |
|
| 28 |
| md_impmean (logbag, mean imputer) | 1747.7 | -12.4 | 11.5 | 16/5 | 9.50 |
|
| 29 |
| md_logbag_c03 (C=0.3) | 1750.5 | -9.7 | 8.5 | 23/9 | 9.48 |
|
| 30 |
| md_enet (saga elasticnet l1r=.5) | 1756.5 | -3.7 | 3.4 | 15/5 | 9.48 |
|
| 31 |
| logbag (baseline) | 1760.2 | 0.0 | - | - | 9.50 |
|
| 32 |
| md_l1 (saga L1, C=1) | 1766.7 | +6.6 | 10.4 | 13/6 | 9.50 |
|
| 33 |
-
| md_hgbmono_slow (mono, lr.05x400) | 1768.6 | +8.5 | 73.6 | 23/25 | 9.38 |
|
| 34 |
| md_logbag_c3 (C=3) | 1768.8 | +8.6 | 10.2 | 7/9 | 9.50 |
|
| 35 |
| md_logbag_b30 (bags=30) | 1781.2 | +21.0 | 13.7 | 11/12 | 9.50 |
|
| 36 |
| md_logbag_b60 (bags=60) | 1783.2 | +23.0 | 13.7 | 13/16 | 9.52 |
|
| 37 |
-
| md_hgbmono (mono, gbm defaults) | 1788.6 | +28.4 | 71.0 | 22/26 | 9.54 |
|
| 38 |
| md_gbm (unconstrained control) | 1809.5 | +49.3 | 85.7 | 24/24 | 9.94 |
|
| 39 |
|
| 40 |
-
##
|
| 41 |
|
| 42 |
-
| label | SCORE | delta | SE | W/L | swaps | vs
|
| 43 |
|---|---|---|---|---|---|---|
|
| 44 |
-
|
|
| 45 |
-
|
|
| 46 |
-
|
|
| 47 |
-
|
|
| 48 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
## One-liners
|
| 51 |
-
- gbm control reproduces prior
|
| 52 |
-
- Constrained
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
-
|
| 61 |
-
|
| 62 |
-
-
|
| 63 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
Next: bowl completion, anchored twin of bowl winner, then 3-way order source on vm7.
|
|
|
|
| 1 |
+
# md — Model families & hyperparameters (building-GroupKFold) — FINAL
|
| 2 |
|
| 3 |
+
STATUS: DONE (deadline-day morning). Baseline logbag = 1760.2, swaps 9.50
|
| 4 |
+
(regenerated file verified identical before any post-outage pairing; harness
|
| 5 |
+
deterministic, so pre-outage reads are retained). Coordinator queue completed:
|
| 6 |
+
HGB bowl finished, count check on winner done, 3-way composition on frontier done.
|
| 7 |
|
| 8 |
+
## Headline
|
| 9 |
+
Regularized MONOTONE HGB is the only real model-family signal in this lane, and it
|
| 10 |
+
is REAL but REDUNDANT: its ordering core survives value-anchoring (-44.7 ± 38.6)
|
| 11 |
+
yet adds nothing (+1.0 ± 24.9) on top of the vm7+wiener.20 frontier stack.
|
| 12 |
+
**No survivor to promote from the md lane.**
|
| 13 |
+
|
| 14 |
+
## Main table (base variant, paired vs /tmp/cv_bgkf.csv, 48 origin-sums)
|
| 15 |
|
| 16 |
| label | SCORE | delta | SE | W/L | swaps |
|
| 17 |
|---|---|---|---|---|---|
|
|
|
|
| 25 |
| md_bl_hgbd1_w3 (mean blend w=.3) | 1702.0 | -58.1 | 38.4 | 27/18 | 9.52 |
|
| 26 |
| md_hgb_d2s (mono d2, lr.05x400) | 1704.6 | -55.5 | 72.2 | 29/19 | 9.50 |
|
| 27 |
| md_hgb_d2l25 (mono d2, l2=5) | 1710.3 | -49.9 | 66.1 | 27/21 | 9.52 |
|
| 28 |
+
| md_rk_hgbd2_w5 (ANCHORED TWIN of d2, w=.5) | 1715.5 | -44.7 | 38.6 | 24/19 | 9.60 |
|
| 29 |
| md_hgbmono_l2 (mono, depth3, l2=1) | 1715.1 | -45.1 | 70.2 | 27/21 | 9.35 |
|
| 30 |
| md_hgb_leaf50 (mono, leaf50, l2=1) | 1716.1 | -44.1 | 71.4 | 25/23 | 9.44 |
|
| 31 |
+
| md_blcm_hgbd1 (blend d1, count-matched) | 1719.7 | -40.5 | 31.2 | 23/22 | 9.65 |
|
| 32 |
+
| md_hgb_d2l2 (mono d2, l2=2) | 1721.2 | -39.0 | 75.3 | 24/24 | 9.46 |
|
| 33 |
+
| md_hgb_l2s (mono d3, l2=1, lr.05x400) | 1726.2 | -34.0 | 69.9 | 23/25 | 9.46 |
|
| 34 |
+
| md_hgb_d2l10 (mono d2, l2=10) | 1732.8 | -27.4 | 62.8 | 26/22 | 9.62 |
|
| 35 |
+
| md_rk_hgbd1_w5 (anchored twin of d1, w=.5) | 1744.4 | -15.8 | 33.0 | 16/25 | 9.71 |
|
| 36 |
| md_robust (logbag + RobustScaler) | 1747.8 | -12.4 | 13.8 | 24/8 | 9.46 |
|
| 37 |
| md_impmean (logbag, mean imputer) | 1747.7 | -12.4 | 11.5 | 16/5 | 9.50 |
|
| 38 |
| md_logbag_c03 (C=0.3) | 1750.5 | -9.7 | 8.5 | 23/9 | 9.48 |
|
| 39 |
| md_enet (saga elasticnet l1r=.5) | 1756.5 | -3.7 | 3.4 | 15/5 | 9.48 |
|
| 40 |
| logbag (baseline) | 1760.2 | 0.0 | - | - | 9.50 |
|
| 41 |
| md_l1 (saga L1, C=1) | 1766.7 | +6.6 | 10.4 | 13/6 | 9.50 |
|
| 42 |
+
| md_hgbmono_slow (mono d3, lr.05x400) | 1768.6 | +8.5 | 73.6 | 23/25 | 9.38 |
|
| 43 |
| md_logbag_c3 (C=3) | 1768.8 | +8.6 | 10.2 | 7/9 | 9.50 |
|
| 44 |
| md_logbag_b30 (bags=30) | 1781.2 | +21.0 | 13.7 | 11/12 | 9.50 |
|
| 45 |
| md_logbag_b60 (bags=60) | 1783.2 | +23.0 | 13.7 | 13/16 | 9.52 |
|
| 46 |
+
| md_hgbmono (mono d3, gbm defaults) | 1788.6 | +28.4 | 71.0 | 22/26 | 9.54 |
|
| 47 |
| md_gbm (unconstrained control) | 1809.5 | +49.3 | 85.7 | 24/24 | 9.94 |
|
| 48 |
|
| 49 |
+
## Frontier composition (variant vm7 = BASE + vmin7_rank; ref reproduced exactly)
|
| 50 |
|
| 51 |
+
| label | SCORE | delta | SE | W/L | swaps | vs vm7+wnr20 ref |
|
| 52 |
|---|---|---|---|---|---|---|
|
| 53 |
+
| md_vm7_wnr20 (frontier ref, logbag+wiener.20) | 1688.4 | -71.8 | 23.5 | 24/16 | 9.58 | 0 |
|
| 54 |
+
| md_ord3_d2_vm7 (.55 logbag/.25 hgb_d2/.20 wiener) | 1689.3 | -70.8 | 30.2 | 21/21 | 9.65 | +1.0 ± 24.9 |
|
| 55 |
+
| md_ord3_d2w15_vm7 (.65/.15/.20) | 1695.2 | -65.0 | 29.8 | 21/19 | 9.67 | +6.8 ± 17.4 |
|
| 56 |
+
| md_ord3_d1l25_vm7 (.55/.25/.20, stumps) | 1705.3 | -54.9 | 31.7 | 22/21 | 9.62 | +17.0 ± 27.3 |
|
| 57 |
+
|
| 58 |
+
## vrel-family runs (base QUARANTINED by coordinator — record only, not claimed)
|
| 59 |
+
md_hgb_d1_vrel -119.5 ± 63.5 | md_bl_hgbd1l25_vrel -117.8 ± 46.1 (vs vrel-logbag:
|
| 60 |
+
-51.0 ± 34.2) | md_bl_hgbd1_vrel -93.0 ± 45.5 | md_blcm_hgbd1_vrel -92.6 ± 46.8 |
|
| 61 |
+
md_logbag_vrel reference -66.8 ± 41.4 (reproduced frontier read exactly).
|
| 62 |
|
| 63 |
## One-liners
|
| 64 |
+
- gbm control reproduces prior (+49.3). Monotone constraints alone: +49 -> +28.
|
| 65 |
+
- Constrained+regularized trend: depth3 l2=1 -45 / l2=5 -68; depth2 l2=1 -86.4 (bowl
|
| 66 |
+
bottom); stumps -79.0 flat swaps. d2 bowl is NOT smooth (l2=2 -39, l2=5 -50,
|
| 67 |
+
l2=10 -27): l2=1 is the bottom, the rest is per-config noise (SE 63-75). No solo
|
| 68 |
+
config passes 2*SE.
|
| 69 |
+
- COUNT CHECKS on the winner: d2's value-anchored twin (champion p42 values, half-
|
| 70 |
+
weight d2 ordering, count machinery untouched) keeps -44.7 ± 38.6 at swaps 9.60 —
|
| 71 |
+
a genuine ordering core, not a count exploit (raw d2 swaps 9.25 drift explained
|
| 72 |
+
as value-channel, not sum(p42): count-matched blend also kept -40.5 ± 31.2).
|
| 73 |
+
- FRONTIER TEST (the decisive one): as third order source on vm7+wiener.20 the HGB
|
| 74 |
+
increment vanishes — +1.0 ± 24.9 (w .25), +6.8 ± 17.4 (w .15), stumps +17.0.
|
| 75 |
+
HGB's cross-fleet ordering signal is subsumed by wiener + vmin7_rank.
|
| 76 |
+
- Linear family is FLAT: C=0.3 -9.7 ± 8.5 (W/L 23/9) is the only mild positive;
|
| 77 |
+
MORE BAGS HURT (+21.0/+23.0 at 30/60 — keep bags=15); saga L1/enet, RobustScaler,
|
| 78 |
+
mean-imputer all within noise. Champion logbag(C=1, bags=15) is locally optimal.
|
| 79 |
+
- Not run post-outage (deprioritized per coordinator, low prior): xtrees, mlp,
|
| 80 |
+
torch-MLP, Cox p42=1-S(42), class_weight (count-coupled by construction).
|
| 81 |
+
|
| 82 |
+
## Conclusion / recommendation
|
| 83 |
+
NO SURVIVOR from the md lane. Keep champion logbag(C=1, bags=15) as the p42 model;
|
| 84 |
+
do not add bags; do not swap scalers/imputers/penalties. Monotone depth<=2 HGB is a
|
| 85 |
+
validated, quarantine-clean, decorrelated ORDER source in isolation (-44.7 anchored)
|
| 86 |
+
— worth keeping on the shelf ONLY as a fallback third source if the wiener order
|
| 87 |
+
source is ever quarantined; on the current vm7+aux_ho+wiener stack it adds nothing.
|
| 88 |
|
| 89 |
+
Artifacts: /tmp/cv_bgkfcand_md_*.csv (post-outage runs), driver /tmp/md2/{run,cmp}.py,
|
| 90 |
+
models /tmp/md2/mdmodels.py (HGBMono monotone map, MeanBlend/CM, OrderBlend3).
|
|
|
agent_reports_bgkf/rk_findings.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# rk: learning-to-rank (Campaign 2, building-GroupKFold)
|
| 2 |
|
| 3 |
-
STATUS:
|
| 4 |
|
| 5 |
Baseline: champion logbag/tsroute, SCORE 1760.2, swaps 9.5/scenario (48 paired all-building scenario sums).
|
| 6 |
|
|
@@ -38,3 +38,90 @@ Conclusion: pending — within-origin contrast, days-weighting, vrel/rank-input
|
|
| 38 |
- None past 2*SE yet; variance is RankNet seed noise -> trying bigger seed bag next.
|
| 39 |
|
| 40 |
Conclusion (interim): promising non-survivor; best rk_rnod50 at 1.8 SE.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# rk: learning-to-rank (Campaign 2, building-GroupKFold)
|
| 2 |
|
| 3 |
+
STATUS: CLOSED (coordinator adjudication 2026-08-23 — see final section at bottom). No adoption; rk_4w shelved (torch dependency, no sign-test edge over the shipped frontier).
|
| 4 |
|
| 5 |
Baseline: champion logbag/tsroute, SCORE 1760.2, swaps 9.5/scenario (48 paired all-building scenario sums).
|
| 6 |
|
|
|
|
| 38 |
- None past 2*SE yet; variance is RankNet seed noise -> trying bigger seed bag next.
|
| 39 |
|
| 40 |
Conclusion (interim): promising non-survivor; best rk_rnod50 at 1.8 SE.
|
| 41 |
+
|
| 42 |
+
---
|
| 43 |
+
# DEADLINE-DAY RESUME (post-outage rebuild)
|
| 44 |
+
|
| 45 |
+
STATUS: running. /tmp and venv lost overnight; driver REBUILT from recipes above
|
| 46 |
+
(/tmp/rk2/rankers.py, run.py, run3w.py) and validated EXACTLY against recorded
|
| 47 |
+
reads on the restored environment (venv -> blind-repo symlink, sklearn 1.9.0 /
|
| 48 |
+
torch 2.13.0; regenerated baseline verified 1760.1688):
|
| 49 |
+
- rk_rnod50s7 re-run: SCORE 1700.09, delta -60.08 +/- 28.24 (2.13 SE), 26W/10L,
|
| 50 |
+
swaps 9.52 — identical to the decimal. Count check clean: swing max +/-1,
|
| 51 |
+
changed 5/48, mean +0.02.
|
| 52 |
+
- rk_3wrepro re-run: 1688.36, -71.81 +/- 23.50 — frontier base reproduced exactly.
|
| 53 |
+
|
| 54 |
+
IMPORTANT CORRECTION to the pre-crash "FINAL" section above: the vrel base was
|
| 55 |
+
QUARANTINED after that section was written — all rk_vrnod* rows are reads on a
|
| 56 |
+
contaminated base and are WITHDRAWN from recommendation. The champion-base
|
| 57 |
+
survivor rk_rnod50s7 and its replication stand.
|
| 58 |
+
|
| 59 |
+
Pre-crash reads recovered from session (CSVs lost, re-running now): 3-way on
|
| 60 |
+
frontier base (vm7 values; order = A*logbag(vm7) + 0.20*wiener + C*ranknet):
|
| 61 |
+
- C=0.15 (A=0.65): -89.3 +/- 25.6, 25W/15L, swaps 9.62
|
| 62 |
+
- C=0.25 (A=0.55): -90.8 +/- 25.8, 27W/16L, swaps 9.56
|
| 63 |
+
Queue: confirm C=0.25; ranker with hot_90/oob_90+vmin7_rank inputs; aux_ho
|
| 64 |
+
4-way slot test; count checks; final table.
|
| 65 |
+
|
| 66 |
+
## Resume results (confirmed on restored env; all paired vs regenerated baseline 1760.1688)
|
| 67 |
+
| label | SCORE | delta | SE | W/L | swaps |
|
| 68 |
+
|---|---|---|---|---|---|
|
| 69 |
+
| rk_3w552025h | 1657.9 | -102.2 | 27.7 | 31W/11L | 9.62 |
|
| 70 |
+
| rk_3w552025 | 1669.3 | -90.8 | 25.8 | 27W/16L | 9.56 |
|
| 71 |
+
| rk_3wrepro (frontier base) | 1688.4 | -71.8 | 23.5 | 24W/16L | 9.58 |
|
| 72 |
+
| rk_rnod50s7 (champion base) | 1700.1 | -60.1 | 28.2 | 26W/10L | 9.52 |
|
| 73 |
+
|
| 74 |
+
- rk_3w552025h = NEW LANE BEST (3.69 sigma vs champion): vm7-logbag values, order
|
| 75 |
+
= 0.55*logbag(vm7) + 0.20*wiener + 0.25*RankNet(fvm7h), where fvm7h = champion
|
| 76 |
+
13 + vmin7_rank + hot_90 + oob_90 as RankNet inputs (wd's aux-ho pair helps my
|
| 77 |
+
ranker too). Marginal ON frontier base: -30.4 +/- 19.5 (17W/14L).
|
| 78 |
+
- rk_3w552025 (no hot/oob inputs) confirmed pre-crash read exactly: -90.8;
|
| 79 |
+
marginal on frontier base -19.0 +/- 14.6.
|
| 80 |
+
- COUNT AUDIT (phase-56 style) on rk_3w552025h: 41/48 scenarios count-identical
|
| 81 |
+
to champion; count-flat scenarios ALONE score -108.1 — win is pure ordering,
|
| 82 |
+
count shifts (7 scenarios, -2..+3) slightly DILUTE it. Clean.
|
| 83 |
+
- Win distribution: warmup -132, panel -113, nearend +1 — NOT warmup-concentrated.
|
| 84 |
+
- Running: seed-bag replication (seeds 100-106) of rk_3w552025h; aux_ho 4-way
|
| 85 |
+
slot test (control 0.50lb/0.20wnr/0.30auxho vs treatment 0.35/0.20/0.25auxho
|
| 86 |
+
+ 0.20ranknet, vm7 values anchored).
|
| 87 |
+
|
| 88 |
+
---
|
| 89 |
+
# LANE CLOSED (coordinator, 2026-08-23 ~09:10)
|
| 90 |
+
|
| 91 |
+
The agent was lost twice (detached-chain stall, then a machine-sleep API
|
| 92 |
+
kill) but its full run queue completed on disk; the coordinator finished
|
| 93 |
+
the adjudication from the CSVs. All pairings on the 48 per-origin
|
| 94 |
+
all-building sums vs baseline 1760.2.
|
| 95 |
+
|
| 96 |
+
## Final table
|
| 97 |
+
| label | composition (vm7 values; order weights) | delta | SE | W/L | notes |
|
| 98 |
+
|---|---|---|---|---|---|
|
| 99 |
+
| rk_4w | .35 lb + .20 wnr + .20 rknet + .25 auxho | -111.5 | 27.7 | 29/13 | 4.02 sigma, Wilcoxon .001 |
|
| 100 |
+
| rk_3w552025h | .55 lb + .20 wnr + .25 rknet(fvm7h) | -102.2 | 27.7 | 31/11 | count audit clean |
|
| 101 |
+
| rk_3w552025hb | same, seeds 100-106 | -93.5 | 27.9 | 26/15 | seed-block replication |
|
| 102 |
+
| rk_3w552025 | rknet without hot/oob inputs | -90.8 | 25.8 | 26/16 | |
|
| 103 |
+
| rk_3wrepro | .80 lb + .20 wnr (= ft frontier) | -71.8 | 23.5 | 23/16 | exact reproduction |
|
| 104 |
+
| rk_4wctl | .50 lb + .20 wnr + .30 auxho (NO rknet) | -65.7 | 22.7 | 24/16 | |
|
| 105 |
+
| rk_rnod50s7 | champion base + rknet w=.5 days | -60.1 | 28.2 | 25/10 | |
|
| 106 |
+
|
| 107 |
+
## Head-to-heads (the decisive ones)
|
| 108 |
+
- rk_4w vs ft frontier: -39.7 +/- 20.3 (1.96 sigma) but W/L 15/15,
|
| 109 |
+
Wilcoxon .131 — mean-supported ONLY, no sign-test support.
|
| 110 |
+
- rk_4w vs 4wctl (the RankNet increment): -45.8 +/- 20.9 (2.19 sigma),
|
| 111 |
+
Wilcoxon .101 — few scenarios carry it.
|
| 112 |
+
- rk_4wctl vs ft frontier: +6.1 +/- 18.8 — **aux_ho as an ORDER source
|
| 113 |
+
adds NOTHING beyond wiener+vmin7_rank** (settles the wd adoption-form
|
| 114 |
+
question: the raw composition's excess was value-channel).
|
| 115 |
+
- rk_4w vs wd raw vm7+auxho+wnr20: +14.9 +/- 38.0 — statistical tie.
|
| 116 |
+
|
| 117 |
+
## Verdict
|
| 118 |
+
**No adoption from this lane.** rk_4w is the campaign's strongest fully-
|
| 119 |
+
anchored vs-baseline number (all compositions injected at blend weight 1.0
|
| 120 |
+
= value-preserving by construction; swaps identical to the vm7 primary's
|
| 121 |
+
up to tie-handling), and RankNet's contribution replicates across disjoint
|
| 122 |
+
seed blocks — but its increment OVER the shipped frontier has no sign-test
|
| 123 |
+
support (15W/15L/18T), and RankNet is a torch dependency (MLP 32-16, Adam,
|
| 124 |
+
300 epochs x 7 seeds) the competition Docker is not known to carry, with
|
| 125 |
+
cross-version determinism risk. SHELF: rankers.py + run3w.py preserved in
|
| 126 |
+
agent_reports_bgkf/artifacts/; recipe = ThreeWay in run3w.py
|
| 127 |
+
(A=.35 lb / B=.20 wiener / C=.20 RankNet(fvm7h) / D=.25 aux_ho).
|