Thibaut Forest Claude Fable 5 commited on
Commit ·
c91595d
1
Parent(s): 87cd73d
Campaign 3 interim: autopsy overturns silent-death assumption (top-band precision 48%, near-miss band = 3593 late-days); alpha/day_q audits; team-repo intel; 30+ candidates refuted vs frontier
Browse files
agent_reports_bgkf/c3_findings.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Campaign 3 — beyond the frontier (deadline-day, user directive: explore
|
| 2 |
+
# ALL options, radical included; podium target <1400)
|
| 3 |
+
|
| 4 |
+
Baseline: champion 1760.2. Frontier (shipped, 3f9bf12): ft2 = 1688.4
|
| 5 |
+
(-71.8). All numbers below paired vs BOTH; adoption bar unchanged.
|
| 6 |
+
|
| 7 |
+
## THE AUTOPSY (the day's central finding — reviews the core assumption)
|
| 8 |
+
|
| 9 |
+
Device-level autopsy of every in-window death in the frontier run
|
| 10 |
+
(/tmp/c3/autopsy.py, autopsy.csv):
|
| 11 |
+
|
| 12 |
+
- 454 visible deaths; recall 48% (217 caught, 237 missed); missed deaths
|
| 13 |
+
carry 3,934 late-days ≈ 70% of remaining cost. Included-but-late is
|
| 14 |
+
trivial (104 late-days) — timing is NOT a problem (day_q sweep: 0.0).
|
| 15 |
+
- **The "unrankable silent deaths" assumption is FALSE**: only 45/237
|
| 16 |
+
missed are voltage-silent (487 late-days). 2,832 late-days sit on
|
| 17 |
+
SIGNALED misses at median rank 5 of ~75 — rank-gap quartiles [2,3,8]
|
| 18 |
+
from an inclusion cutoff of ~2 per fold-fleet.
|
| 19 |
+
- Total count is exactly calibrated (456 swaps vs 454 deaths) but top-band
|
| 20 |
+
precision is ~48%: false includes occupy ranks 1-2 over real deaths at
|
| 21 |
+
3-10. The near-miss band (gap 1-10) holds 211 deaths / 3,593 late-days —
|
| 22 |
+
larger than the whole distance to 1400.
|
| 23 |
+
|
| 24 |
+
False-include contrast (autopsy2): the false includes are genuinely the
|
| 25 |
+
most-degraded devices — v_last 2.436, 41 days below 2.45, tau 7.7d — they
|
| 26 |
+
plateau just above the 2.40 floor (36% do die, median 83d out, just past
|
| 27 |
+
the window). Near-missed deaths sit HIGHER (2.477) on steady declines and
|
| 28 |
+
cross fast. Every feature the model has measures level/slope; none
|
| 29 |
+
measures the SHAPE of the approach.
|
| 30 |
+
|
| 31 |
+
## Assumption audits
|
| 32 |
+
|
| 33 |
+
- **alpha (report-only, quarantined)**: on bgkf, alpha 1.0 = -64.5 ± 38.3
|
| 34 |
+
beyond the frontier (swaps 9.6→11.4). Local optimum has MOVED from the
|
| 35 |
+
old dev-CV 0.8. Public refuted 1.5 hard (+527); 1.0 is untested
|
| 36 |
+
publicly. The team repo's risk_scale independently pins its optimum at
|
| 37 |
+
its tested boundary 1.2 — consistent cross-repo signal that incumbent
|
| 38 |
+
appetite is slightly lean ON HELD-OUT-style instruments. Count-coupled →
|
| 39 |
+
NOT shipped; user's call.
|
| 40 |
+
- day_q 0.02/0.10: exactly 0.0 vs frontier. Timing machinery optimal.
|
| 41 |
+
- Devices with NaN v_last dropped from CV planning rows: 0 in-window
|
| 42 |
+
deaths affected — no hidden cost.
|
| 43 |
+
|
| 44 |
+
## Everything measured (delta vs ft2 frontier; negative = better)
|
| 45 |
+
|
| 46 |
+
| candidate | vs ft2 | verdict |
|
| 47 |
+
|---|---|---|
|
| 48 |
+
| w25 (wiener .25) | +3.8 ± 12.0 | plateau at .20-.25; keep .20 |
|
| 49 |
+
| ks3/ks12 (wiener shrinkage) | +6 | insensitive |
|
| 50 |
+
| gbm3/dth3 third order source | +5 / crash→n.a. | subsumed (again) |
|
| 51 |
+
| vm30/vm730/vm790 windows | +18..+48 | 7d is right |
|
| 52 |
+
| vproj / vstale / vsq / vfht | +17..+95 | dead |
|
| 53 |
+
| vranks (+r_vlast,r_tau) | -20.3 BUT swaps 9.0 | anchored twin +12.2 → count channel, dead |
|
| 54 |
+
| o3v (vmin7 as order, not feature) | +149 | feature form is essential |
|
| 55 |
+
| gap-dynamics (logistic) | +46..+89 | my shape features fail in linear form |
|
| 56 |
+
| ghgb (HGB + gap feats, anchored) | -44.7 ± 50.3 | n.s., noisy; the one live sub-thread |
|
| 57 |
+
| contagion (room/bldg/batch deaths) | +59..+92 | dead — no cross-device signal |
|
| 58 |
+
| team-feature port (13 feats, logbag) | +97..+167 | dead in our model form |
|
| 59 |
+
| team-feature port (HGB anchored) | +15..+23 | n.s. |
|
| 60 |
+
|
| 61 |
+
## Team-repo intel (rul-mgt-zero, cloned; theirs ≈ 1493.65 PUBLIC vs our 1661.45)
|
| 62 |
+
|
| 63 |
+
- Their CV is IN-FLEET (48 scenarios, same devices) — their 1412.6 "best"
|
| 64 |
+
is NOT comparable to our building-held-out 1688.4. Public is the only
|
| 65 |
+
common scale: they are ~168 better.
|
| 66 |
+
- Their edge decomposes: (1) multi-level supervision — "does it cross L
|
| 67 |
+
within h days" for L up to 2.9V = abundant self-supervised events, with
|
| 68 |
+
threshold-relative features (raw_lead -223, rmargin_raw -193 in THEIR
|
| 69 |
+
harness); (2) poskeep_target sampling (-155.6; keep every positive +
|
| 70 |
+
focus budget on the queried cell — NOT portable, we already train on all
|
| 71 |
+
rows); (3) CP-SAT + stochastic planner optimizing all nine cost
|
| 72 |
+
components (overtime/daily/weekly limits — our tsroute is a heuristic);
|
| 73 |
+
(4) label mechanics: EOL = 7-day rolling median crossing (raw stream
|
| 74 |
+
leads it); 10-30C gate blanks 8.6% of days (their "wide" stream).
|
| 75 |
+
- Their gen-9 rttt_accel (compounding time-to-level) targets EXACTLY our
|
| 76 |
+
near-miss knee population. Their gen-10: state-dependent dV/dT (β = 3.59
|
| 77 |
+
+ 22.03·(2.9−v) mV/°C, within-day identified) = internal-resistance
|
| 78 |
+
proxy, unexploited in both repos.
|
| 79 |
+
- CROSS-POLLINATION CANDIDATES (to the team repo — they lack them): our
|
| 80 |
+
WienerFHT value-preserving order blend (-50 here, model-free, sklearn),
|
| 81 |
+
vmin7_rank, and a building-held-out transfer check (their harness cannot
|
| 82 |
+
see cross-fleet failure; ours was built for it).
|
| 83 |
+
|
| 84 |
+
## Wave X (in flight): crossing-label order sources (self-supervised
|
| 85 |
+
## x25-in-42d, logbag + HGB forms), rttt_accel, raw-trajectory HGB,
|
| 86 |
+
## cost-weighted order. Results appended on completion.
|