Thibaut Forest Claude Fable 5 commited on
Commit
c91595d
·
1 Parent(s): 87cd73d

Campaign 3 interim: autopsy overturns silent-death assumption (top-band precision 48%, near-miss band = 3593 late-days); alpha/day_q audits; team-repo intel; 30+ candidates refuted vs frontier

Browse files
Files changed (1) hide show
  1. agent_reports_bgkf/c3_findings.md +86 -0
agent_reports_bgkf/c3_findings.md ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Campaign 3 — beyond the frontier (deadline-day, user directive: explore
2
+ # ALL options, radical included; podium target <1400)
3
+
4
+ Baseline: champion 1760.2. Frontier (shipped, 3f9bf12): ft2 = 1688.4
5
+ (-71.8). All numbers below paired vs BOTH; adoption bar unchanged.
6
+
7
+ ## THE AUTOPSY (the day's central finding — reviews the core assumption)
8
+
9
+ Device-level autopsy of every in-window death in the frontier run
10
+ (/tmp/c3/autopsy.py, autopsy.csv):
11
+
12
+ - 454 visible deaths; recall 48% (217 caught, 237 missed); missed deaths
13
+ carry 3,934 late-days ≈ 70% of remaining cost. Included-but-late is
14
+ trivial (104 late-days) — timing is NOT a problem (day_q sweep: 0.0).
15
+ - **The "unrankable silent deaths" assumption is FALSE**: only 45/237
16
+ missed are voltage-silent (487 late-days). 2,832 late-days sit on
17
+ SIGNALED misses at median rank 5 of ~75 — rank-gap quartiles [2,3,8]
18
+ from an inclusion cutoff of ~2 per fold-fleet.
19
+ - Total count is exactly calibrated (456 swaps vs 454 deaths) but top-band
20
+ precision is ~48%: false includes occupy ranks 1-2 over real deaths at
21
+ 3-10. The near-miss band (gap 1-10) holds 211 deaths / 3,593 late-days —
22
+ larger than the whole distance to 1400.
23
+
24
+ False-include contrast (autopsy2): the false includes are genuinely the
25
+ most-degraded devices — v_last 2.436, 41 days below 2.45, tau 7.7d — they
26
+ plateau just above the 2.40 floor (36% do die, median 83d out, just past
27
+ the window). Near-missed deaths sit HIGHER (2.477) on steady declines and
28
+ cross fast. Every feature the model has measures level/slope; none
29
+ measures the SHAPE of the approach.
30
+
31
+ ## Assumption audits
32
+
33
+ - **alpha (report-only, quarantined)**: on bgkf, alpha 1.0 = -64.5 ± 38.3
34
+ beyond the frontier (swaps 9.6→11.4). Local optimum has MOVED from the
35
+ old dev-CV 0.8. Public refuted 1.5 hard (+527); 1.0 is untested
36
+ publicly. The team repo's risk_scale independently pins its optimum at
37
+ its tested boundary 1.2 — consistent cross-repo signal that incumbent
38
+ appetite is slightly lean ON HELD-OUT-style instruments. Count-coupled →
39
+ NOT shipped; user's call.
40
+ - day_q 0.02/0.10: exactly 0.0 vs frontier. Timing machinery optimal.
41
+ - Devices with NaN v_last dropped from CV planning rows: 0 in-window
42
+ deaths affected — no hidden cost.
43
+
44
+ ## Everything measured (delta vs ft2 frontier; negative = better)
45
+
46
+ | candidate | vs ft2 | verdict |
47
+ |---|---|---|
48
+ | w25 (wiener .25) | +3.8 ± 12.0 | plateau at .20-.25; keep .20 |
49
+ | ks3/ks12 (wiener shrinkage) | +6 | insensitive |
50
+ | gbm3/dth3 third order source | +5 / crash→n.a. | subsumed (again) |
51
+ | vm30/vm730/vm790 windows | +18..+48 | 7d is right |
52
+ | vproj / vstale / vsq / vfht | +17..+95 | dead |
53
+ | vranks (+r_vlast,r_tau) | -20.3 BUT swaps 9.0 | anchored twin +12.2 → count channel, dead |
54
+ | o3v (vmin7 as order, not feature) | +149 | feature form is essential |
55
+ | gap-dynamics (logistic) | +46..+89 | my shape features fail in linear form |
56
+ | ghgb (HGB + gap feats, anchored) | -44.7 ± 50.3 | n.s., noisy; the one live sub-thread |
57
+ | contagion (room/bldg/batch deaths) | +59..+92 | dead — no cross-device signal |
58
+ | team-feature port (13 feats, logbag) | +97..+167 | dead in our model form |
59
+ | team-feature port (HGB anchored) | +15..+23 | n.s. |
60
+
61
+ ## Team-repo intel (rul-mgt-zero, cloned; theirs ≈ 1493.65 PUBLIC vs our 1661.45)
62
+
63
+ - Their CV is IN-FLEET (48 scenarios, same devices) — their 1412.6 "best"
64
+ is NOT comparable to our building-held-out 1688.4. Public is the only
65
+ common scale: they are ~168 better.
66
+ - Their edge decomposes: (1) multi-level supervision — "does it cross L
67
+ within h days" for L up to 2.9V = abundant self-supervised events, with
68
+ threshold-relative features (raw_lead -223, rmargin_raw -193 in THEIR
69
+ harness); (2) poskeep_target sampling (-155.6; keep every positive +
70
+ focus budget on the queried cell — NOT portable, we already train on all
71
+ rows); (3) CP-SAT + stochastic planner optimizing all nine cost
72
+ components (overtime/daily/weekly limits — our tsroute is a heuristic);
73
+ (4) label mechanics: EOL = 7-day rolling median crossing (raw stream
74
+ leads it); 10-30C gate blanks 8.6% of days (their "wide" stream).
75
+ - Their gen-9 rttt_accel (compounding time-to-level) targets EXACTLY our
76
+ near-miss knee population. Their gen-10: state-dependent dV/dT (β = 3.59
77
+ + 22.03·(2.9−v) mV/°C, within-day identified) = internal-resistance
78
+ proxy, unexploited in both repos.
79
+ - CROSS-POLLINATION CANDIDATES (to the team repo — they lack them): our
80
+ WienerFHT value-preserving order blend (-50 here, model-free, sklearn),
81
+ vmin7_rank, and a building-held-out transfer check (their harness cannot
82
+ see cross-fleet failure; ours was built for it).
83
+
84
+ ## Wave X (in flight): crossing-label order sources (self-supervised
85
+ ## x25-in-42d, logbag + HGB forms), rttt_accel, raw-trajectory HGB,
86
+ ## cost-weighted order. Results appended on completion.