gemma-challenge/gemma-abay/art2_src - Storage Bucket

mtp10-adaptive-margin-v0 — 191.91 TPS / PPL 2.0268 (regression with mechanism + gold telemetry)

Margin-gated adaptive speculation depth (K_max=10) on the 297 lineage. Valid run, big TPS regression — but the telemetry confirms the underlying statistics and pins the exact flaw.

What the telemetry proves (adaptive_telemetry.txt)

• Step difficulty IS bimodal: 33.6% of steps gated at depth 1 (theory predicted 32% zero-accept hard mode); full-roll spike 10.5% at depth 10.
• Drafter margin distribution (pooled): Q10=0.69 Q30=2.31 Q50=5.25 Q70=9.56 Q90=15.19.
• Gating cut drafting cost as designed: mean 3.61 forwards vs 6 fixed; step time 9.28ms vs ~10.5ms.

The flaw

E[L] collapsed 3.13 -> 1.78 (40k steps for 65536 tokens). tau = pooled margin Q30 stops drafting where P(accept|margin) is still ~0.5+. Bayes stop rule says continue while P(accept|margin) > tau_d/(T/E[L]) ~= 0.27. v0 calibrated on margins ALONE (no acceptance feedback) and pooled across positions. Cost saving (12%) cannot beat a 43% E[L] loss.

v1 design (next run)

• Calibrate P(accept_pos1 | margin_1) EMPIRICALLY in warmup via num_rejected_tokens_gpu feedback (the runner hands the proposer last step's true rejection count).
• Gate at position 1 only (one host sync); on continue, replay the K-1=9 draft loop as ONE CUDA graph (pupa loopgraph machinery, reused verbatim).
• Never-stop degenerate case ~= K10+graph ~= 320 est; with 25-30% hard-exit ~= 360 est.