Three failure modes of IGUANA training — isolated, with their fixes

Dotted line = iter 591 (epoch 2 = every prompt's second visit, K=1). The unseeded run breaks its climb and runs with ~3× the gradient noise from there on; the seeded run sails through. Contained (not catastrophic) because the other two fixes are on.

Red validity starts near 1.0 only because it counts post-resampling survivors (inflated); the model beneath is being pushed away from every sample — no climb phase, monotone erosion, dead by iter ~180. Switch the metric to Loss: negative from iteration 1 (the signature).

Climb-then-crater: concentrating on covered good sequences first helps (validity 0.1 → 0.85 by ~iter 100) — but the estimator never sees the mass q stops covering, so the support ratchet runs one way, into a near-deterministic degenerate proposal (0 by ~iter 160) with the gradient norm running away. The anchored reference stays flat indefinitely. (The objective itself is mean-seeking — this is estimator failure, not objective preference; see the Mode-3 card above.)

Plain-sampling eval (ess=0) — the proposal-quality metric. Both stable arms improve (validity +30% rel., pass@1 +0.05–0.06 behind these numbers); the SMC-eval (ess=0.9) curve is flat at the posterior value as expected (shown dashed where complete). Early: iter 2000 ≈ 3.4 epochs of 591 prompts — the 10-prompt probes needed ~100 epochs to peak. Frozen ≈ seeded, within 18-prompt noise. Red dash: the skyline on these 18 prompts — E_σ[correct] = 0.244 [0.148, 0.347].

The main-table numbers. Left/top: WACC (macro mean, prompt-bootstrap CIs) against the skyline — the posterior ceiling E_σ[correct] = 0.323 [0.284, 0.363] on these exact 254 prompts (macro; pooled ratio-of-sums variant: 0.377 [0.331, 0.422]; 59/254 prompts have zero correct-given-valid base samples). Base already sits at 0.302: the no_error target leaves ~2 points of total headroom, and both trained arms land inside the skyline CI — the target, not the optimizer, binds. Bottom: pass@K of the proposal itself (ess=0 particles are iid draws; unbiased estimator, identical prompts for all arms). pass@1 rises 0.192 → 0.266 (+39% rel.) at ~3.4 epochs, β=0.02 > β=0.1 at every K, The deep-k picture (two independently-seeded waves pooled, 150 draws/prompt, 222 common prompts — one shard refilling): EARLY checkpoints (@2000) dominate at small k (pass@1 +43% rel.) but converge to a statistical tie with base by k≈100–150. The PEAK checkpoint changes the story: β=0.1 @10000 dominates base at EVERY k — pass@50 0.715 vs 0.687, pass@100 0.766 vs 0.739, pass@150 0.793 vs 0.770 (+2.3 pts ≈ 5 prompts). Contrast with RLVR (Yue et al. 2504.13837, significant cross-under at large k on code): at its peak, stabilized inclusive-KL training shows coverage GAIN through the deepest k measured — concentration and coverage, not a trade. NEW: the CORRECTNESS-potential arm peaks at WACC 0.349 / pass@1 0.323 at iter 3000 — the FIRST arm above the no_error skyline, and the best pass@1 measured (+68% rel. over base) — then declines (0.319/0.247 @7000) as the training-batch solve rate collapses; see the per-epoch chart below. Recipe: identical to the β=0.02 no_error arm — KL anchor β=0.02 to the base model, ess=0, tempered, frozen Z — plus the terminal reward (real DS-1000 test execution lands multiplicatively on the weights, so the target is p·φ_noerr·correct) and Z re-measured for THIS target (frozen at P(valid AND correct), mean 0.195 — freezing the no_error rates would miscalibrate the baseline b(x)=Z(x) by ~2.5×). The anchor matters twice here: correctness is a much sparser reward (Ẑ≈0.2 vs ~0.5), and sparser targets have higher stability thresholds (the LCB lesson) — β=0.02 holds. Its own skyline is 1.0, not 0.323: the richer target raises the ceiling, and the probe-scale degradation was the 10-prompt memorization artifact. Trajectory: complete 254-prompt cells at iters 1000–7000 are plotted; @4000/@6000 cells still partial. The red dash on the pass@1 group marks the skyline: the pass@1 of a proposal that exactly matches the posterior; both trained arms are below it here (0.266 vs 0.323), i.e. concentration has not yet reached posterior sharpness at 3.4 epochs.

Both anchor strengths show the same rise–peak–decline shape; β sets the timing and the height. β=0.02 peaks at ~iter 2000 (pass@1 0.266) then recedes; β=0.1 needs 10,000 iterations to reach the same pass@1 but at a higher peak everywhere else — WACC 0.324 (touching the skyline 0.323) and pass@50 0.717 (+4.8 over base, a genuine support gain) — before receding by 19,500. Practical rule: weak anchor if compute-limited, strong anchor + checkpoint selection if not. Frozen-Z at iter 2000 posts the highest pass@1 of any measured cell (0.283, on 222/254 prompts — one shard pending) before declining by 3500 — same shape, possibly a slightly higher/earlier peak than seeded. Curves now show the full peak-bracketing resolution (β=0.02 every 500 iters around its peak; β=0.1 every 2000–3000 across its rise and decline).

Training against the CORRECTNESS potential (terminal test execution; skyline = 1 by construction) at 10-prompt scale briefly matches the no_error arm (0.300 @500) then degrades to the held-out asymptote (~0.24) and stays there — while the no_error arm (β=0.02) holds its 0.30 plateau to 12,500. Interpretation: with 10 prompts and a sparse reward (Ẑ≈0.2), the correctness signal teaches the solutions to those ten problems (memorization) instead of a transferable skill. The full-split correctness arm (591 prompts, running) is the decisive test — the memorization shortcut doesn't exist at that scale.

Below: the FULL-SPLIT correctness arm's training-batch solve rate (windowed mean of #particles passing valid∧correct per batch; dashed line = the frozen-Z prior rate 0.195, i.e. the base model's solve rate on the training prompts). Because the terminal reward measures correctness directly, this curve aggregates each FULL EPOCH (591 iterations = every training prompt exactly once), so points are directly comparable — no prompt-composition noise. The full β=0.02 arc is now a rise–collapse–RECOVERY: 0.161 → 0.188 (epoch 7, iters ~3500–4100, matching the held-out peak WACC 0.349 @3000), then a full collapse to 0.000 for four consecutive epochs (10–13, iters ~5300–7700 — the held-out decline to 0.319 @7000 is this collapse seen from outside), then a spontaneous recovery back to 0.169 by epoch 17. The recovery is the anchor doing its job: β=0.02 keeps the collapsed proposal within reach of the prior, so coverage regrows — unanchored arms never come back. The β=0.1 twin runs lower and slower, had a shallow dip at epochs 3–4, and is climbing steadily: 0.079 → 0.105 at epoch 10, still rising. (A prior-rate reference line is omitted for now: the rollout-measured prior (0.195) used 400-token generations vs the run\u2019s 200, so it is not directly comparable — a truncation-matched prior can be added later.)

All ess=0 runs are plain importance sampling, so a new runner replaces the genlm SMC E-step with one direct vLLM generate + two scoring forwards (uw = φ·p_T/q_T, adapter on/off); the IGUANA weight math is the same shared code (~13s/iter vs ~35s; an epoch ≈ 2.1h). On it, three arms differing ONLY in the stabilization mechanism: b002-IS — the production anchored recipe verbatim (uw-clip 1.0, w-cap ±0.5, grad-clip, KL anchor β=0.02); timv — Beta-shrinkage Ẑ(x) (μ₀=0.5965 = rollout grand mean; n₀=100, m₀=50) + RELATIVE use-time clip uw ≤ 10·Ẑ(x) (unclipped uw feed the running Ẑ; bounds the centered weights to [−0.1, 0.9] by construction), NO absolute clip, NO w-cap, NO grad clip, NO ANCHOR; MSC — Markovian Score Climbing (CIS kernel): weights SELECT one training sample per prompt, never scale a gradient — no normalizer, no baseline, no clips, NO ANCHOR. Below: per-EPOCH training validity (591 iters = each prompt once). Both anchor-free arms climb monotonically through 9–10 epochs — an order of magnitude past every unanchored collapse we measured (probe @~160, LCB @~600) — and MSC runs at or above the anchored control.

Held-out (254 prompts, plain-IS eval backend; its base cell reproduces the genlm backend within noise: wacc 0.322 vs 0.302, pass@k within ~1.6pt). MSC's trajectory so far: pass@1 rises to 0.280 @3000 (+44% rel. over the same-backend base 0.195) with WACC pinned at the no_error ceiling (~0.31) — anchor-free concentration gains matching the anchored recipe. The honest counterpoint: pass@50 drifts DOWN 0.685 → 0.665 → 0.646 → 0.634 (@0→1000→2000→3000) — no collapse, but a steady coverage tax of ~1.5pt per 1000 iters at k=50 that the anchored arm did not pay (its @2000 pass@50 stayed at base level). timv cells @500–@3000 are mid-eval and will backfill here.

Single-knob variants of the reference recipe, best checkpoint each (iters ≤ 2000, except the frozen-Z twin whose checkpoint cadence is 2000/4000/6000 — its best is iter 4000). Whiskers: 95% CI from the macro-mean prompt bootstrap over the 12 held-out prompts (base model: plain [0.093, 0.312], SMC [0.077, 0.296]). With only 12 prompts the intervals are wide and all arms overlap each other and the base model — this panel ranks the knobs; the full-split runs are the confirmatory eval. Dashed lines mark the base model (iteration 0) on the same held-out 12 (plain 0.195, SMC-eval 0.178 — every arm clears both) and the skyline: the posterior ceiling E_σ[correct] on these exact 12 prompts, 139/594 = 0.234 (pooled ratio-of-sums; bootstrap 95% CI [0.116, 0.346]; 4 of 12 prompts have zero correct-given-valid base samples). β=0.02's plain 0.304 sits ABOVE the skyline — legitimate at finite M: plain-eval WACC at M=10 is proposal-dependent, and a proposal with more correct mass than the posterior beats the asymptote. The no_error target is exhausted here. Weakening the anchor to β=0.02 is the biggest single win — its plain-sampling WACC (0.304) sits essentially at the no_error target's posterior ceiling (≈0.31 measured on the full 254), i.e. this target is nearly exhausted; further gains need the correctness potential. Warm-starting from the offline SFT (anchored to the init, not the base) gives the best SMC-eval number and composes offline + online training.

Where the gain actually lives. Red dashes: each prompt's own skyline — E_σ[correct] measured on its base rollouts (correct-given-valid, N=100). The four zero-skyline prompts (319, 388, 42, 799) stay at ≈0 after fine-tuning: training toward the posterior cannot manufacture correctness the posterior does not contain (799's +0.012 is the lone exception — the tuned proposal occasionally emits a correct program the base produced 0/100 times). All macro improvement comes from the 8 solvable prompts, and on 5 of them the tuned proposal ends above its own skyline in plain eval (909: 0.771 vs 0.667; 770: 0.624 vs 0.297) — legitimate at finite M, where WACC is proposal-dependent. Regressions are mild and mode-specific: 728 in plain (started far above its skyline), 875 in SMC eval.

One representative particle per regime. The collapse row is not an illustration — it is the verbatim output of the unanchored run after the entropy ratchet completes.

df = df.set_index('dt')
all_dates = pd.date_range(df.index.min(), df.index.max())
result = df.groupby('user').val.reindex(all_dates, fill_value=233)
result = result.to_frame(name='val').reset_index()
df = df.set_index('user')
result = pd.concat([df,result])

result = [tf.compat.as_text(x_) for x_ in x]</code>

result = (df
              .apply(pd.Series.value_counts)
              .unstack()
              .astype(float)
      )

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

A controlled simultaneous A/B on the binding LCB target (strict public tests; prior rate Z̄=0.51): identical recipes, hardware, and data — the ONLY difference is the KL anchor. The unanchored arm collapses by iter ~300; the β=0.02 arm survives to ~600 and then collapses too, sitting degenerate through iter 2800+ with no recovery — support abandonment is absorbing in both. UPDATED CONCLUSION: the anchor delays collapse, but the required dose is TARGET-DEPENDENT — on DS-1000 (denser reward, 200-token sequences) the stability boundary is β ∈ (0.005, 0.01]; on LCB-strict (many Ẑ≈0.03 prompts, 1024-unit sequences → faster KL drift) it lies above 0.02. This matches the support-floor scaling (floor ∝ e^−1/β against a domain-dependent leak rate). RESOLVED: the β=0.1 arm is STABLE AND LEARNING — validity climbs from the prior rate 0.51 to ~0.55 by iter 1000, through both predecessors’ death windows, with calm gradients. The LCB stability threshold sits in (0.02, 0.1]; held-out evals of its checkpoints are running. (Earlier LCB probes: bare arm died @~700, frozen-Z-only @~300.)