RESULTS.md

Results — FLUX.2 klein 4B -> compressed student

Eval = held-out velocity-matching loss vs teacher (lower=closer; same fixed first-16 batch across all rows, so quant and surgery sit on ONE axis). wall=measured s/img @512/4-step batch-1 on A100; flop=estimated transformer-only ratio.

★★ 2026-06-14 — NVFP4 HEAD-TO-HEAD (image-space metrics, N=512) — separate axis

A matched, paired comparison on N=512 MJHQ-30k prompts, 512px, 4 steps, guidance 1.0, seed=idx, using image-space fidelity-to-teacher metrics (LPIPS/PSNR/FID vs the teacher's own outputs) + a PickScore/CLIP semantic check. This is NOT the velocity-loss axis above — do not compare the numbers across sections; compare only within this table. All models share the same Qwen3 TE + VAE (only the transformer quant varies). Full write-up: report/HEADTOHEAD_klein4b_nvfp4.md; raw numbers outputs/eval/h2h/metrics.json; speed outputs/nvfp4/benchmark_headtohead.json.

model	bits	PickScore↑	CLIP↑	LPIPS↓	PSNR↑	FID↓(teacher)	FID(real)	real-kernel speed
A teacher (bf16)	16	21.64	30.95	—	—	—	89.6	1.0× (0.464 s/img @512)
D plain NVFP4 r0	W4A4	21.62	30.95	0.2076	17.44	39.31	88.5	— (fake-q)
ours r128 (fake-q)	W4A4	21.61	30.99	0.1732	18.50	33.37	89.6	— (fake-q)
C ours r128 (REAL Nunchaku kernel)	W4A4	21.62	30.94	0.1668	18.71	33.54	90.1	1.76×@512 / 1.87×@1024, 12.6 GB
E BFL official FP8	W8A8	21.65	30.94	0.0798	23.02	18.81	89.6	— (needs TensorRT)
Δ low-rank branch (r0→r128)		≈0	≈0	−19.7%	+1.27 dB	−14.7%	~flat	—

Findings. (1) The SVDQuant low-rank branch helps at NVFP4 W4A4 — r0→r128: LPIPS −19.7%, PSNR +1.27 dB, FID-vs-teacher −14.7% (fake-q-vs-fake-q ablation agrees: −16.6% / +1.06 dB / −15.1%); reproduces the prior N=256 result at N=512. (2) The real Nunchaku FP4 kernel reproduces (slightly beats) the fake-quant (LPIPS 0.167 vs 0.173) → the gain holds on the deployed model. (3) No semantic loss — PickScore/CLIP flat across all incl. teacher. (4) BFL official FP8 is closest to teacher (8-bit; high-precision/low-speedup point) vs our 4-bit/2.5×-kernel point. (5) FID-vs-real is flat (88–90 incl. teacher) — tracks the klein-vs-MJHQ style gap, not quant; reported, not discriminating. (6) BFL official NVFP4 (model B) could not be run — cutlass tensor-core swizzled layout, needs BFL's TensorRT runtime (see report/HEADTOHEAD_klein4b_nvfp4.md §5); D (our plain r0) is the labeled controlled "plain NVFP4" stand-in, E is the real BFL baseline. No BFL number was fabricated.

★ 2026-06-13 — NVFP4 (Blackwell-native FP4) + first REAL kernel speed (Nunchaku)

Two things landed this day: (1) NVFP4 added to our fake-quant SVDQuant (flux2distill/svdquant.py) and swept; (2) the first real low-bit kernel speed numbers, by calling Nunchaku's compiled NVFP4 W4A4 GEMM directly. NVFP4 beats INT4 on BOTH quality and speed — it's the format for this box (and any Blackwell / RTX 50 / B200). Same eval axis as the 2026-06-10 cells below.

NVFP4 format: E2M1 elements (the 8 magnitudes {0,.5,1,1.5,2,3,4,6}·sign) + group-16 blocks + FP8(E4M3) block scales, applied to the 4-bit residual weights (and optionally activations). The low-rank branch stays bf16 — it is the high-precision error-/outlier-absorbing path. Knobs: WFMT={int,nvfp4}, AFMT={int,nvfp4,fp8} on scripts/12 (driver scripts/run_nvfp4_cell.sh).

Quality — NVFP4 sweep (klein-4B, plain+refine, no-smooth, same held-out velocity loss)

#	weights	acts	rank	wrecon mean	eval-loss	vs reference
1	NVFP4 g16	NVFP4 g16	32	0.0865	0.0390	—
2	NVFP4 g16	NVFP4 g16	64	0.0817	0.0364	INT4 W4A4 r64 = 0.0742 (2.0× worse)
3	NVFP4 g16	NVFP4 g16	128	0.0742	0.0303	INT4 W4A4 r128 = 0.0610 (2.0×); ≈ INT4 W4A8 champ 0.0297
4	NVFP4 g16	FP8 E4M3	64	0.0817	0.0204	INT4 W4A8 r64 = 0.0297 (prior overall best)
5	NVFP4 g16	FP8 E4M3	128	0.0742	0.0169 ★	−43% vs the 0.0297 INT4 W4A8 champion

NEW OVERALL QUANT CHAMPION: NVFP4-weights + FP8-acts, r128 = 0.0169. Dirs outputs/nvfp4_*.

Findings: (1) NVFP4 weights ≫ INT4 weights — ~2× lower loss at matched rank; driver is the finer group-16 + E2M1 float grid (unit test: outlier column 0.064 vs INT4-g64 0.115). (2) NVFP4 W4A4 r128 (0.0303) matches the INT4 W4A8 champion (0.0297) while keeping activations at 4-bit — full W4A4 at 8-bit-act quality. (3) FP8 acts buy more quality (0.0169) but cost speed (below). Visually the champions are teacher-indistinguishable on the text + hand probes (the quant-sensitive ones).

Speed — REAL kernels, klein-4B layer shapes, T=1536, RTX PRO 4500 Blackwell (sm_120)

scripts/23_nvfp4_kernel_bench.py calls Nunchaku's compiled svdq_gemm_w4a4_cuda (NVFP4 W4A4) at each of klein-4B's 5 distinct Linear shapes, summed over all 100 Linears. FP8 row = torch._scaled_mm (NOT Nunchaku — a proxy for the W4+FP8-act variant, which has no fused kernel on Blackwell).

path	ms/step	speedup vs bf16	deploys
bf16	73.8	1.00×	baseline
NVFP4 W4A4 r64 (real Nunchaku kernel)	26.8	2.75×	cell 2 (0.0364)
NVFP4 W4A4 r128 (real Nunchaku kernel)	29.7	2.49×	cell 3 (0.0303)
FP8 proxy r64	60.3	1.22×	cell 4 (0.0204)
FP8 proxy r128	60.8	1.21×	cell 5 (0.0169)

Matches the 9B end-to-end Nunchaku number (FP4 254 ms/step = 2.69× vs bf16 684; full pipeline 1.29 s/img @1024²/4-step, 24.95 GB). Rank tax (real W4A4 kernel): r64→r128 ≈ 11% (2.75×→2.49×) for the 0.0364→0.0303 quality gain — rank is a quality knob with a small, real speed cost.

End-to-end (real Nunchaku kernel, klein-4B Linears swapped to NVFP4 W4A4, rest bf16): bf16 → NVFP4 is 1.24× @512px / 1.18× @1024px with ~28% less VRAM (16.9→12.0 GB @512). Far below the 2.5× per-layer GEMM because attention is bf16 + O(N²) and dominates (more so at 1024px), VAE + text-encode are fixed bf16 overhead, and a Linears-only swap is unfused. The 9B Nunchaku FULL pipeline hit 2.69× precisely because it ALSO fuses attention/quant and is more GEMM-heavy — so the lever for a real 4B speedup is the fully-fused NunchakuFlux2 model (fused attention), not just quantized Linears. Rank tax is negligible end-to-end here (low-rank branch tiny vs the rest). scripts/24_nunchaku_e2e_speed.py.

★ FULLY-FUSED end-to-end (the real lever): converting klein-4B to the NunchakuFlux2Transformer2DModel fused path (fused_qkv_norm_rottary + attention_fp16 + W4A4 GEMM, no bf16 SDPA) gives 1.76× @512px and 1.87× @1024px (per-step 2.2×/2.29×, VRAM −26%: 18.6→13.7 GB @1024). This is the real number — fusing attention lifts the Linears-only 1.2× to 1.9×, and unlike the Linears-only swap it improves with resolution. The transformer step (2.3×) ≈ the per-layer GEMM (2.5×); end-to-end is capped by the 4B's VAE+text-encode (30% of its small total). scripts/25_fused_4b_speed.py (dummy weights — speed is value-independent; built from source via /workspace/build_nunchaku). Deployable single artifact (fast + correct weights) — ✅ DONE.

★ DEPLOYABLE NVFP4-fused klein-4B (correct weights + real kernel). Wrote our own NVFP4 weight exporter (flux2distill/nunchaku_export.py): per-Linear SVD low-rank + iterative refine → NVFP4 residual (E2M1 codes + per-group-16 FP8 wscales + per-tensor alpha, wcscales=1), packed into Nunchaku's MMA layout via their pack_weight/pack_micro_scale/pack_lowrank_weight. Convention validated (scripts/26): with pre-quantized acts the real kernel reconstructs our intended weight to 2.99%. scripts/27_convert_full_4b.py converts all 120 fused Linears (handling the qkv fusion + the single-block to_qkv_mlp_proj/to_out splits), loads them into NunchakuFlux2Transformer2DModel, and generates correct images on the real FP4 kernel — teacher-indistinguishable on the text ("THE OPEN PAGE" legible) and hand probes (montages outputs/nvfp4/deploy/). This is the full deployable model: NVFP4 quality (≈0.0303) + the fused ~1.9× speed + −26% VRAM in one artifact. (NB the fused packed-rotary path requires batch=1 generation.)

The quality↔speed fork (both deployable on Blackwell, pick one):

• Speed: NVFP4 W4A4 r128 → 0.0303 @ 2.49× (real Nunchaku kernel, loads today). r64 → 0.0364 @ 2.75×.
• Quality: NVFP4-W + FP8-A r128 → 0.0169, but FP8 compute caps at ~2× bf16 (FP8 tensor cores are half FP4 throughput) and has no fused kernel (1.2× measurable today). The tradeoff is physical.

Why INT4 is the wrong format on Blackwell (hardware, sourced): the 5th-gen tensor cores natively do FP4/FP6/FP8/INT8/BF16/… but dropped INT4 (Turing/Ampere/Ada had INT4 IMMA; sm_120 doesn't). Nunchaku ships INT4 for Turing/Ampere/Ada and NVFP4 for Blackwell; get_precision() returns fp4 here. Forcing svdq-int4 on this card → 1677 ms/step (slower than bf16). So INT4 W4A4 stays the deployable format for the huge RTX 20/30/40 base; NVFP4 is for Blackwell/B200 — complementary, one per generation.

⚠️ 2026-06-10 — box rebuilt AGAIN; eval axis SHIFTED; old numbers not comparable

The box was rebuilt (A100-80GB → RTX PRO 4500 Blackwell 32 GB, system python, torch 2.12+cu130, transformers 5.9→5.10.2, diffusers git Jun-1→Jun-10). Re-evaluating the UNCHANGED grid-best checkpoint (r64 plain+refine, zero missing/unexpected keys on load) gives 0.0325 (vel-relerr 0.1661) vs the recorded 0.0446 (0.1896) — a −27% instrument shift, NOT a model change. Cause: the 16-sample eval ran through different SDPA kernels, different Qwen3 prompt embeddings (transformers bump), and possibly different seeded σ/noise draws; with N=16 that easily moves the mean. Rule: numbers are only comparable WITHIN one box era. The 4×3 grid + mechanism-ablation tables below are the OLD axis (still internally consistent); every 2026-06-10+ experiment is compared against same-box re-evals. Re-anchored baseline: r64 plain+refine (α=0.5, smoothed) = 0.0325 (outputs/recheck_r64_plain_refine, montages in its eval/).

Montage read of that baseline on the new box (teacher|quant, 8 probes): texture/scene/large-text probes (storefront, lake, neon, spiderweb, fisherman) ≈ indistinguishable; the quant misses the counting probe (2 fried eggs vs the teacher's correct 3) and mangles the hand probe (folded/ merged fingers); chalkboard small-print is gibberish in BOTH (teacher limitation). Counting + hand are the sensitive probes to watch across the SMOOTH=0 ablation.

✅ SMOOTH=0 ablation (2026-06-10, new box) — CONFIRMED: drop SmoothQuant at W4A8

The #1 queued experiment, run at 3 ranks (plain+refine, 300-calib, all new-axis; each SMOOTH=0 build compared against a same-box re-eval of its smoothed α=0.5 twin). Dirs: outputs/abl_c300_r{16,32,64}_plain_refine_nosmooth vs outputs/recheck_r{16,32,64}_plain_refine.

rank	smoothed α=0.5 (re-eval)	SMOOTH=0	Δ eval-loss	wrecon mean (sm → ns)	ns wrecon max
16	0.0405 / rel 0.1855	0.0348 / 0.1719	−14.1%	0.1251 → 0.1041	0.1379
32	0.0362 / rel 0.1753	0.0331 / 0.1675	−8.6%	0.1193 → 0.1003	0.1286
64	0.0325 / rel 0.1661	0.0297 / 0.1588	−8.6%	0.1110 → 0.0944	0.1124

Findings:

1. SMOOTH=0 wins at every rank — new overall best: r64 plain+refine no-smooth = 0.0297. SMOOTH=0 is now the DEFAULT for all W4A8 builds.
2. The win grows as rank shrinks (−8.6% at r64/r32 → −14.1% at r16): the SVD branch was partly compensating smoothing damage, and with less capacity more damage shows through. Confirms the mechanism from the rank-0 ablation (smoothing widens the 4-bit weight spread).
3. No-smooth buys ~one rank tier for free: ns-r16 (0.0348) beats smoothed-r32 (0.0362); ns-r32 (0.0331) ≈ smoothed-r64 (0.0325). I.e. same quality at ~4% more compression.
4. Visual (8-probe montages): the hand probe — the most quant-sensitive — is visibly FIXED vs the smoothed baseline (plausible spread fingers at all 3 ranks vs mangled/merged at the smoothed r64 baseline). The counting probe (3 eggs) is still missed by EVERY quant (2 eggs), smoothed or not, all ranks — a rank/smooth-independent semantic drift. Large in-image text is always preserved.
5. wrecon improves ~15% at every rank and the worst layer drops to 0.11–0.14 (vs 0.15–0.26 smoothed) — exactly the predicted mechanism.

✅ α sweep (2026-06-10) — the dial has NO good setting; low α is the WORST

α ∈ {off, 0.1, 0.25, 0.5} × r{32,64}, plain+refine, 300-calib, new axis (dirs outputs/abl_c300_r{32,64}_plain_refine_a{10,25}; off/0.5 cells from above):

α	r64 eval-loss	r32 eval-loss	r64 wrecon	r32 wrecon
off	0.0297	0.0331	0.0944	0.1003
0.1	0.0380 ⚠ worst	0.0408 ⚠ worst	0.0999	0.1057
0.25	0.0317	0.0349	0.1007	0.1069
0.5	0.0325	0.0362	0.1110	0.1193

Identical ordering at both ranks (off < 0.25 < 0.5 ≪ 0.1) — a replicated U-shape:

1. No α beats off — every dose of migration hurts at W4A8; SMOOTH=0 is the permanent default.
2. α=0.1 is the worst point, not the safest: at low α the factor ≈ 1/max|W|^(1-α) (the weight-equalizing extreme). wrecon barely moves (0.0999 vs 0.1007 at r64) yet eval-loss jumps +20% — the rescale rotates residual quant error into output-relevant directions. The sharpest demonstration yet that weight-recon ≠ the eval objective; never tune α by wrecon.
3. Montages agree: the hand probe (cleanly rendered at SMOOTH=0) regresses to merged fingers at α=0.1. Counting (3 eggs) fails everywhere, α-independent.
4. Determinism note: a repeat eval of an unchanged cell reproduces its loss exactly (0.0331 → 0.0331), and rebuilt cells reproduce wrecon bit-for-bit — cell deltas on this box are real, not run-to-run noise.

The old-axis grid below retains its per-knob story (refine reliable, whitening unstable at 300-calib) but all its α=0.5 absolute numbers are superseded by no-smooth.

W4A4 ablation (2026-06-10, 3 cells) — the smoothing flip CONFIRMED; naive A4 not viable yet

All plain+refine, 300-calib, new axis. ABITS=4 (per-token dynamic 4-bit acts — same scheme as A8, just 16 levels). Dirs outputs/abl_c300_r{64,128}_w4a4_plain_refine_{nosmooth,a50}.

cell	eval-loss	vel-relerr	wrecon mean	vs W4A8 twin
r64 SMOOTH=0 (A8 recipe)	0.5103	0.6582	0.0944	0.0297 → 17× worse
r64 α=0.5	0.3885	0.5743	0.1111	smoothing helps +24%
r128 α=0.5	0.3060	0.5097	0.0992	rank helps +21%

Findings:

1. The smoothing flip is real and symmetric. At A8 smoothing hurt −9%; at A4 it helps +24%. Its value is purely a function of activation bit-width — at A4 the per-token 4-bit quant is destroyed by channel outliers (one outlier forces the whole token's scale → small channels round to 0), which is exactly what migration mitigates. Mechanism fully closed.
2. Rank matters more at A4 than A8 (r64→r128: −21%): the low-rank branch runs on FULL- precision activations, so extra rank routes more computation around the 4-bit bottleneck — at A4 rank is an activation shield, not just a weight-outlier absorber.
3. Naive W4A4 is NOT viable: best cell 0.3060 is ~10× the W4A8 best (0.0297). Visually: no-smooth A4 shatters in-image text ("THE OPEN PAGE" → "PINE OPEEN I AAGE"); α=0.5 restores readable text; r128 renders the storefront cleanly — but composition/anatomy stay broken (hand probe: two wrong-gesture hands). (No cross-axis comparison to the old surgery numbers.)
4. Next lever (queued): per-group activation quant — give each group of ~64 channels its own dynamic scale (the weight side already does this; Atom/QServe-style). Attacks the outlier problem per-token/per-group with zero weight-side cost — expected to beat the whole α dial.

W4A4 α-up sweep (2026-06-10, 3 more cells) — A4 has an INTERIOR optimum at α≈0.75

plain+refine, 300-calib, new axis. Dirs outputs/abl_c300_r{64,128}_w4a4_plain_refine_a{75,100}.

α	r64 eval-loss	r128 eval-loss	wrecon mean/max (r64 · r128)
off	0.5103	—	0.0944/0.1124 · —
0.5	0.3885	0.3060	0.1111/0.1486 · 0.0992/0.1344
0.75	0.2819	0.2080 ★ W4A4 best	0.1345/0.2585 · 0.1168/0.1960
1.0	—	0.2397	— · 0.1469/0.3423

Findings:

1. α ≈ 0.75 at (per-token) A4* — the curve descends to 0.75 then turns at 1.0: full flattening wrecks the weights (worst-layer 0.34) faster than it relieves the activations, even with r128+refine absorbing. Campaign symmetry: the optimal α tracks the bottleneck (A8 → off; per-token A4 → 0.75; nothing in between wins at either).
2. W4A4 improved 0.5103 → 0.2080 (−59%) via smoothing+rank alone — still ~7× the W4A8 champion (0.0297). Visual at the best cell: in-image text nearly clean (one corrupted glyph), coherent compositions return, but counting/gesture still wrong. Not deployable yet.
3. ⚠ Paper-spec correction (from re-reading SVDQuant): the paper's W4A4 uses per-GROUP activations (group 64, like its weights; NVFP4 = group 16) — our per-token activations reproduce their baselines (ViDiT-Q/MixDQ, which fail catastrophically exactly like our cells). So per-group acts isn't an enhancement, it's the missing piece of the actual SVDQuant W4A4 recipe → implemented below; step change confirmed.

✅ W4A4 per-group activations (2026-06-10) — the fix; W4A4 becomes viable

Implemented a_group (per-group dynamic act scales along channels; AGROUP env on scripts/12, recorded in quant_config.json). Unit test: one 60× outlier channel → per-token A4 rel-err 0.59, g64 0.11 (5×). 2×2 grid {r64,r128} × {SMOOTH=0, α=0.5}, AGROUP=64, plain+refine, 300-calib. Dirs outputs/abl_c300_r{64,128}_w4a4g64_{nosmooth,a50}.

W4A4 g64 acts	SMOOTH=0	α=0.5	(per-token, for scale)
r64	0.0742	0.0759	0.5103 (ns) / 0.3885 (a50)
r128	0.0610 ★ W4A4 best	0.0620	— / 0.3060 (a50)

Findings:

1. Per-group acts (g64) is THE W4A4 fix: −85% (0.5103 → 0.0742 at r64-ns) — far beyond everything the α/rank campaign bought combined. W4A4 best now 0.0610 (r128-ns), ~2× the W4A8 champion (0.0297) instead of 17×.
2. With per-group acts, smoothing is dead weight again (a50 slightly worse at both ranks) — the outlier problem belongs to quantizer granularity, not weight-side migration, at every bit-width. The W4A4 recipe converges to the same clean form as W4A8: plain SVD + refine, NO smoothing, per-group W and A, rank to taste.
3. This recipe is calibration-free (no smooth → absmax unused; no whiten → Gram unused; acts dynamic) — calib size/content is irrelevant to the current champions; it only matters if whitening returns or for future QAT. (NB the jasperai/monet calib set is NOT paintings — it's diverse photographic data, 260/400 captions mention text/signs — so content-narrowness was never a confound; "Monet" is just the dataset name.)
4. Qualitative (24 montages reviewed): per-group fixes gross text destruction, anatomy collapse and composition smearing entirely; residual A4 damage = symbolic precision — single-glyph text errors ("PAGE"→"PAYE"/"PACE"), counting flicker (2 vs 3 eggs, seed-fragile, non-monotone in loss), slight identity drift. no-smooth cells are visually cleanest (hands track the metric).
5. Act group-size ladder (unit test): per-token 0.60 → g128 0.17 → g64 0.13 → g32 0.10 → g16 0.077 — g16 (NVFP4's native group) is the queued next knob (sim-only for INT4; deployable as NVFP4 on Blackwell).

ACTIVE TRACK — W4A8 SVDQuant (fake-quant quality study; A100-era grid below)

Quantize all 100 block Linears: smooth → SVD rank-r low-rank (16-bit) → 4-bit residual + 8-bit per-token activations. smaller = quantized-weight bytes vs bf16 (a real low-bit kernel realizes this; fake-quant here measures quality only — no wall/flop yet on A100).

Three composable knobs (all W4A8, α=0.5, group=64, same fixed eval batch):

• plain = SVD of smoothed weight (base SVDQuant paper's headline derivation)
• whiten = activation-aware SVD minimizing OUTPUT error ‖X̂(Ŵ−L)‖ (ASVD/SVD-LLM idea; our add)
• +refine = iterative low-rank refinement, re-fit L to absorb 4-bit quant error (SVDQuant §4.2)

FULL 4×3 GRID (2026-06-01) — every method × every rank, fixed 300-img calib

Closes the old "L-shape, not a grid" gap: each cell built one-at-a-time on the SAME 300-image calib from data/monet_cache (latents), so all 12 are directly comparable. eval velocity-loss (lower=closer to teacher); wrecon = mean weight-recon rel-err. Dirs: outputs/abl_c300_r{R}_{variant}.

rank	smaller	plain	plain+refine	whiten	whiten+refine
16	3.67×	0.0620	0.0655	0.0656	0.0556
32	3.59×	0.0586	0.0574	0.0545	0.0476
64	3.43×	0.0487	0.0446 ← grid best	0.0588	0.0451

Best per rank: r16 whiten+refine 0.0556; r32 whiten+refine 0.0476; r64 plain+refine 0.0446 (whiten+refine 0.0451 ≈ tie). Overall best = r64 plain+refine 0.0446 @ 3.43×.

Full per-cell metrics (all 12, real measured)

eval-loss = held-out velocity-matching loss · vel-relerr = velocity field rel-L2 vs teacher · wrecon = mean weight-recon rel-err (100 layers) · orecon = mean output-recon rel-err (what whitening optimizes; only computed for the whitened cells).

rank	variant	smaller	eval-loss	vel-relerr	wrecon-relerr	orecon-relerr
16	plain	3.67×	0.0620	0.2235	0.1269	—
16	plain+refine	3.67×	0.0655	0.2297	0.1251	—
16	whiten	3.67×	0.0656	0.2299	0.1290	0.0733
16	whiten+refine	3.67×	0.0556	0.2117	0.1273	0.0680
32	plain	3.59×	0.0586	0.2174	0.1224	—
32	plain+refine	3.59×	0.0574	0.2151	0.1193	—
32	whiten	3.59×	0.0545	0.2095	0.1257	0.0719
32	whiten+refine	3.59×	0.0476	0.1959	0.1226	0.0646
64	plain	3.43×	0.0487	0.1980	0.1163	—
64	plain+refine	3.43×	0.0446	0.1896	0.1110	—
64	whiten	3.43×	0.0588	0.2177	0.1209	0.0695
64	whiten+refine	3.43×	0.0451	0.1907	0.1155	0.0595

All three metrics track together (lower wrecon/orecon ↔ lower eval-loss) within a rank, with the notable exception that plain+refine attains the lowest wrecon at each rank yet only the lowest eval-loss at r64 — confirming weight-recon ≠ the eval objective (refine minimizes weight error, which only aligns with the velocity loss once rank is high enough). Build/eval logs: tmp/abl_r*_*.{build,eval}.log.

Findings (these OVERTURN the prior L-shape conclusion that "each upgrade compounds"):

1. Refine is the reliable workhorse — it helps at every rank/metric EXCEPT r16-plain (0.0620→0.0655, where minimizing weight error overfits and drifts from the output-optimal point). Every rank's best variant uses refine.
2. Whitening ALONE is unreliable at 300-calib — non-monotonic in rank: hurts r16 (0.0656>0.0620), helps r32 (0.0545<0.0586), hurts r64 (0.0588>0.0487). It's overfitting the noisy 300-image Gram; at high rank it fits more directions to bad stats and generalizes worse than plain.
3. Strong whiten×refine interaction — refine runs IN the whitened metric, correcting whitening's overfitting. At r16 neither upgrade alone helps yet together −10% (0.0620→0.0556). At r32 they stack to the row best (0.0476). At r64 whitening adds nothing over plain+refine (0.0451 vs 0.0446).
4. At high rank, skip whitening — r64 plain+refine (0.0446) beats/ties everything and needs no Gram (simpler + faster build). Whitening only earns its keep at moderate rank (r32) or paired w/ refine.
5. Sweet spot is a choice: r64 plain+refine 0.0446 @ 3.43× (max quality, simplest) vs r32 whiten+refine 0.0476 @ 3.59× (a bit more compression). Both ~4–5× below the surgery frontier (0.231).

These 300-calib numbers land close to the prior 100-calib report (r32 wr 0.0476 vs 0.0494; r64 wr 0.0451 vs 0.0454) but the per-knob story is different — whitening's instability is the headline. Montages for all 12 cells (8 probe prompts each) under outputs/abl_c300_*/eval/.

OPEN — whitening needs a higher-calib re-test. Its non-monotonic, often-harmful behavior at 300 calib is consistent with Gram under-estimation. The deferred 2000-image calib re-sweep (plan.md §5) is a follow-up: does whitening become reliably beneficial with richer activation statistics? Full methodology + math (whitening, Cholesky→eigh, refinement) in report/QUANT_REPORT.md.

Mechanism ablation — what each piece buys (2026-06-01) · ⚠️ SmoothQuant HURTS at W4A8

Stripping the pipeline down to isolate each mechanism (300-calib, same eval). smaller ≈ 3.76× for the rank-0 rows (no low-rank bytes).

config	eval-loss	vel-relerr	wrecon mean / max	note
RTN W4A8 (no smooth, no SVD, s=1)	0.0573	0.2149	0.1112 / 0.1504	naive floor — yet beats the next two
SmoothQuant W4A8 (rank-0, α=0.5, no SVD)	0.0729	0.2424	0.1356 / 0.2633	smoothing makes it WORSE
+ SVD rank-16 plain (α=0.5)	0.0620	0.2235	0.1269 / 0.198	grid plain r16
+ SVD rank-64 plain	0.0487	0.1980	0.1163 / 0.155	grid plain r64
grid best: r64 plain+refine (α=0.5)	0.0446	0.1896	0.1110 / —	best so far

Headline finding: SmoothQuant at α=0.5 is actively harmful for W4A8. Removing it (RTN, s=1) beats SmoothQuant rank-0 by −21% (0.0729→0.0573) AND beats the smoothed SVD cells at r16/r32. Mechanism: SmoothQuant migrates outliers out of activations into weights — a win only when activations are the hard part (low-bit A4). At W4A8 the 8-bit activations are already easy, so migration buys nothing there and widens the weight distribution, making the 4-bit weight quant harder (worst-layer wrecon 0.15→0.26). Implication: the entire α=0.5 grid is mis-tuned — the SVD branch was partly compensating for smoothing damage. Re-running with no-smooth / low-α should beat 0.0446. Next: run the best config with SMOOTH=0, then an α sweep {0, 0.25, 0.5}. Knob: SMOOTH=0 env on scripts/12 (s=1). Montages: outputs/abl_c300_r0_nosvd{,_nosmooth}/eval/.

SHELVED TRACK — block surgery (depth-prune single blocks → surrogates → distill)

config	params	smaller	wall	flop	eval-loss	status
teacher 4B (baseline)	3.876B	—	1.00x	1.00x	—	reference
v1 per-token drop-12 (SVD-energy)	2.441B	37%	1.45x	1.64x	—	COLLAPSED (non-functional)
per-token drop-6 (importance)	3.158B	19%	1.19x	1.24x	0.308	ok, soft
linattn drop-6 (simple elu+1)	3.177B	18%	1.15x	1.23x	0.253	ok
linattn drop-6 +RoPE+conv+warmstart	3.177B	18%	1.15x	1.23x	0.231	BEST QUALITY
linattn drop-8 +focused+FFN(all)	2.995B	23%	1.20x	1.28x	0.269	best colors/local
linattn drop-10 mixed (4 FFN+6 light)	2.737B	29%	1.26x	1.42x	0.322	KILLED ~step200