ablations/README.md

# Ablation Results — CSV index

Prose narrative of the Phase-0 findings (baselines, dense floor, acceptance, temporal, fine-graining, 1e18): [FINDINGS.md](FINDINGS.md).

Consolidated result tables from the temporal-MoE ablation program, gathered from the `a6000` / `h100` orchestration branches (comms + code excluded). One row per file: a one-line description and the branch commit(s) whose result-distillation produced it.

\* Quality numbers: end-of-training **test-set** evals (`ce_test_final`) are the canonical series (protocol-consistent across all runs); earlier write-ups sometimes quoted during-training val evals ~0.004 CE higher — both series preserved in `phase0_isoflop_points.csv`.

**Provenance** — `a6000@<sha>` / `h100@<sha>` is the commit on that branch that wrote the source data (read that commit's message for the exact run); files split across both boxes cite both. Pre-consolidation and pre-combine versions of every file live in this branch's earlier commits (`7e89690` consolidate → `2934c9d` combine → `abc63c3` fold/rename → `56758d4` seed-merge).

| CSV | What it is | Generated by |
|---|---|---|
| `alignment_cells.csv` | ΔBPB-vs-dense-baseline and A3 routing-overlap per ablation cell across paradigms. | a6000@5e3614ac |
| `alignment_frontier.csv` | Test-BPB vs A3-cacheability frontier per temporal cell (the diversity↔alignment tradeoff). | h100@d9de0a30 |
| `anomaly_pred.csv` | Predictability of swap 'anomalies' (positive frac, logistic AUC, ridge R²) at several prefetch horizons. | a6000@09530444 |
| `b1_eval_bpb.csv` | B1 eval: test-BPB vs baseline (Δ) across τ_ρ / EMA-β router configs. | a6000@85c6f4e7 |
| `block_replay.csv` | Block-wise (chunked) routing replay — retained mass, coverage, union, turnover per block boundary. | a6000@97de0d04 |
| `downstream_evals.csv` | Downstream lm-eval task accuracies per budget/checkpoint/model/task. | a6000@10a8bf6a |
| `e1_swap_rate_by_layer.csv` | Mean rolling-residency swap rate + p95 swap-burst length per layer. | a6000@cd588fd6, h100@cd588fd6 |
| `e1_victim_cache_hitrate.csv` | Expert-reload hit-rate vs victim-cache size. | a6000@cd588fd6, h100@cd588fd6 |
| `e2_streamed_diversity.csv` | Streamed expert diversity — union / effective-experts per model. | a6000@cd588fd6, h100@cd588fd6 |
| `e3_mass_vs_set_consistency.csv` | Mass- vs set-consistency of routing across positions. | a6000@cd588fd6, h100@cd588fd6 |
| `e4_swap_vs_retained_mass.csv` | Swap rate vs retained routing mass across τ thresholds. | a6000@cd588fd6, h100@cd588fd6 |
| `e5_eviction_policy_headroom.csv` | Set/mass coverage across eviction policies (headroom). | a6000@cd588fd6, h100@cd588fd6 |
| `e6_per_layer_ranking.csv` | Per-layer hit-rate / swap-rate / expert-lifetime ranking. | a6000@cd588fd6, h100@cd588fd6 |
| `e7_demand_smoothing.csv` | EMA-β demand smoothing — swap rate vs set/mass coverage. | a6000@cd588fd6, h100@cd588fd6 |
| `e8_document_boundary.csv` | Routing churn at document (EOD) boundaries. | a6000@cd588fd6, h100@cd588fd6 |
| `expert_selection_per_token.csv` | Per-token expert selections (full-MoE vs temporal) for the 8/15/38M models (model in active_params_M col). | a6000@53aa24ba, h100@53aa24ba |
| `flame192_leftflank_1e18.csv` | 1e18 isoFLOP LEFT-flank (hidden-192/~26M) 5-cell final CE panel: dense + {MoE,temporal}×{coarse,fine}. | a6000@ee73c64d, h100@5ed5a240 |
| `flame38m_overnight_seeds.csv` | Overnight seed extension (h100 split): fine-pair seed-2 (temporal+MoE), local dense floor, coarse-MoE seed-3 — end-of-run full-val + test CE/BPB. | h100@5a30342a |
| `flame38m_overnight_seeds_a6000.csv` | Overnight coarse-temporal seed-3 on the a6000 split (same schema). | a6000@e0a98204 |
| `flame38m_1e18_cells.csv` | 1e18 38M-middle 5-cell panel finals (dense/MoE/temporal × coarse/fine); coarse per-seed curves in t18_1e18_curves.csv. | temporal-moe-impl@66f786b7, h100@60b0e351/e53596d6 |
| `flame512_1e18_rightflank.csv` | 1e18 isoFLOP RIGHT-flank (hidden-512/98M) 5-cell final CE panel. | h100@d4827a1e |
| `flame512_1e18_rightflank_curves.csv` | Per-iteration training curves for the 1e18 right-flank cells. | h100@d4827a1e |
| `karen_center_replay.csv` | 'Karen' (aux-free momentum) center-of-mass replay — A3/eff/union/swap per seed. | a6000@91855c19 |
| `karen_promotion_s2_1e17.csv` | Karen s2@1e17 promotion checkpoint — val/test BPB + residency stats. | h100@ee5b26db |
| `learned_locality_vs_scale.csv` | Temporal vs full-MoE routing overlap ('learned locality') vs model scale. | a6000@cd588fd6, h100@cd588fd6 |
| `mechinterp_floors.csv` | Null/floor AUC baselines for the locus features (incl. the softmax-routing baseline model). | a6000@3ce9b63e/5125733c |
| `mechinterp_freerider.csv` | 'Free-rider' expert stats — distinct experts per sequence, tokens per expert. | a6000@1f416ef5 |
| `mechinterp_lens.csv` | Logit-lens effective-vocab & semantic dispersion per (layer,expert), static+weighted variants, incl. softmax baseline. | a6000@3ce9b63e/6a4d7bf0 |
| `mechinterp_locus.csv` | Token-AUC vs context-AUC 'locus' of expert specialization per (layer,expert); variant col = base/kfull/kwin k-config. | a6000@29c500ab/3ce9b63e/41c8d18a/804e10b6/dd4e900e |
| `mechinterp_logitlens.csv` | Unweighted logit-lens effective-vocab + semantic dispersion per (layer,expert). | a6000@53c381bd |
| `mechinterp_structural.csv` | Structural routing stats (PR, generalist frac, router entropy, eff-rank, centroid dist) per model. | a6000@3ce9b63e |
| `mechinterp_structural_1e19.csv` | Structural stats replicated at 1e19 (3 cells, same schema + budget col). | h100@8aef6c4c |
| `mechinterp_locus_1e19.csv` | Token-vs-context locus probes at 1e19 (per layer/expert AUCs; iid null 0.500-0.502, shift null 0.502-0.505 = residency autocorrelation). | h100@696da366 |
| `mechinterp_demand_1e19.csv` | Causal history-only demand-prediction AUC at 1e19 (0.604 moe / 0.926 t-coarse / 0.979 t-fine). | h100@fea7d5fa |
| `mechinterp_lens_1e19.csv` | Data-weighted logit lens at 1e19 (per layer/expert eff-vocab; static no-signal ref ~30k of 50k at this scale). | h100@fc47dd0c |
| `unmask_eval_1e19.csv` | Constraint swap at 1e19, eval-only: unmask temporal (+0.21/+0.20 BPB) and impose on moe_coarse (+0.43 BPB). | h100@54c7d767 |
| `momr_replay.csv` | Log-ratio momentum ('momr') replay — set/mass coverage, swap rate, union across β. | h100@256ae85c |
| `oracle_a3.csv` | Oracle A3-cacheability (with induced swap rate) vs prefetch horizon per k. | a6000@74d1d62a |
| `oracle_horizon_map.csv` | Oracle A3 gain (Δ over baseline) vs prefetch horizon per k. | a6000@37ef31f8 |
| `p2_fair_1swap.csv` | P2 systems: fair single-swap B=1 decode timing breakdown (router/GEMM/DMA/overlap). | a6000@f6e740aa |
| `p2_swap_bench.csv` | P2 swap microbench — compute/swap/DMA timings, %DMA hidden, VRAM per resident-set R. | a6000@e4c1a1ff |
| `p2b_swaprate_emul.csv` | Emulated swap-rate vs decode tok/s. | a6000@4ffd0c11 |
| `p3_integrated.csv` | P3 integrated decode — stock vs overlap vs serial tok/s and exposed ms. | a6000@4bdea753 |
| `p3b_unified.csv` | P3b unified-kernel decode variants — tok/s, VRAM reduction, correctness. | a6000@37a74bf9 |
| `p3c_overlap.csv` | P3c copy-overlap decode variants — tok/s, VRAM, correctness. | a6000@edc6bb7c |
| `p3d_fair.csv` | P3d fair-comparison decode variants — tok/s, VRAM, correctness. | a6000@d6f9b879 |
| `perf_baseline.csv` | Training perf baseline — s/iter, TFLOPs/GPU, peak GB per shape/mode. | a6000@eba7d3f4 |
| `phase0_isoflop_points.csv` | Per-point final CE/BPB for the 1e16/1e17 isoFLOP program (dense/MoE/temporal × coarse/fine × shapes, incl. s=2 and eviction variants); records the end-of-training test-set eval AND the published during-training val value where they differ. | temporal-moe-impl@66f786b7 (log.md), main (plot_g3_curves.py) |
| `phase0_lr_tuning.csv` | LR-selection + cross-size stability runs behind the locked peak-LR 3e-3. | temporal-moe-impl@66f786b7 (log.md) |
| `rolling_coverage_lifetime_vs_K.csv` | Rolling-residency hit-rate & expert lifetime vs resident-cache size K/k. | a6000@cd588fd6, h100@cd588fd6 |
| `rsweep.csv` | Resident-set-size (R) sweep — BPB + A3/coverage/eff-experts/swap-rate per cell. | a6000@b65454d8 |
| `seed_replicates.csv` | Seed-replicate CE/BPB across temporal & full-MoE variants (a6000+h100 combined; box col marks source). | a6000@89028580, h100@0562e4a4 |
| `specialization_m3.csv` | Expert-specialization M3 — centroid distances & pairwise cosine per cell. | a6000@0ec187a2 |
| `specialization_probe.csv` | Per-(layer,expert) specialization probe: residency share, selectivity, gate-when-selected, centroid dist. | a6000@0ec187a2 |
| `specialization_summary.csv` | Per-cell specialization summary (PR percentiles, generalist frac, eff-rank, gate stats). | a6000@0ec187a2 |
| `serving_benchmarks.csv` | llama.cpp real-engine serving results: decode a/b/c/d kernel setups + prefill A/D/C decomposition + matched-ubatch 16k sweep + vanilla-offload floor curve (TEMPORAL_SWAP_N pinned misses/layer on the depth-re-pinned comparable model). Toolchain now fully committed (gen_random_qwen3moe.py, build_models.sh, run_serving_bench.sh, refreshed systems_bench.patch). | fork@6094183 (a6000, local, orig rows), a6000@50af4263 (floor) |
| `stability_weights.csv` | Per-expert weight statics (Frobenius/maxabs/excess-kurtosis per matrix; routed/shared/dense-FFN/router-row), 1e18 38M + 1e19 cells. | h100@9e07cf38 |
| `stability_residency.csv` | Per-expert resident/selected fractions on the fixed eval batch (temporal cells), joins 1:1 to stability_weights router rows. | h100@5a7908d1 |
| `stability_activations.csv` | Per-expert activation stats (output norms, FFN-intermediate max/kurtosis, router logits resident/non-resident, gates), 1e18 + 1e19. | h100@2634e09b (38M), h100@ec9007c8 (1e19) |
| `stability_trunk.csv` | Trunk stats per layer/head (max attention logit, block out/in L2 ratios, residual-stream dims), 1e18 + 1e19. | h100@2634e09b (38M), h100@ec9007c8 (1e19) |
| `stability_gradnorms.csv` | Grad-norm series (every 10 iters) from surviving train.logs, 38M + 1e19 runs, spike census. | h100@efc43768 |
| `stability_fakequant.csv` | Fake-quant (per-group-128 RTN, routed experts only) test CE/BPB at 16/8/4/3 bits, 1e18 + 1e19; divisor 2.9780. | h100@8f5064e6 (38M), h100@ec9007c8 (1e19) |
| `t18_1e18_curves.csv` | 1e18 (38M) isoFLOP training curves — coarse full-MoE + coarse temporal cells. | h100@60b0e351/e53596d6 |
| `t19_1e19_curves.csv` | 1e19 isoFLOP training curves for the 4 t19 cells (dense / moe_coarse / coarse_temporal / fine_temporal). | h100@03a8e488/6a69b7f7/d2b7cd7e/de80543f |
| `t19_lmeval_stderr.csv` | t19 lm-eval rerun with per-task stderr (pinned-venv harness; accs reproduce t19_lmeval within noise). | h100@9bec50ac |
| `t19_lmeval.csv` | t19 downstream lm-eval accuracies (dense / moe_coarse / temporal_coarse). | h100@a42340f1 |
| `t19_lmeval_finegrain.csv` | t19 fine-grained temporal downstream lm-eval accuracies. | h100@3b8d9da8 |
| `t19_moe_coarse_probe.csv` | t19 coarse full-MoE routing probe — load balance, gate PR, self-consistency. | h100@5df8095d |
| `t19_temporal_probe.csv` | t19 temporal residency probe (coarse 6/64 + fine 18/192; test-CE, h100 train.log-verified) — A3 mass/set coverage, union, eff-experts. Combines t19_probe_triple + t19_fine_probe_triple. | h100@6a69b7f7 (coarse), h100@d2b7cd7e (fine) |
| `temporal_router_momentum_sweep.csv` | Temporal-router momentum (γ_m×β_m) sweep — swap rate & set/mass coverage (audit-scored). | a6000@6f9e02bc |
| `temporal_router_momentum_sweep_selfscored.csv` | Same momentum sweep, self-scored coverage. | a6000@6f9e02bc |
| `temporal_router_tau_ema_sweep.csv` | Temporal-router τ (min-logit eviction threshold) × EMA-β sweep — swap rate & coverage (audit-scored). | a6000@6f9e02bc |
| `temporal_router_tau_ema_sweep_selfscored.csv` | Same τ×EMA sweep, self-scored coverage. | a6000@6f9e02bc |
| `unmask_eval.csv` | Unmask / cross-regime eval — native vs cross-regime metric per scale & paradigm. | a6000@d3fbc238 |
| `olmoe_adapt_impose.csv` | OLMoE-1B-7B-0125 Stage-0 zero-shot impose (R=k=8 of 64) + R-sweep coherence + audited-slice restatement (base 0.6727 / impose 2.7507 BPB, D=3.1089 byte-derived). | h100@29556cc6, restated h100@dbc9461a |
| `olmoe_adapt_corpus_audit.md` | Adaptation program provenance: no-unconsumed-tail finding, parent-pool corpus recipe (1B tok, dedup counts), ReMoE/MELINOE budget priors, throughput root cause (HF per-expert python dispatch loop). | h100@92ccb2f2 + updates |
| `olmoe_adapt_sweep.csv` | Stage-2 router-only LR sweep under R=8 (3e-5/1e-4/3e-4 × 0.25B, full 50M eval series; winner 3e-4, 70.7% recovery, late crossover). | h100@358818ab |
| `olmoe_adapt_bakeoff.csv` | Stage-2b escalation bake-off, all 11 arms + curves + eval-noise sigma (A/B/D routing 70%; C norms 91.4% == E LoRA; CE winner 93.2%; F' full-FT 93.4% = constraint price; G distill hurts; Er8/Er64 rank saturation; H zone-anneal null). | h100 per-arm commits 0065–0101 |
| `olmoe_adapt_lmeval_impose.csv` | Stage-0 3-task downstream impose (arc_easy/piqa/obqa collapse under cold R=8 mask). | h100@dbc9461a |
| `olmoe_adapt_downstream.csv` | Stage-3 downstream 10-task lm-eval, 5 cells: base-free / impose-R8 / CE-adapted-R8 / OLMo-1B-0724 / OLMo-7B-0724 (74.4% accuracy recovery; CE-adapt 0.589 vs dense-1B 0.601; base 0.682 vs dense-7B 0.677). | h100@15200735 + dense bracket h100@01170b04 |
| `olmoe_adapt_forensics.csv` | Stage-3 de-lex probe battery, CE-adapt vs base-impose (locus flips token->context: ctx-tok -0.004 -> +0.049; generalists -> 0; demand AUC ~0.96 both = non-discriminator). | h100@0124 result commit |
| `olmoe_adapt_telemetry.csv` | Stage-3 routing telemetry of CE-adapted @R=8 (swap 0.9997 ~= ceiling, dwell 13.4 tok, usage entropy 0.9988). | h100@0125 result commit |
| `olmoe_scratch_ladder.csv` | Baseline-equivalence: c4val harness validation (ours 2.4730 vs OLMoE-0924 wandb 2.4807) + from-scratch 0924 checkpoint ladder on our slice (20B..524B) -> crossings ~28B (C) / ~46B (CE) tokens, agrees with wandb train-curve 25.7/42.8B. | h100@61db892b + ladder commits |
| `olmoe_minflow_calib.csv` | O-series captured-mass ladder per layer (windowed + full-4096: static 0.213 / greedy 0.277 / MinFlow-m1 0.327 / top-8 bound 0.405; flat-router coverage finding top-24=0.684). | h100@2aa82d49 + full h100@0137 |
| `olmoe_minflow_msweep.csv` | O-1 swap-budget m-sweep {1,2,4} (m>1 diagnostic-only, non-deployable: m× fetch bandwidth). | h100@f54f3fb3 |
| `olmoe_minflow_bpb.csv` | O-2/O-close reward->BPB transfer + the three decisive pairs: replay headroom 0.0231±0.0076; free-vs-drift 0.4034±0.0178; CE winner-gate scan-oracle -0.0113±0.0021 (live scan BEATS forced free-logit oracle -> offline scheduling dead). | h100@0110–0138 commits, final 0130a03a |
| `olmoe_minflow_capture_meta.json` | O-0 reward-field capture metadata (rank-mass histogram, coverage). | h100@2aa82d49 |
| `olmoe_cal0.csv` | Cal-0 closed-form moment-matching probe: REJECTED (raw 1.1% / clipped 31.5% recovery; not benign under free routing; cos~0.01 vs learned gains). | h100@e6795ec0 |
| `olmoe_cal2.csv` | Cal-2 calibrated-init training screen: NULL, undone-to-single-basin (final 0.9262 vs C@50M 0.8791; cos-to-init 1.0->0.17, cos-to-C -0.03->0.44; init axis closed). | h100@9f0ca8ae |
| `olmoe_adapt_RESULTS.md` | Full adaptation-study writeup (11-arm table, four-part mechanism story, recommendations). | h100@0101 closeout |
| `adapt_ckpts/*.safetensors` | Router-only checkpoints (~4MB each): LR-sweep arms, bake-off router arms (B/C/D), Cal-2. LoRA-bearing ckpts (474MB) remain pod-local by policy. | h100 arm commits |