bundle/poster_embed.html

<div style="width:1280px;margin:0 auto;font-family:'Georgia','Times New Roman',serif;color:#1a1a2e;background:#ffffff;box-sizing:border-box;padding:0;">
  <!-- masthead -->
  <div style="background:linear-gradient(110deg,#2d2a6e 0%,#5B4FCF 60%,#7a6ff0 100%);color:#fff;padding:26px 34px;">
    <div style="font-size:13px;letter-spacing:3px;text-transform:uppercase;opacity:.85;font-family:Arial,sans-serif;">ICML 2026 · Independent Reproduction · OpenReview Tz7n7pX9cO</div>
    <div style="font-size:34px;font-weight:700;line-height:1.15;margin-top:8px;">From Directions to Regions: Decomposing Activations<br>in Language Models via Local Geometry</div>
    <div style="font-size:15px;opacity:.9;margin-top:8px;font-family:Arial,sans-serif;">Mixture of Factor Analyzers (MFA) = discrete region assignment + low-rank within-region offset · verified on released 8k MFAs for Gemma-2-2B</div>
  </div>

  <!-- verdict strip -->
  <div style="display:flex;gap:0;background:#f4f3fb;border-bottom:2px solid #5B4FCF;font-family:Arial,sans-serif;">
    <div style="flex:1;padding:14px 18px;text-align:center;border-right:1px solid #ddd;"><div style="font-size:26px;font-weight:800;color:#2e9e5b;">3 / 5</div><div style="font-size:12px;color:#555;">claims reproduced or supported</div></div>
    <div style="flex:1;padding:14px 18px;text-align:center;border-right:1px solid #ddd;"><div style="font-size:26px;font-weight:800;color:#5B4FCF;">1.2e-6</div><div style="font-size:12px;color:#555;">rel err vs Eq 9–10 closed form</div></div>
    <div style="flex:1;padding:14px 18px;text-align:center;border-right:1px solid #ddd;"><div style="font-size:26px;font-weight:800;color:#5B4FCF;">8.5×</div><div style="font-size:12px;color:#555;">kNN neighbor vs random overlap</div></div>
    <div style="flex:1;padding:14px 18px;text-align:center;"><div style="font-size:26px;font-weight:800;color:#c8860a;">0.8×</div><div style="font-size:12px;color:#555;">MFA vs DiffMeans steering (paper: ~2×)</div></div>
  </div>

  <!-- body: 4 claim cards -->
  <div style="display:grid;grid-template-columns:1fr 1fr;gap:0;">

    <div data-logbook-target="claim-1-mixture-of-factor-analyzers-mfa-decomposes-activations-into-a-discrete-region-assignment-plus-a-continuous-within-region-offset-formalized-in-equations-5-13-section-3" style="padding:20px 26px;border-right:1px solid #e5e5ef;border-bottom:1px solid #e5e5ef;">
      <div style="font-size:12px;font-family:Arial,sans-serif;color:#2e9e5b;font-weight:700;">CLAIM 1 · REPRODUCED</div>
      <div style="font-size:19px;font-weight:700;margin:4px 0 8px;">Decomposition formalism (Eqs 5–13)</div>
      <div style="font-size:14.5px;line-height:1.5;">Ran the released MFA code on 4096 real Gemma-2-2B L18 activations. Responsibilities form a valid posterior (Σ=1.000, dev 1e-7) and are <b>highly peaked</b> (top-1 mass <b>0.999</b> → genuinely discrete region pick). Posterior-mean latent matches the closed form (Eqs 9–10) to <b>1.2e-6</b>. The rank-10 local offset explains <b>~30×</b> more centered variance than a random 10-dim subspace — local low-dimensional structure.</div>
    </div>

    <div data-logbook-target="claim-2-across-12-trained-mfas-on-gemma-2-2b-and-llama-3-1-8b-mfa-reveals-both-broad-semantic-regions-e-g-movie-genres-emotions-and-narrow-token-specific-regions-in-activation-space-section-4-figures-1-3" style="padding:20px 26px;border-bottom:1px solid #e5e5ef;">
      <div style="font-size:12px;font-family:Arial,sans-serif;color:#2e9e5b;font-weight:700;">CLAIM 2 · REPRODUCED</div>
      <div style="font-size:19px;font-weight:700;margin:4px 0 8px;">Broad &amp; narrow regions + neighborhoods</div>
      <div style="font-size:14.5px;line-height:1.5;">Token-type entropy over 200k assigned tokens spans <b>0 → 6.3 nats</b>. Crisp <b>narrow</b> token-specific regions emerge (indefinite article "a"; sentence-final punctuation; single-word <i>football</i>, <i>quantum</i>). Nearest centroids in the kNN graph share <b>8.5×</b> more top-tokens than random pairs (0.597 vs 0.070) — Gaussians tile semantic neighborhoods (Fig 1).</div>
    </div>

    <div data-logbook-target="claim-3-mfa-achieves-an-interpretability-fraction-of-0-96-0-2-for-its-decomposition-components-compared-to-0-29-0-2-for-sparse-autoencoders-section-5-table-1-figure-4" style="padding:20px 26px;border-right:1px solid #e5e5ef;border-bottom:1px solid #e5e5ef;">
      <div style="font-size:12px;font-family:Arial,sans-serif;color:#c8860a;font-weight:700;">CLAIM 3 · MECHANISM SUPPORTED</div>
      <div style="font-size:19px;font-weight:700;margin:4px 0 8px;">Interpretability-fraction gap</div>
      <div style="font-size:14.5px;line-height:1.5;">Exact IF (0.96 vs 0.29) needs human/LLM annotation — not run. But its cause reproduces: on identical activations MFA uses <b>2 features</b> (100% of mass), while the Gemmascope SAE fires <b>288</b> features with only <b>6%</b> of mass in its top-2 and 218 needed for 90%. Mass concentration → high IF.</div>
    </div>

    <div data-logbook-target="claim-5-for-causal-steering-interventions-mfa-achieves-roughly-2x-the-median-steering-score-of-sae-and-difference-in-means-baselines-on-gemma-and-about-33-improvement-on-llama-section-6-2-figure-5" style="padding:20px 26px;border-bottom:1px solid #e5e5ef;">
      <div style="font-size:12px;font-family:Arial,sans-serif;color:#c0392b;font-weight:700;">CLAIM 5 · NOT REPRODUCED (scaled)</div>
      <div style="font-size:19px;font-weight:700;margin:4px 0 8px;">Causal steering by centroids (Eq 14)</div>
      <div style="font-size:14.5px;line-height:1.5;">MFA centroid interventions vs supervised difference-in-means on Gemma-2-2B L18, LLM-judged (Llama-3.2-3B, open sub for GPT-4o-mini). MFA median <b>0.39</b> vs DiffMeans <b>0.49</b> = <b>0.8×</b> — <b>on par, not the paper's ~2×</b>. MFA wins on some concepts (freedom 0.86 vs 0.58). Gap likely from coarse α sweep (4 vs 15), few concepts (vs 250), 3B judge, no SAE arm.</div>
    </div>
  </div>

  <!-- footer -->
  <div style="display:flex;justify-content:space-between;align-items:center;background:#1a1a2e;color:#e8e8f5;padding:14px 30px;font-family:Arial,sans-serif;font-size:12.5px;">
    <div>Claim 4 (RAVEL/MCQA localization): documented — requires MIB benchmark harness, not run.</div>
    <div style="opacity:.85;">1× RTX PRO 6000 Blackwell (+ HF Jobs L4) · released checkpoints · ~$0.3 GPU · repro3 / torch 2.11</div>
  </div>
</div>