---
tags:
- interpretability
- jacobian-lens
- nvfp4
license: apache-2.0
---

# Laguna-XS-2.1 J-lens: bf16 vs NVFP4 A/B

Two Jacobian lenses of [poolside/Laguna-XS-2.1](https://huggingface.co/poolside/Laguna-XS-2.1) (~33B fine-grained MoE, 40 layers, d=2048,
256 experts/8 active) fitted on **byte-identical prompts** (200 × wikitext-103, seq 128,
all 39 source layers, target 39) — one from the bf16 checkpoint, one from the official
NVFP4 twin. The twins' non-expert tensors are bit-identical (verified: lm_head, final
norm, embeddings, attention, router), so every lens difference is attributable to
**NVFP4 quantization of expert weights** (plus one ~0.6% export-time perturbation of the
3 layer-0 dense-MLP tensors, documented in the report).


![Capture d’écran 2026-07-17 à 19.20.19](https://cdn-uploads.huggingface.co/production/uploads/651e96991b97c9f33d26bde6/FpF_njYXndvS_BSVShawM.png)

## Files

- `laguna_bf16_jacobian_lens.pt`, `laguna_nvfp4_jacobian_lens.pt` — merged n=200 lenses (fp16 payload; fp16 rounding is irrelevant at 1−CKA ≈ 2e-9)
- `shards_{bf16,nvfp4}/` — 8 per-GPU shard lenses per arm (fp32, 25 prompts each; shard i = same prompts in both arms)
- `convergence_{bf16,nvfp4}/` — per-shard convergence CSVs
- `results/` — `ab_summary.json`, per-analysis npz, figures, `REPORT.md`

Probe: P = W_U[ids]·γ (plain-γ RMSNorm, no softcap), ids = 4096 frozen seed-0 over
vocab 100352; identical across arms. Fit code: `open-jlens-data` repo
(`code/moe/fit_moe.py` laguna dispatch, `nvfp4_experts.py`, `analyze_laguna_ab.py`).
Fitted + analyzed 2026-07-17. Context: this A/B calibrates the NVFP4 caveat of the
Inkling ~950B lens ([PrimeIntellect/inkling-jlens](https://huggingface.co/PrimeIntellect/inkling-jlens)) — same modelopt recipe family;
transfer caveat: Laguna experts are 512-dim vs Inkling 3072-dim.

## xs2/ — Laguna-XS.2 version-drift extension (E9)

`xs2/` holds the same fit for **poolside/Laguna-XS.2** (bf16, identical 200 prompts/probe/tokenizer)
plus the version comparison vs XS-2.1. Verdict: **version drift is a regime change** — same-layer
CKA median 0.638 = 43× the NVFP4 effect and 54× prompt noise — while the depth architecture
survives (both versions k=2 at layer 27/28, cross-matrix depth correlation 0.990). The versions
share no token basis (same-token unembedding rows near-orthogonal), so each is read through its
own probe. Perturbation ladder on identical prompts (1−CKA): noise 0.007 < NVFP4 0.008 ≪ version 0.362.