title: Weight-Space Geometry of Offline Reasoning Training
emoji: 🧭
colorFrom: indigo
colorTo: pink
sdk: static
app_file: index.html
pinned: false
license: mit
short_description: Interactive weight-space geometry of six reasoning losses
Weight-Space Geometry of Offline Reasoning Training
An interactive blog companion to the ICML 2026 MechInterp Workshop paper ("Same Data, Different Losses, Same Circuits?"). Six offline reasoning losses (SFT, RFT, DFT, RIFT, Offline GRPO, DPO) are trained on identical math rollouts from one base model (Qwen3-4B, attention-only LoRA). The page lets you turn the same knobs the authors did — pick method pairs, scrub across all 36 transformer layers, and watch the geometry respond.
What's interactive
- Cosine map — 6×6 ⇄ 8×8 (toggle the on-policy methods); click a cell to inspect that pair.
- Layer-by-layer cosine — pick pairs, drag the layer slider, read off any block.
- CKA per layer — see DPO's late-layer representation collapse.
- Output subspace (top-1 SVD) — same answer, different basis.
- Principal angles, update geometry (‖ΔW‖ + effective rank), mode connectivity, accuracy (GSM8K / AIME26), and seed / learning-rate robustness.
Everything is rendered client-side with Plotly from the
paper's released metrics in data/ — no backend, no GPU.
Run locally
python3 -m http.server 8000 # then open http://localhost:8000
Layout
index.html the article
style.css distill-style theme (light + dark)
app.js loads data/*.json, builds every chart
data/*.json consolidated, viz-ready metrics
_src/ raw repo extracts + build_data.py (not needed at runtime)
Credit
Based on AlexWortega/capabilityvectors (ICML 2026 Mechanistic Interpretability Workshop). All numbers are computed from that work's published analysis JSON.
Deploying to a Hugging Face Space (run these yourself)
huggingface-cli login
huggingface-cli repo create <space-name> --type space --space_sdk static
git clone https://huggingface.co/spaces/<user>/<space-name> hf_space
cp -r index.html style.css app.js data README.md hf_space/ # _src/ optional
cd hf_space && git add -A && git commit -m "Interactive blog" && git push
The Space goes live at https://huggingface.co/spaces/<user>/<space-name>.