---
title: Weight-Space Geometry of Offline Reasoning Training
emoji: 🧭
colorFrom: indigo
colorTo: pink
sdk: static
app_file: index.html
pinned: false
license: mit
short_description: Interactive weight-space geometry of six reasoning losses
---

# Weight-Space Geometry of Offline Reasoning Training

An **interactive blog** companion to the ICML 2026 MechInterp Workshop paper
(*"Same Data, Different Losses, Same Circuits?"*). Six offline reasoning losses (SFT, RFT, DFT, RIFT, Offline GRPO, DPO) are trained on
*identical* math rollouts from one base model (Qwen3-4B, attention-only LoRA). The page lets
you turn the same knobs the authors did — pick method pairs, scrub across all 36 transformer
layers, and watch the geometry respond.

## What's interactive

- **Cosine map** — 6×6 ⇄ 8×8 (toggle the on-policy methods); click a cell to inspect that pair.
- **Layer-by-layer cosine** — pick pairs, drag the layer slider, read off any block.
- **CKA per layer** — see DPO's late-layer representation collapse.
- **Output subspace (top-1 SVD)** — same answer, different basis.
- **Principal angles**, **update geometry** (‖ΔW‖ + effective rank), **mode connectivity**,
  **accuracy** (GSM8K / AIME26), and **seed / learning-rate** robustness.

Everything is rendered client-side with [Plotly](https://plotly.com/javascript/) from the
paper's released metrics in [`data/`](data/) — no backend, no GPU.

## Run locally

```bash
python3 -m http.server 8000   # then open http://localhost:8000
```

## Layout

```
index.html      the article
style.css       distill-style theme (light + dark)
app.js          loads data/*.json, builds every chart
data/*.json     consolidated, viz-ready metrics
_src/           raw repo extracts + build_data.py (not needed at runtime)
```

## Credit

Based on **AlexWortega/capabilityvectors** (ICML 2026 Mechanistic Interpretability Workshop).
All numbers are computed from that work's published analysis JSON.

---

### Deploying to a Hugging Face Space (run these yourself)

```bash
huggingface-cli login
huggingface-cli repo create <space-name> --type space --space_sdk static
git clone https://huggingface.co/spaces/<user>/<space-name> hf_space
cp -r index.html style.css app.js data README.md hf_space/   # _src/ optional
cd hf_space && git add -A && git commit -m "Interactive blog" && git push
```

The Space goes live at `https://huggingface.co/spaces/<user>/<space-name>`.