AlexWortega's picture
Retitle: Weight-Space Geometry of Offline Reasoning Training
f1a5602 verified
|
Raw
History Blame Contribute Delete
2.44 kB
metadata
title: Weight-Space Geometry of Offline Reasoning Training
emoji: 🧭
colorFrom: indigo
colorTo: pink
sdk: static
app_file: index.html
pinned: false
license: mit
short_description: Interactive weight-space geometry of six reasoning losses

Weight-Space Geometry of Offline Reasoning Training

An interactive blog companion to the ICML 2026 MechInterp Workshop paper ("Same Data, Different Losses, Same Circuits?"). Six offline reasoning losses (SFT, RFT, DFT, RIFT, Offline GRPO, DPO) are trained on identical math rollouts from one base model (Qwen3-4B, attention-only LoRA). The page lets you turn the same knobs the authors did — pick method pairs, scrub across all 36 transformer layers, and watch the geometry respond.

What's interactive

  • Cosine map — 6×6 ⇄ 8×8 (toggle the on-policy methods); click a cell to inspect that pair.
  • Layer-by-layer cosine — pick pairs, drag the layer slider, read off any block.
  • CKA per layer — see DPO's late-layer representation collapse.
  • Output subspace (top-1 SVD) — same answer, different basis.
  • Principal angles, update geometry (‖ΔW‖ + effective rank), mode connectivity, accuracy (GSM8K / AIME26), and seed / learning-rate robustness.

Everything is rendered client-side with Plotly from the paper's released metrics in data/ — no backend, no GPU.

Run locally

python3 -m http.server 8000   # then open http://localhost:8000

Layout

index.html      the article
style.css       distill-style theme (light + dark)
app.js          loads data/*.json, builds every chart
data/*.json     consolidated, viz-ready metrics
_src/           raw repo extracts + build_data.py (not needed at runtime)

Credit

Based on AlexWortega/capabilityvectors (ICML 2026 Mechanistic Interpretability Workshop). All numbers are computed from that work's published analysis JSON.


Deploying to a Hugging Face Space (run these yourself)

huggingface-cli login
huggingface-cli repo create <space-name> --type space --space_sdk static
git clone https://huggingface.co/spaces/<user>/<space-name> hf_space
cp -r index.html style.css app.js data README.md hf_space/   # _src/ optional
cd hf_space && git add -A && git commit -m "Interactive blog" && git push

The Space goes live at https://huggingface.co/spaces/<user>/<space-name>.