Blame for code/labs/lab1_diff.md

# Lab 1: Find the edit by diffing the two weight files

**Big thing:** You will compare all 488 weight tensors (873,438,784 numbers) of base Qwen3.5-0.8B and the paper's released model bit for bit. From the weights alone, you recover the full edit: `layers = [15, 19]`, `64` columns per layer, rank-one direction `u`, and strength `alpha = 6`.

**Why it matters:** When you have the weight files, you do not have to take a paper's description on faith. Tensor diffs and an SVD turn a 0.87-billion-parameter checkpoint diff into one short formula.

**Time and hardware:** Under a minute on CPU on `sampler` or `trainer` (no GPU needed).

**Theory guide:** Section 9, step 4.

## Before you run: predict

Write a short guess for each question before running the script.

1. Out of the 488 weight tensors in Qwen3.5-0.8B, how many differ between the base model and the released model `qwen35-p50c`?
2. Each edited `down_proj.weight` matrix has shape `1,024 x 3,584`, and the edit touches `64` columns (`64 x 1,024 = 65,536` numbers). Will the file have `65,536` changed entries or slightly fewer, and why?
3. In Lab 0, the second singular value of `W' - W` was `0.0000`. For the real weight change, will the second singular value be exactly zero or slightly above zero?

## Run it

Connect to one of the lab VMs (`sampler` or `trainer`), activate the lab environment, and run `labs/lab1_diff.py` on CPU. This script does not touch a GPU.

```bash
ssh sampler
. /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code
python labs/lab1_diff.py
```

The script compares every tensor in `/nvme/wsp-lab/models/Qwen3.5-0.8B` against `/nvme/wsp-lab/models/qwen35-p50c`, writes `results/lab1/diff.json`, and saves `results/figures/lab1_diff.png`.

## What you should see

```text
2 of 488 tensors differ (873M numbers in all):
   model.language_model.layers.15.mlp.down_proj.weight  shape [1024, 3584]
      entries changed: 65,260 of 3,670,016, all inside 64 columns (neurons)
      singular values of the change: [1.7489, 0.001, 0.0009, 0.0009] ...  rank-one share 0.999994
      fitted alpha = 6.0003; the matrix moved by 10.6%
   model.language_model.layers.19.mlp.down_proj.weight  shape [1024, 3584]
      entries changed: 64,928 of 3,670,016, all inside 64 columns (neurons)
      singular values of the change: [1.3988, 0.0009, 0.0008, 0.0007] ...  rank-one share 0.999991
      fitted alpha = 6.0002; the matrix moved by 8.1%
|cos| between the two directions: 0.849 (1 = same direction, 0 = unrelated; the SVD sign is arbitrary, Lab 3 fixes it)
wrote /nvme/wsp-lab/code/results/lab1
```

`results/lab1/diff.json` and `results/edit_info.json` record the exact numbers:

| Measurement | Layer 15 (`down_proj.weight`) | Layer 19 (`down_proj.weight`) |
|---|---|---|
| Matrix shape (rows x columns) | `1,024 x 3,584` (`3,670,016` entries) | `1,024 x 3,584` (`3,670,016` entries) |
| Entries changed | `65,260` | `64,928` |
| Columns (neurons) containing every change | `64` | `64` |
| First 8 changed columns | `0, 2, 11, 25, 34, 35, 38, 39` | `1, 5, 8, 11, 13, 15, 17, 18` |
| Top 4 singular values of `W1 - W0` | `1.7489, 0.0010, 0.0009, 0.0009` | `1.3988, 0.0009, 0.0008, 0.0007` |
| Rank-one share ($s_1^2 / \sum_i s_i^2$) | `0.999994` | `0.999991` |
| Fitted strength `alpha` | `6.0003` | `6.0002` |
| Relative matrix change ($\|W_1 - W_0\|_F / \|W_0\|_F$) | `10.6%` (`0.10636`) | `8.1%` (`0.08106`) |
| Absolute cosine between $u_{15}$ and $u_{19}$ | `0.849` | `0.849` |

![Top row: size of the change in each of the 3,584 columns of layers 15 and 19, with 64 spikes and exact zeros elsewhere. Bottom row: the top 10 singular values on a log scale, with the first far above the rest.](../results/figures/lab1_diff.png)

In `lab1_diff.png`, the top row plots the Euclidean length (norm) of the change in each of the 3,584 columns for layers 15 and 19. You see 64 red spikes rising from a floor of exact zeros; in layer 19, the tallest spike near the left edge is neuron 11. The bottom row plots the top 10 singular values on a logarithmic scale. Bar 1 sits above `10^0` (`1.7489` and `1.3988`), while bars 2 through 10 sit more than three orders of magnitude lower around `10^-3`.

## Why it works

### 1. The detective pipeline: from two files to four numbers

The script inspects the two model folders in four steps, using no prior knowledge of how the paper built its edit.

```mermaid
flowchart LR
  A["488 tensors in base and p50c<br/>(873,438,784 numbers)"] --> B["Bitwise check:<br/>only 2 tensors differ<br/>(layers 15 and 19 down_proj)"]
  B --> C["Column norms of D = W1 - W0:<br/>nonzero in 64 of 3,584 columns"]
  C --> D["SVD of D[:, cols]:<br/>first singular value holds<br/>> 99.999% of the squared change<br/>left vector = u"]
  D --> E["Least-squares fit on W0:<br/>recover alpha = 6.0003 and 6.0002"]
```

First, `safe_open` streams all 488 tensors in `model.safetensors-00001-of-00001.safetensors` and checks `torch.equal(tb, te)`. Exactly 486 are bit-for-bit identical; only `model.language_model.layers.15.mlp.down_proj.weight` and `model.language_model.layers.19.mlp.down_proj.weight` differ.

Second, the script subtracts the base weights from the edited weights in 32-bit floating point (`fp32`): `D = W1.float() - W0.float()`. Checking which columns have a non-zero sum of absolute differences (`D.abs().sum(0) > 0`) shows that every changed entry falls inside 64 columns in layer 15 and 64 columns in layer 19, leaving the other `3,520` columns untouched. Column `11` in layer 19's list `[1, 5, 8, 11, ...]` is the single neuron the agent zeroed out early in its search (+6.25 percentage points on practice problems) before moving to a 64-neuron edit.

### 2. Why 65,260 and 64,928 entries changed instead of 65,536

64 columns times 1,024 rows is `65,536` entries per layer, yet only `65,260` entries change in layer 15 (`276` unchanged) and `64,928` in layer 19 (`608` unchanged).

The reason is number precision. The weights are stored on disk in `bf16` (bfloat16, a 16-bit floating-point format with 2 to 3 significant decimal digits). Where `u` or `u . W0[:, j]` is near zero, the update `-alpha * u[i] * (u . W0[:, j])` is smaller than half a `bf16` step around `W0[i, j]`. Casting the edited tensor from `fp32` back to `bf16` rounds those entries back to their original bits.

### 3. Taking the change apart with SVD

To check whether all 64 columns moved along one direction `u` or 64 separate directions, `recover(W0, W1)` in `wsp/edits.py` takes the `1,024 x 64` slice `D[:, cols]` and computes its Singular Value Decomposition (SVD):

$$D[:, \text{cols}] = U \Sigma V^T = \sum_{k=1}^{64} s_k u_k v_k^T$$

SVD writes a matrix as a sum of rank-one pieces. The singular values $s_1 \ge s_2 \ge \dots \ge s_{64} \ge 0$ give the size of each piece, and the columns $u_k$ of $U$ are mutually perpendicular unit vectors.

- In layer 15, $s_1 = 1.7489$, while $s_2 = 0.0010$, $s_3 = 0.0009$, and $s_4 = 0.0009$. The first piece holds `0.999994` (99.9994%) of the squared Frobenius norm ($s_1^2 / \sum_k s_k^2$).
- In layer 19, $s_1 = 1.3988$, while $s_2 = 0.0009$, $s_3 = 0.0008$, and $s_4 = 0.0007$. The rank-one share is `0.999991` (99.9991%).

The change is a rank-one outer product; the tail of singular values near `0.001` is consistent with `bf16` rounding noise. The top left singular vector `u = U[:, 0]` is the edit direction `u` up to a sign.

### 4. Recovering `alpha` and checking the bits

With `u` and `cols` known from `D`, the unrounded change on the chosen columns is $-\alpha P$, where $P = u (u^T W_0[:, \text{cols}])$ is the projection of the base columns onto `u`. Here $P$ depends only on the line through `u`, since $(-u)(-u)^T = u u^T$. A least-squares fit in `recover` finds the scalar `alpha` that best matches `D[:, cols]` to `-alpha * P`:

$$\alpha = -\frac{\sum_{i,j} D_{i,j} P_{i,j}}{\sum_{i,j} P_{i,j}^2}$$

This gives `6.0003` for layer 15 and `6.0002` for layer 19, matching the paper's `alpha = 6` to four digits.

```mermaid
flowchart TD
  SVD["SVD of D[:, cols]<br/>gives u (up to sign +/-1)"] --> P["Compute rank-1 template<br/>P = u (u^T W0[:, cols])<br/>(sign of u cancels out)"]
  P --> Fit["Least-squares ratio<br/>alpha = -sum(D * P) / sum(P * P)<br/>gives 6.0003 and 6.0002"]
  Fit --> Rebuild["Plug u, cols, alpha=6 into formula<br/>Compare with released bf16 bits:<br/>matches to <= 0.00049 (1 bf16 step)"]
```

In `scripts/build_models.py` (logged in `results/edit_info.json`), we plug the recovered `u`, `cols`, and `alpha` into `projection_edit` on `W0` and compare the resulting `bf16` tensor against `W1`. Out of `3,670,016` entries per layer, all but `5,875` in layer 15 (`5,924` at `alpha = 6.0`) and `6,158` in layer 19 (`6,166` at `alpha = 6.0`) match bit for bit. The largest gap anywhere is `0.00048828125` (about `0.00049`), one `bf16` rounding step.

Comparing the unit vectors $u_{15}$ and $u_{19}$ gives $|u_{15} \cdot u_{19}| = 0.849$. An absolute cosine of `1.0` means two vectors lie on the same line, pointing the same way or opposite ways, and `0.0` means they are orthogonal (unrelated). A value of `0.849` is an angle of about 32 degrees: the loop directions at blocks 15 and 19 are closely related in the 1,024-dimensional stream, but distinct.

## Check yourself

1. Why does `labs/lab1_diff.py` report the absolute value `|cos| = 0.849` between `u1` and `u2` instead of a signed cosine?
<details><summary>Answer</summary>

SVD decomposes a matrix into $u_1 s_1 v_1^T$, where flipping the signs of both $u_1$ and $v_1$ leaves the product unchanged: $(-u_1) s_1 (-v_1)^T = u_1 s_1 v_1^T$. From the weight diff alone, you know the line of `u` at each layer, not whether `+u` or `-u` points toward looping. Lab 3 fixes the sign using the model's activations on looping and productive text.
</details>

2. What causes the second singular value of `W1 - W0` to be around `0.0010` in layer 15 instead of `0`?
<details><summary>Answer</summary>

Saving the edited weights in `bf16` (16-bit floating point) rounds each of the 65,536 edited entries, adding small rounding errors that appear in the SVD as a singular-value floor near `0.001`.
</details>

3. The edit flips each edited column's part along `u` and multiplies it by 5. Yet `||W1 - W0|| / ||W0||` is only `10.6%` in layer 15 and `8.1%` in layer 19. Is the edit really a small change?
<details><summary>Answer</summary>

No. That 10.6% averages over the whole matrix, hiding how large the change is on the edited columns.

The 64 edited columns hold 1.6% of the matrix's squared norm in layer 15 (`||W0[:, cols]||^2 / ||W0||^2 = 0.0157`) and 1.7% in layer 19.

Inside those columns, the change is large. Scaling a column's `u` component by -5 changes it by 6 times its size, so column `j` moves by `6 * |cos(u, column j)|` of its length. Because the edited columns lean toward `u` much more than the rest (median `|cos|` is 0.095 vs. 0.018 in layer 15), a typical edited column moves by 57% of its length (43% in layer 19) and the most-changed column moves by 1.7 times its length (2.3 times in layer 19). Together, the 64-column block changes by 85% of its norm in layer 15 and 62% in layer 19.

Multiplying those two ratios gives the matrix-wide figure: `sqrt(0.0157) * 0.85 = 0.125 * 0.85 = 0.106`. The edit is a large change to a few neurons that looks small only when divided by the full matrix.
</details>

## Try this next

1. **Inspect how many neurons layers 15 and 19 share.** Run a one-line check on `results/lab1/diff.json` from the VM shell:
   ```bash
   python -c 'import json; d=json.load(open("results/lab1/diff.json"))["changed"]; c15,c19=[set(v["cols"]) for v in d.values()]; print("shared column indices:", sorted(c15 & c19))'
   ```
   It prints `[11, 34, 63]`. Neuron numbers do not link layers. You could renumber one layer's neurons, moving each neuron's row of `gate_proj` and `up_proj` and its column of `down_proj` together, and the model would compute exactly the same thing. So neuron 11 in layer 15 has no built-in tie to neuron 11 in layer 19. Two random 64-neuron subsets of 3,584 neurons would share `64 * 64 / 3,584 = 1.1` indices on average, and would share 3 or more about 10% of the time.

   The picks are not random, though. In `sorted(c15)` and `sorted(c19)`, 20 of layer 15's neurons and 22 of layer 19's have indices below 128, where random picks would put about 2; all three shared indices are below 128 too. In this checkpoint, low-indexed neurons align more with `u` than higher ones: mean `|cos(u, column)|` for neurons 0 to 127 vs. 2,560 and up is 0.036 vs. 0.021 in layer 15, and 0.051 vs. 0.022 in layer 19. Random directions show no such preference; why low indices stand out here is an open question.
2. **Diff another model variant from `/nvme/wsp-lab/models`.** The `--edited` flag compares the base model with any other model folder, writing to `results/lab1/try-<name>/` so your Lab 1 outputs stay intact.
   ```bash
   python labs/lab1_diff.py --edited qwen35-all2   # about a minute and a half
   python labs/lab1_diff.py --edited qwen35-a1     # a few seconds
   ```
   `qwen35-all2` rebuilds an earlier edit from the agent's run (strength 2 on every column along the released `u`). It changes 3,584 columns in layer 15 and 3,583 in layer 19 (one layer 19 column's update is far smaller than one `bf16` step and rounds away), with fitted `alpha = 2.0000` and `1.9999`. The rank-one share dips to about 0.9995 because 3.2 million changed entries each carry rounding noise. The matrices move by only 7.1% and 6.7%, less than the released edit despite touching 56 times as many columns. `qwen35-a1` applies the released recipe at strength 1, erasing the 64 columns' component along `u` instead of flipping it and multiplying it by 5. It touches the same 64 columns with `alpha = 1.0002` and `0.9996`, moving the matrices by 1.8% and 1.4% (about one sixth of the released edit's 10.6% and 8.1%). With `--edited qwen35-l15only`, the script reports `1 of 488 tensors differ`.
3. **Check the round-trip `bf16` error in `results/edit_info.json`.** Run:
   ```bash
   python -c 'import json; d=json.load(open("results/edit_info.json"))["layers"]; print({k: {m: v[m] for m in v if "rebuild" in m} for k, v in d.items()})'
   ```
   Compare `rebuild_alpha_6.0003_entries_differing` (`5,875` in layer 15) and `rebuild_alpha_6.0000_entries_differing` (`5,924` in layer 15), and verify that `max_abs_diff` is `0.00048828125` in every case.