weight-space-programming / labs /lab1_diff.html
thegovind's picture
Publish the weight-space programming labs (part 2)
84bcfb1 verified
Raw History Blame Contribute Delete
112 kB
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="only light">
<meta name="theme-color" content="#f5f5f5">
<title>Lab 1: Find the edit by diffing the two weight files · Weight-space programming lab</title>
<meta name="description" content="Big thing: You will compare all 488 weight tensors (873,438,784 numbers) of base Qwen3.5-0.8B and the paper&#x27;s released model bit for bit. From the weights alone, you recover the full edit: layers =…">
<link rel="icon" href="../static/favicon.svg" type="image/svg+xml">
<link rel="preload" href="../static/fonts/geist-latin-wght-normal.woff2" as="font" type="font/woff2" crossorigin>
<link rel="stylesheet" href="../static/site.css">
<link rel="stylesheet" href="../static/katex/katex.min.css">
<script>document.documentElement.classList.add("js")</script>
<script defer src="../static/site.js"></script>
</head>
<body data-copy="Copy" data-copied="Copied">
<a class="skip" href="#main">Skip to content</a>
<header class="top">
<div class="top-in">
<a class="brand" href="../index.html">Weight-space programming lab</a>
<nav class="top-links" aria-label="Site"><a href="../index.html">Home</a><a href="../quickstart.html">Quick start</a><a href="../index.html#the-nine-labs" aria-current="true">Labs</a><a href="../reference/theory-guide.html">Theory</a><a href="../results/summary.html">Results</a><a href="https://huggingface.co/spaces/thegovind/weight-space-programming/tree/main/code/" target="_blank" rel="noopener">Code</a></nav>
</div>
</header>
<div class="page">
<main id="main" class="doc">
<article class="prose">
<h1 id="lab-1-find-the-edit-by-diffing-the-two-weight-files">Lab 1: Find the edit by diffing the two weight files</h1>
<details class="tocm"><summary>On this page</summary><nav aria-label="On this page"><ul><li class="l2"><a href="#before-you-run-predict">Before you run: predict</a></li><li class="l2"><a href="#run-it">Run it</a></li><li class="l2"><a href="#what-you-should-see">What you should see</a></li><li class="l2"><a href="#why-it-works">Why it works</a></li><li class="l3"><a href="#1-the-detective-pipeline-from-two-files-to-four-numbers">1. The detective pipeline: from two files to four numbers</a></li><li class="l3"><a href="#2-why-65260-and-64928-entries-changed-instead-of-65536">2. Why 65,260 and 64,928 entries changed instead of 65,536</a></li><li class="l3"><a href="#3-taking-the-change-apart-with-svd">3. Taking the change apart with SVD</a></li><li class="l3"><a href="#4-recovering-alpha-and-checking-the-bits">4. Recovering <code>alpha</code> and checking the bits</a></li><li class="l2"><a href="#check-yourself">Check yourself</a></li><li class="l2"><a href="#try-this-next">Try this next</a></li></ul></nav></details>
<p><strong>Big thing:</strong> You will compare all 488 weight tensors (873,438,784 numbers) of base Qwen3.5-0.8B and the paper's released model bit for bit. From the weights alone, you recover the full edit: <code>layers = [15, 19]</code>, <code>64</code> columns per layer, rank-one direction <code>u</code>, and strength <code>alpha = 6</code>.</p>
<p><strong>Why it matters:</strong> When you have the weight files, you do not have to take a paper's description on faith. Tensor diffs and an SVD turn a 0.87-billion-parameter checkpoint diff into one short formula.</p>
<p><strong>Time and hardware:</strong> Under a minute on CPU on <code>sampler</code> or <code>trainer</code> (no GPU needed).</p>
<p><strong>Theory guide:</strong> Section 9, step 4.</p>
<h2 id="before-you-run-predict">Before you run: predict<a class="anchor" href="#before-you-run-predict" aria-hidden="true" tabindex="-1">#</a></h2>
<p>Write a short guess for each question before running the script.</p>
<ol>
<li>Out of the 488 weight tensors in Qwen3.5-0.8B, how many differ between the base model and the released model <code>qwen35-p50c</code>?</li>
<li>Each edited <code>down_proj.weight</code> matrix has shape <code>1,024 x 3,584</code>, and the edit touches <code>64</code> columns (<code>64 x 1,024 = 65,536</code> numbers). Will the file have <code>65,536</code> changed entries or slightly fewer, and why?</li>
<li>In Lab 0, the second singular value of <code>W' - W</code> was <code>0.0000</code>. For the real weight change, will the second singular value be exactly zero or slightly above zero?</li>
</ol>
<h2 id="run-it">Run it<a class="anchor" href="#run-it" aria-hidden="true" tabindex="-1">#</a></h2>
<p>Connect to one of the lab VMs (<code>sampler</code> or <code>trainer</code>), activate the lab environment, and run <code>labs/lab1_diff.py</code> on CPU. This script does not touch a GPU.</p>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">ssh sampler
. /nvme/wsp-lab/code/scripts/env.sh &amp;&amp; cd /nvme/wsp-lab/code
python labs/lab1_diff.py
</code></pre></div>
<p>The script compares every tensor in <code>/nvme/wsp-lab/models/Qwen3.5-0.8B</code> against <code>/nvme/wsp-lab/models/qwen35-p50c</code>, writes <code>results/lab1/diff.json</code>, and saves <code>results/figures/lab1_diff.png</code>.</p>
<h2 id="what-you-should-see">What you should see<a class="anchor" href="#what-you-should-see" aria-hidden="true" tabindex="-1">#</a></h2>
<div class="code"><div class="code-head"><span class="code-lang">text</span></div><pre><code class="language-text">2 of 488 tensors differ (873M numbers in all):
model.language_model.layers.15.mlp.down_proj.weight shape [1024, 3584]
entries changed: 65,260 of 3,670,016, all inside 64 columns (neurons)
singular values of the change: [1.7489, 0.001, 0.0009, 0.0009] ... rank-one share 0.999994
fitted alpha = 6.0003; the matrix moved by 10.6%
model.language_model.layers.19.mlp.down_proj.weight shape [1024, 3584]
entries changed: 64,928 of 3,670,016, all inside 64 columns (neurons)
singular values of the change: [1.3988, 0.0009, 0.0008, 0.0007] ... rank-one share 0.999991
fitted alpha = 6.0002; the matrix moved by 8.1%
|cos| between the two directions: 0.849 (1 = same direction, 0 = unrelated; the SVD sign is arbitrary, Lab 3 fixes it)
wrote /nvme/wsp-lab/code/results/lab1
</code></pre></div>
<p><code>results/lab1/diff.json</code> and <code>results/edit_info.json</code> record the exact numbers:</p>
<div class="table-wrap"><table>
<thead>
<tr>
<th>Measurement</th>
<th>Layer 15 (<code class="cell"><span>down_proj.weight</span></code>)</th>
<th>Layer 19 (<code class="cell"><span>down_proj.weight</span></code>)</th>
</tr>
</thead>
<tbody>
<tr>
<td>Matrix shape (rows x columns)</td>
<td><code class="cell"><span>1,024 x 3,584</span></code> (<code class="cell"><span>3,670,016</span></code> entries)</td>
<td><code class="cell"><span>1,024 x 3,584</span></code> (<code class="cell"><span>3,670,016</span></code> entries)</td>
</tr>
<tr>
<td>Entries changed</td>
<td><code class="cell"><span>65,260</span></code></td>
<td><code class="cell"><span>64,928</span></code></td>
</tr>
<tr>
<td>Columns (neurons) containing every change</td>
<td><code class="cell"><span>64</span></code></td>
<td><code class="cell"><span>64</span></code></td>
</tr>
<tr>
<td>First 8 changed columns</td>
<td><code class="cell"><span>0,</span> <span>2,</span> <span>11,</span> <span>25,</span> <span>34,</span> <span>35,</span> <span>38,</span> <span>39</span></code></td>
<td><code class="cell"><span>1,</span> <span>5,</span> <span>8,</span> <span>11,</span> <span>13,</span> <span>15,</span> <span>17,</span> <span>18</span></code></td>
</tr>
<tr>
<td>Top 4 singular values of <code class="cell"><span>W1 - W0</span></code></td>
<td><code class="cell"><span>1.7489,</span> <span>0.0010,</span> <span>0.0009,</span> <span>0.0009</span></code></td>
<td><code class="cell"><span>1.3988,</span> <span>0.0009,</span> <span>0.0008,</span> <span>0.0007</span></code></td>
</tr>
<tr>
<td>Rank-one share (<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msubsup><mi>s</mi><mn>1</mn><mn>2</mn></msubsup><mi mathvariant="normal">/</mi><msub><mo>∑</mo><mi>i</mi></msub><msubsup><mi>s</mi><mi>i</mi><mn>2</mn></msubsup></mrow><annotation encoding="application/x-tex">s_1^2 / \sum_i s_i^2</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.1138em;vertical-align:-0.2997em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-2.4519em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2481em;"><span></span></span></span></span></span></span><span class="mord">/</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop"><span class="mop op-symbol small-op" style="position:relative;top:0em;">∑</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.162em;"><span style="top:-2.4003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2997em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-2.4413em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2587em;"><span></span></span></span></span></span></span></span></span></span>)</td>
<td><code class="cell"><span>0.999994</span></code></td>
<td><code class="cell"><span>0.999991</span></code></td>
</tr>
<tr>
<td>Fitted strength <code class="cell"><span>alpha</span></code></td>
<td><code class="cell"><span>6.0003</span></code></td>
<td><code class="cell"><span>6.0002</span></code></td>
</tr>
<tr>
<td>Relative matrix change (<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="normal">∣</mi><msub><mi>W</mi><mn>1</mn></msub><mo>−</mo><msub><mi>W</mi><mn>0</mn></msub><msub><mi mathvariant="normal">∣</mi><mi>F</mi></msub><mi mathvariant="normal">/</mi><mi mathvariant="normal">∣</mi><msub><mi>W</mi><mn>0</mn></msub><msub><mi mathvariant="normal">∣</mi><mi>F</mi></msub></mrow><annotation encoding="application/x-tex">|W_1 - W_0|_F / |W_0|_F</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.1389em;">W</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.1389em;">W</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord">∣</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">F</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">/∣</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.1389em;">W</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord">∣</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">F</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>)</td>
<td><code class="cell"><span>10.6%</span></code> (<code class="cell"><span>0.10636</span></code>)</td>
<td><code class="cell"><span>8.1%</span></code> (<code class="cell"><span>0.08106</span></code>)</td>
</tr>
<tr>
<td>Absolute cosine between <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>u</mi><mn>15</mn></msub></mrow><annotation encoding="application/x-tex">u_{15}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">15</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> and <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>u</mi><mn>19</mn></msub></mrow><annotation encoding="application/x-tex">u_{19}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">19</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></td>
<td><code class="cell"><span>0.849</span></code></td>
<td><code class="cell"><span>0.849</span></code></td>
</tr>
</tbody>
</table></div>
<p><img src="../code/results/figures/lab1_diff.png" alt="Top row: size of the change in each of the 3,584 columns of layers 15 and 19, with 64 spikes and exact zeros elsewhere. Bottom row: the top 10 singular values on a log scale, with the first far above the rest." loading="lazy" decoding="async" width="1690" height="845" /></p>
<p>In <code>lab1_diff.png</code>, the top row plots the Euclidean length (norm) of the change in each of the 3,584 columns for layers 15 and 19. You see 64 red spikes rising from a floor of exact zeros; in layer 19, the tallest spike near the left edge is neuron 11. The bottom row plots the top 10 singular values on a logarithmic scale. Bar 1 sits above <code>10^0</code> (<code>1.7489</code> and <code>1.3988</code>), while bars 2 through 10 sit more than three orders of magnitude lower around <code>10^-3</code>.</p>
<h2 id="why-it-works">Why it works<a class="anchor" href="#why-it-works" aria-hidden="true" tabindex="-1">#</a></h2>
<h3 id="1-the-detective-pipeline-from-two-files-to-four-numbers">1. The detective pipeline: from two files to four numbers<a class="anchor" href="#1-the-detective-pipeline-from-two-files-to-four-numbers" aria-hidden="true" tabindex="-1">#</a></h3>
<p>The script inspects the two model folders in four steps, using no prior knowledge of how the paper built its edit.</p>
<figure class="mermaid-svg" role="img" aria-label="Diagram: 1. The detective pipeline: from two files to four numbers" style="--mw:207px"><svg id="m-39bd9364e6e1" width="100%" xmlns="http://www.w3.org/2000/svg" class="flowchart" style="max-width: 276px;" viewBox="0 0 276 822" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#m-39bd9364e6e1{font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#m-39bd9364e6e1 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#m-39bd9364e6e1 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#m-39bd9364e6e1 .error-icon{fill:#552222;}#m-39bd9364e6e1 .error-text{fill:#552222;stroke:#552222;}#m-39bd9364e6e1 .edge-thickness-normal{stroke-width:1px;}#m-39bd9364e6e1 .edge-thickness-thick{stroke-width:3.5px;}#m-39bd9364e6e1 .edge-pattern-solid{stroke-dasharray:0;}#m-39bd9364e6e1 .edge-thickness-invisible{stroke-width:0;fill:none;}#m-39bd9364e6e1 .edge-pattern-dashed{stroke-dasharray:3;}#m-39bd9364e6e1 .edge-pattern-dotted{stroke-dasharray:2;}#m-39bd9364e6e1 .marker{fill:#666;stroke:#666;}#m-39bd9364e6e1 .marker.cross{stroke:#666;}#m-39bd9364e6e1 svg{font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;font-size:16px;}#m-39bd9364e6e1 p{margin:0;}#m-39bd9364e6e1 .label{font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;color:#000000;}#m-39bd9364e6e1 .cluster-label text{fill:#333;}#m-39bd9364e6e1 .cluster-label span{color:#333;}#m-39bd9364e6e1 .cluster-label span p{background-color:transparent;}#m-39bd9364e6e1 .label text,#m-39bd9364e6e1 span{fill:#000000;color:#000000;}#m-39bd9364e6e1 .node rect,#m-39bd9364e6e1 .node circle,#m-39bd9364e6e1 .node ellipse,#m-39bd9364e6e1 .node polygon,#m-39bd9364e6e1 .node path{fill:#eee;stroke:#999;stroke-width:1px;}#m-39bd9364e6e1 .rough-node .label text,#m-39bd9364e6e1 .node .label text,#m-39bd9364e6e1 .image-shape .label,#m-39bd9364e6e1 .icon-shape .label{text-anchor:middle;}#m-39bd9364e6e1 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#m-39bd9364e6e1 .rough-node .label,#m-39bd9364e6e1 .node .label,#m-39bd9364e6e1 .image-shape .label,#m-39bd9364e6e1 .icon-shape .label{text-align:center;}#m-39bd9364e6e1 .node.clickable{cursor:pointer;}#m-39bd9364e6e1 .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#m-39bd9364e6e1 .arrowheadPath{fill:#333333;}#m-39bd9364e6e1 .edgePaths .path{stroke:#666;stroke-width:1px;}#m-39bd9364e6e1 .flowchart-link{stroke:#666;fill:none;}#m-39bd9364e6e1 .edgeLabel{background-color:white;text-align:center;}#m-39bd9364e6e1 .edgeLabel p{background-color:white;}#m-39bd9364e6e1 .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#m-39bd9364e6e1 .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#m-39bd9364e6e1 .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#m-39bd9364e6e1 .cluster text{fill:#333;}#m-39bd9364e6e1 .cluster span{color:#333;}#m-39bd9364e6e1 .node .collapsed-indicator{fill:#707070;stroke:none;opacity:0.6;}#m-39bd9364e6e1 .node .collapsed-separator{stroke:#707070;stroke-width:0.75px;}#m-39bd9364e6e1 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#m-39bd9364e6e1 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#m-39bd9364e6e1 rect.text{fill:none;stroke-width:0;}#m-39bd9364e6e1 .icon-shape,#m-39bd9364e6e1 .image-shape{background-color:white;text-align:center;}#m-39bd9364e6e1 .icon-shape p,#m-39bd9364e6e1 .image-shape p{background-color:white;padding:2px;}#m-39bd9364e6e1 .icon-shape .label rect,#m-39bd9364e6e1 .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#m-39bd9364e6e1 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#m-39bd9364e6e1 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#m-39bd9364e6e1 .node .neo-node{stroke:#999;}#m-39bd9364e6e1 [data-look="neo"].node rect,#m-39bd9364e6e1 [data-look="neo"].cluster rect,#m-39bd9364e6e1 [data-look="neo"].node polygon{stroke:url(#m-39bd9364e6e1-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-39bd9364e6e1 [data-look="neo"].swimlane.cluster rect{filter:none;}#m-39bd9364e6e1 [data-look="neo"].node path{stroke:url(#m-39bd9364e6e1-gradient);stroke-width:1px;}#m-39bd9364e6e1 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-39bd9364e6e1 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#m-39bd9364e6e1 [data-look="neo"].node circle{stroke:url(#m-39bd9364e6e1-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-39bd9364e6e1 [data-look="neo"].node circle .state-start{fill:#000000;}#m-39bd9364e6e1 [data-look="neo"].icon-shape .icon{fill:url(#m-39bd9364e6e1-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-39bd9364e6e1 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#m-39bd9364e6e1-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-39bd9364e6e1 :root{--mermaid-font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;}</style><g><marker id="m-39bd9364e6e1_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></path></marker><marker id="m-39bd9364e6e1_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></path></marker><marker id="m-39bd9364e6e1_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></path></marker><marker id="m-39bd9364e6e1_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></polygon></marker><marker id="m-39bd9364e6e1_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-39bd9364e6e1_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-39bd9364e6e1_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-39bd9364e6e1_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-39bd9364e6e1_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width: 2; stroke-dasharray: 1, 0;"></path></marker><marker id="m-39bd9364e6e1_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width: 2; stroke-dasharray: 1, 0;"></path></marker><marker id="m-39bd9364e6e1_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width: 2.5;"></path></marker><marker id="m-39bd9364e6e1_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width: 2.5; stroke-dasharray: 1, 0;"></path></marker><g class="root"><g class="clusters"></g><g class="edgePaths"><path d="M138,110L138,114.167C138,118.333,138,126.667,138,134.333C138,142,138,149,138,152.5L138,156" id="m-39bd9364e6e1-L_A_B_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_A_B_0" data-points="W3sieCI6MTM4LCJ5IjoxMTB9LHsieCI6MTM4LCJ5IjoxMzV9LHsieCI6MTM4LCJ5IjoxNjB9XQ==" data-look="classic" marker-end="url(#m-39bd9364e6e1_flowchart-v2-pointEnd)"></path><path d="M138,286L138,290.167C138,294.333,138,302.667,138,310.333C138,318,138,325,138,328.5L138,332" id="m-39bd9364e6e1-L_B_C_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_B_C_0" data-points="W3sieCI6MTM4LCJ5IjoyODZ9LHsieCI6MTM4LCJ5IjozMTF9LHsieCI6MTM4LCJ5IjozMzZ9XQ==" data-look="classic" marker-end="url(#m-39bd9364e6e1_flowchart-v2-pointEnd)"></path><path d="M138,462L138,466.167C138,470.333,138,478.667,138,486.333C138,494,138,501,138,504.5L138,508" id="m-39bd9364e6e1-L_C_D_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_C_D_0" data-points="W3sieCI6MTM4LCJ5Ijo0NjJ9LHsieCI6MTM4LCJ5Ijo0ODd9LHsieCI6MTM4LCJ5Ijo1MTJ9XQ==" data-look="classic" marker-end="url(#m-39bd9364e6e1_flowchart-v2-pointEnd)"></path><path d="M138,662L138,666.167C138,670.333,138,678.667,138,686.333C138,694,138,701,138,704.5L138,708" id="m-39bd9364e6e1-L_D_E_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_D_E_0" data-points="W3sieCI6MTM4LCJ5Ijo2NjJ9LHsieCI6MTM4LCJ5Ijo2ODd9LHsieCI6MTM4LCJ5Ijo3MTJ9XQ==" data-look="classic" marker-end="url(#m-39bd9364e6e1_flowchart-v2-pointEnd)"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_A_B_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g class="label" data-id="L_B_C_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g class="label" data-id="L_C_D_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g class="label" data-id="L_D_E_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g></g><g class="nodes"><g class="node default" id="m-39bd9364e6e1-flowchart-A-0" data-look="classic" transform="translate(138, 59)"><rect class="basic label-container" style="" x="-130" y="-51" width="260" height="102"></rect><g class="label" style="" transform="translate(-100, -36)"><rect></rect><foreignObject width="200" height="72"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>488 tensors in base and p50c<br>(873,438,784 numbers)</p></span></div></foreignObject></g></g><g class="node default" id="m-39bd9364e6e1-flowchart-B-1" data-look="classic" transform="translate(138, 223)"><rect class="basic label-container" style="" x="-130" y="-63" width="260" height="126"></rect><g class="label" style="" transform="translate(-100, -48)"><rect></rect><foreignObject width="200" height="96"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>Bitwise check:<br>only 2 tensors differ<br>(layers 15 and 19 down_proj)</p></span></div></foreignObject></g></g><g class="node default" id="m-39bd9364e6e1-flowchart-C-3" data-look="classic" transform="translate(138, 399)"><rect class="basic label-container" style="" x="-130" y="-63" width="260" height="126"></rect><g class="label" style="" transform="translate(-100, -48)"><rect></rect><foreignObject width="200" height="96"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>Column norms of D = W1 - W0:<br>nonzero in 64 of 3,584 columns</p></span></div></foreignObject></g></g><g class="node default" id="m-39bd9364e6e1-flowchart-D-5" data-look="classic" transform="translate(138, 587)"><rect class="basic label-container" style="" x="-130" y="-75" width="260" height="150"></rect><g class="label" style="" transform="translate(-100, -60)"><rect></rect><foreignObject width="200" height="120"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>SVD of D[:, cols]:<br>first singular value holds<br>&gt; 99.999% of the squared change<br>left vector = u</p></span></div></foreignObject></g></g><g class="node default" id="m-39bd9364e6e1-flowchart-E-7" data-look="classic" transform="translate(138, 763)"><rect class="basic label-container" style="" x="-130" y="-51" width="260" height="102"></rect><g class="label" style="" transform="translate(-100, -36)"><rect></rect><foreignObject width="200" height="72"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>Least-squares fit on W0:<br>recover alpha = 6.0003 and 6.0002</p></span></div></foreignObject></g></g></g></g></g><defs><filter id="m-39bd9364e6e1-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><defs><filter id="m-39bd9364e6e1-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><linearGradient id="m-39bd9364e6e1-gradient" gradientUnits="objectBoundingBox" x1="0%" y1="0%" x2="100%" y2="0%"><stop offset="0%" stop-color="hsl(0, 0%, 83.3333333333%)" stop-opacity="1"></stop><stop offset="100%" stop-color="hsl(0, 0%, 88.9215686275%)" stop-opacity="1"></stop></linearGradient></svg></figure>
<p>First, <code>safe_open</code> streams all 488 tensors in <code>model.safetensors-00001-of-00001.safetensors</code> and checks <code>torch.equal(tb, te)</code>. Exactly 486 are bit-for-bit identical; only <code>model.language_model.layers.15.mlp.down_proj.weight</code> and <code>model.language_model.layers.19.mlp.down_proj.weight</code> differ.</p>
<p>Second, the script subtracts the base weights from the edited weights in 32-bit floating point (<code>fp32</code>): <code>D = W1.float() - W0.float()</code>. Checking which columns have a non-zero sum of absolute differences (<code>D.abs().sum(0) &gt; 0</code>) shows that every changed entry falls inside 64 columns in layer 15 and 64 columns in layer 19, leaving the other <code>3,520</code> columns untouched. Column <code>11</code> in layer 19's list <code>[1, 5, 8, 11, ...]</code> is the single neuron the agent zeroed out early in its search (+6.25 percentage points on practice problems) before moving to a 64-neuron edit.</p>
<h3 id="2-why-65260-and-64928-entries-changed-instead-of-65536">2. Why 65,260 and 64,928 entries changed instead of 65,536<a class="anchor" href="#2-why-65260-and-64928-entries-changed-instead-of-65536" aria-hidden="true" tabindex="-1">#</a></h3>
<p>64 columns times 1,024 rows is <code>65,536</code> entries per layer, yet only <code>65,260</code> entries change in layer 15 (<code>276</code> unchanged) and <code>64,928</code> in layer 19 (<code>608</code> unchanged).</p>
<p>The reason is number precision. The weights are stored on disk in <code>bf16</code> (bfloat16, a 16-bit floating-point format with 2 to 3 significant decimal digits). Where <code>u</code> or <code>u . W0[:, j]</code> is near zero, the update <code>-alpha * u[i] * (u . W0[:, j])</code> is smaller than half a <code>bf16</code> step around <code>W0[i, j]</code>. Casting the edited tensor from <code>fp32</code> back to <code>bf16</code> rounds those entries back to their original bits.</p>
<h3 id="3-taking-the-change-apart-with-svd">3. Taking the change apart with SVD<a class="anchor" href="#3-taking-the-change-apart-with-svd" aria-hidden="true" tabindex="-1">#</a></h3>
<p>To check whether all 64 columns moved along one direction <code>u</code> or 64 separate directions, <code>recover(W0, W1)</code> in <code>wsp/edits.py</code> takes the <code>1,024 x 64</code> slice <code>D[:, cols]</code> and computes its Singular Value Decomposition (SVD):</p>
<div class="math-display"><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>D</mi><mo stretchy="false">[</mo><mo>:</mo><mo separator="true">,</mo><mtext>cols</mtext><mo stretchy="false">]</mo><mo>=</mo><mi>U</mi><mi mathvariant="normal">Σ</mi><msup><mi>V</mi><mi>T</mi></msup><mo>=</mo><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mn>64</mn></munderover><msub><mi>s</mi><mi>k</mi></msub><msub><mi>u</mi><mi>k</mi></msub><msubsup><mi>v</mi><mi>k</mi><mi>T</mi></msubsup></mrow><annotation encoding="application/x-tex">D[:, \text{cols}] = U \Sigma V^T = \sum_{k=1}^{64} s_k u_k v_k^T</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.0278em;">D</span><span class="mopen">[</span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord text"><span class="mord">cols</span></span><span class="mclose">]</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.8913em;"></span><span class="mord mathnormal" style="margin-right:0.109em;">U</span><span class="mord">Σ</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.2222em;">V</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:3.1032em;vertical-align:-1.3021em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.8011em;"><span style="top:-1.8479em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em;">k</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span style="top:-3.05em;"><span class="pstrut" style="height:3.05em;"></span><span><span class="mop op-symbol large-op">∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span class="pstrut" style="height:3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">64</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:1.3021em;"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3361em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3361em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0359em;">v</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8913em;"><span style="top:-2.453em;margin-left:-0.0359em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em;">k</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.247em;"><span></span></span></span></span></span></span></span></span></span></span></div>
<p>SVD writes a matrix as a sum of rank-one pieces. The singular values <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>1</mn></msub><mo>≥</mo><msub><mi>s</mi><mn>2</mn></msub><mo>≥</mo><mo>⋯</mo><mo>≥</mo><msub><mi>s</mi><mn>64</mn></msub><mo>≥</mo><mn>0</mn></mrow><annotation encoding="application/x-tex">s_1 \ge s_2 \ge \dots \ge s_{64} \ge 0</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.786em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">≥</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.786em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">≥</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.7719em;vertical-align:-0.136em;"></span><span class="minner">⋯</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">≥</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.786em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">64</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">≥</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0</span></span></span></span> give the size of each piece, and the columns <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>u</mi><mi>k</mi></msub></mrow><annotation encoding="application/x-tex">u_k</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3361em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> of <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>U</mi></mrow><annotation encoding="application/x-tex">U</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em;"></span><span class="mord mathnormal" style="margin-right:0.109em;">U</span></span></span></span> are mutually perpendicular unit vectors.</p>
<ul>
<li>In layer 15, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>1</mn></msub><mo>=</mo><mn>1.7489</mn></mrow><annotation encoding="application/x-tex">s_1 = 1.7489</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">1.7489</span></span></span></span>, while <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>2</mn></msub><mo>=</mo><mn>0.0010</mn></mrow><annotation encoding="application/x-tex">s_2 = 0.0010</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0.0010</span></span></span></span>, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>3</mn></msub><mo>=</mo><mn>0.0009</mn></mrow><annotation encoding="application/x-tex">s_3 = 0.0009</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0.0009</span></span></span></span>, and <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>4</mn></msub><mo>=</mo><mn>0.0009</mn></mrow><annotation encoding="application/x-tex">s_4 = 0.0009</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">4</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0.0009</span></span></span></span>. The first piece holds <code>0.999994</code> (99.9994%) of the squared Frobenius norm (<span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msubsup><mi>s</mi><mn>1</mn><mn>2</mn></msubsup><mi mathvariant="normal">/</mi><msub><mo>∑</mo><mi>k</mi></msub><msubsup><mi>s</mi><mi>k</mi><mn>2</mn></msubsup></mrow><annotation encoding="application/x-tex">s_1^2 / \sum_k s_k^2</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.1138em;vertical-align:-0.2997em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-2.4519em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2481em;"><span></span></span></span></span></span></span><span class="mord">/</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mop"><span class="mop op-symbol small-op" style="position:relative;top:0em;">∑</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.1864em;"><span style="top:-2.4003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2997em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8141em;"><span style="top:-2.4169em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em;">k</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2831em;"><span></span></span></span></span></span></span></span></span></span>).</li>
<li>In layer 19, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>1</mn></msub><mo>=</mo><mn>1.3988</mn></mrow><annotation encoding="application/x-tex">s_1 = 1.3988</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">1.3988</span></span></span></span>, while <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>2</mn></msub><mo>=</mo><mn>0.0009</mn></mrow><annotation encoding="application/x-tex">s_2 = 0.0009</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0.0009</span></span></span></span>, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>3</mn></msub><mo>=</mo><mn>0.0008</mn></mrow><annotation encoding="application/x-tex">s_3 = 0.0008</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">3</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0.0008</span></span></span></span>, and <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>s</mi><mn>4</mn></msub><mo>=</mo><mn>0.0007</mn></mrow><annotation encoding="application/x-tex">s_4 = 0.0007</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">4</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0.0007</span></span></span></span>. The rank-one share is <code>0.999991</code> (99.9991%).</li>
</ul>
<p>The change is a rank-one outer product; the tail of singular values near <code>0.001</code> is consistent with <code>bf16</code> rounding noise. The top left singular vector <code>u = U[:, 0]</code> is the edit direction <code>u</code> up to a sign.</p>
<h3 id="4-recovering-alpha-and-checking-the-bits">4. Recovering <code>alpha</code> and checking the bits<a class="anchor" href="#4-recovering-alpha-and-checking-the-bits" aria-hidden="true" tabindex="-1">#</a></h3>
<p>With <code>u</code> and <code>cols</code> known from <code>D</code>, the unrounded change on the chosen columns is <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo>−</mo><mi>α</mi><mi>P</mi></mrow><annotation encoding="application/x-tex">-\alpha P</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.7667em;vertical-align:-0.0833em;"></span><span class="mord">−</span><span class="mord mathnormal" style="margin-right:0.0037em;">α</span><span class="mord mathnormal" style="margin-right:0.1389em;">P</span></span></span></span>, where <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>P</mi><mo>=</mo><mi>u</mi><mo stretchy="false">(</mo><msup><mi>u</mi><mi>T</mi></msup><msub><mi>W</mi><mn>0</mn></msub><mo stretchy="false">[</mo><mo>:</mo><mo separator="true">,</mo><mtext>cols</mtext><mo stretchy="false">]</mo><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">P = u (u^T W_0[:, \text{cols}])</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em;"></span><span class="mord mathnormal" style="margin-right:0.1389em;">P</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.0913em;vertical-align:-0.25em;"></span><span class="mord mathnormal">u</span><span class="mopen">(</span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.1389em;">W</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">0</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mopen">[</span><span class="mrel">:</span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord text"><span class="mord">cols</span></span><span class="mclose">])</span></span></span></span> is the projection of the base columns onto <code>u</code>. Here <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>P</mi></mrow><annotation encoding="application/x-tex">P</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.6833em;"></span><span class="mord mathnormal" style="margin-right:0.1389em;">P</span></span></span></span> depends only on the line through <code>u</code>, since <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><mo>−</mo><mi>u</mi><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mo>−</mo><mi>u</mi><msup><mo stretchy="false">)</mo><mi>T</mi></msup><mo>=</mo><mi>u</mi><msup><mi>u</mi><mi>T</mi></msup></mrow><annotation encoding="application/x-tex">(-u)(-u)^T = u u^T</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.0913em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord">−</span><span class="mord mathnormal">u</span><span class="mclose">)</span><span class="mopen">(</span><span class="mord">−</span><span class="mord mathnormal">u</span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.8413em;"></span><span class="mord mathnormal">u</span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span></span></span></span></span></span></span></span>. A least-squares fit in <code>recover</code> finds the scalar <code>alpha</code> that best matches <code>D[:, cols]</code> to <code>-alpha * P</code>:</p>
<div class="math-display"><span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>α</mi><mo>=</mo><mo>−</mo><mfrac><mrow><munder><mo>∑</mo><mrow><mi>i</mi><mo separator="true">,</mo><mi>j</mi></mrow></munder><msub><mi>D</mi><mrow><mi>i</mi><mo separator="true">,</mo><mi>j</mi></mrow></msub><msub><mi>P</mi><mrow><mi>i</mi><mo separator="true">,</mo><mi>j</mi></mrow></msub></mrow><mrow><munder><mo>∑</mo><mrow><mi>i</mi><mo separator="true">,</mo><mi>j</mi></mrow></munder><msubsup><mi>P</mi><mrow><mi>i</mi><mo separator="true">,</mo><mi>j</mi></mrow><mn>2</mn></msubsup></mrow></mfrac></mrow><annotation encoding="application/x-tex">\alpha = -\frac{\sum_{i,j} D_{i,j} P_{i,j}}{\sum_{i,j} P_{i,j}^2}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal" style="margin-right:0.0037em;">α</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:2.6976em;vertical-align:-1.1218em;"></span><span class="mord">−</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.5758em;"><span style="top:-2.314em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mop"><span class="mop op-symbol small-op" style="position:relative;top:0em;">∑</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.162em;"><span style="top:-2.4003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mpunct mtight">,</span><span class="mord mathnormal mtight" style="margin-right:0.0572em;">j</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.4358em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.1389em;">P</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.7959em;"><span style="top:-2.4231em;margin-left:-0.1389em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mpunct mtight">,</span><span class="mord mathnormal mtight" style="margin-right:0.0572em;">j</span></span></span></span><span style="top:-3.0448em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.413em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span class="pstrut" style="height:3em;"></span><span class="frac-line" style="border-bottom-width:0.04em;"></span></span><span style="top:-3.8258em;"><span class="pstrut" style="height:3em;"></span><span class="mord"><span class="mop"><span class="mop op-symbol small-op" style="position:relative;top:0em;">∑</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.162em;"><span style="top:-2.4003em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mpunct mtight">,</span><span class="mord mathnormal mtight" style="margin-right:0.0572em;">j</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.4358em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.1667em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0278em;">D</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mpunct mtight">,</span><span class="mord mathnormal mtight" style="margin-right:0.0572em;">j</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2861em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.1389em;">P</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mpunct mtight">,</span><span class="mord mathnormal mtight" style="margin-right:0.0572em;">j</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2861em;"><span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:1.1218em;"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></span></div>
<p>This gives <code>6.0003</code> for layer 15 and <code>6.0002</code> for layer 19, matching the paper's <code>alpha = 6</code> to four digits.</p>
<figure class="mermaid-svg" role="img" aria-label="Diagram: 4. Recovering alpha and checking the bits" style="--mw:207px"><svg id="m-0378103ba15d" width="100%" xmlns="http://www.w3.org/2000/svg" class="flowchart" style="max-width: 276px;" viewBox="0 0 276 646" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#m-0378103ba15d{font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#m-0378103ba15d .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#m-0378103ba15d .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#m-0378103ba15d .error-icon{fill:#552222;}#m-0378103ba15d .error-text{fill:#552222;stroke:#552222;}#m-0378103ba15d .edge-thickness-normal{stroke-width:1px;}#m-0378103ba15d .edge-thickness-thick{stroke-width:3.5px;}#m-0378103ba15d .edge-pattern-solid{stroke-dasharray:0;}#m-0378103ba15d .edge-thickness-invisible{stroke-width:0;fill:none;}#m-0378103ba15d .edge-pattern-dashed{stroke-dasharray:3;}#m-0378103ba15d .edge-pattern-dotted{stroke-dasharray:2;}#m-0378103ba15d .marker{fill:#666;stroke:#666;}#m-0378103ba15d .marker.cross{stroke:#666;}#m-0378103ba15d svg{font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;font-size:16px;}#m-0378103ba15d p{margin:0;}#m-0378103ba15d .label{font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;color:#000000;}#m-0378103ba15d .cluster-label text{fill:#333;}#m-0378103ba15d .cluster-label span{color:#333;}#m-0378103ba15d .cluster-label span p{background-color:transparent;}#m-0378103ba15d .label text,#m-0378103ba15d span{fill:#000000;color:#000000;}#m-0378103ba15d .node rect,#m-0378103ba15d .node circle,#m-0378103ba15d .node ellipse,#m-0378103ba15d .node polygon,#m-0378103ba15d .node path{fill:#eee;stroke:#999;stroke-width:1px;}#m-0378103ba15d .rough-node .label text,#m-0378103ba15d .node .label text,#m-0378103ba15d .image-shape .label,#m-0378103ba15d .icon-shape .label{text-anchor:middle;}#m-0378103ba15d .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#m-0378103ba15d .rough-node .label,#m-0378103ba15d .node .label,#m-0378103ba15d .image-shape .label,#m-0378103ba15d .icon-shape .label{text-align:center;}#m-0378103ba15d .node.clickable{cursor:pointer;}#m-0378103ba15d .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#m-0378103ba15d .arrowheadPath{fill:#333333;}#m-0378103ba15d .edgePaths .path{stroke:#666;stroke-width:1px;}#m-0378103ba15d .flowchart-link{stroke:#666;fill:none;}#m-0378103ba15d .edgeLabel{background-color:white;text-align:center;}#m-0378103ba15d .edgeLabel p{background-color:white;}#m-0378103ba15d .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#m-0378103ba15d .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#m-0378103ba15d .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#m-0378103ba15d .cluster text{fill:#333;}#m-0378103ba15d .cluster span{color:#333;}#m-0378103ba15d .node .collapsed-indicator{fill:#707070;stroke:none;opacity:0.6;}#m-0378103ba15d .node .collapsed-separator{stroke:#707070;stroke-width:0.75px;}#m-0378103ba15d div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#m-0378103ba15d .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#m-0378103ba15d rect.text{fill:none;stroke-width:0;}#m-0378103ba15d .icon-shape,#m-0378103ba15d .image-shape{background-color:white;text-align:center;}#m-0378103ba15d .icon-shape p,#m-0378103ba15d .image-shape p{background-color:white;padding:2px;}#m-0378103ba15d .icon-shape .label rect,#m-0378103ba15d .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#m-0378103ba15d .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#m-0378103ba15d .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#m-0378103ba15d .node .neo-node{stroke:#999;}#m-0378103ba15d [data-look="neo"].node rect,#m-0378103ba15d [data-look="neo"].cluster rect,#m-0378103ba15d [data-look="neo"].node polygon{stroke:url(#m-0378103ba15d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-0378103ba15d [data-look="neo"].swimlane.cluster rect{filter:none;}#m-0378103ba15d [data-look="neo"].node path{stroke:url(#m-0378103ba15d-gradient);stroke-width:1px;}#m-0378103ba15d [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-0378103ba15d [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#m-0378103ba15d [data-look="neo"].node circle{stroke:url(#m-0378103ba15d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-0378103ba15d [data-look="neo"].node circle .state-start{fill:#000000;}#m-0378103ba15d [data-look="neo"].icon-shape .icon{fill:url(#m-0378103ba15d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-0378103ba15d [data-look="neo"].icon-shape .icon-neo path{stroke:url(#m-0378103ba15d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#m-0378103ba15d :root{--mermaid-font-family:Geist,Inter,ui-sans-serif,system-ui,sans-serif;}</style><g><marker id="m-0378103ba15d_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></path></marker><marker id="m-0378103ba15d_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></path></marker><marker id="m-0378103ba15d_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></path></marker><marker id="m-0378103ba15d_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></polygon></marker><marker id="m-0378103ba15d_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-0378103ba15d_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-0378103ba15d_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-0378103ba15d_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"></circle></marker><marker id="m-0378103ba15d_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width: 2; stroke-dasharray: 1, 0;"></path></marker><marker id="m-0378103ba15d_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width: 2; stroke-dasharray: 1, 0;"></path></marker><marker id="m-0378103ba15d_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width: 2.5;"></path></marker><marker id="m-0378103ba15d_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width: 2.5; stroke-dasharray: 1, 0;"></path></marker><g class="root"><g class="clusters"></g><g class="edgePaths"><path d="M138,86L138,90.167C138,94.333,138,102.667,138,110.333C138,118,138,125,138,128.5L138,132" id="m-0378103ba15d-L_SVD_P_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_SVD_P_0" data-points="W3sieCI6MTM4LCJ5Ijo4Nn0seyJ4IjoxMzgsInkiOjExMX0seyJ4IjoxMzgsInkiOjEzNn1d" data-look="classic" marker-end="url(#m-0378103ba15d_flowchart-v2-pointEnd)"></path><path d="M138,238L138,242.167C138,246.333,138,254.667,138,262.333C138,270,138,277,138,280.5L138,284" id="m-0378103ba15d-L_P_Fit_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_P_Fit_0" data-points="W3sieCI6MTM4LCJ5IjoyMzh9LHsieCI6MTM4LCJ5IjoyNjN9LHsieCI6MTM4LCJ5IjoyODh9XQ==" data-look="classic" marker-end="url(#m-0378103ba15d_flowchart-v2-pointEnd)"></path><path d="M138,414L138,418.167C138,422.333,138,430.667,138,438.333C138,446,138,453,138,456.5L138,460" id="m-0378103ba15d-L_Fit_Rebuild_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_Fit_Rebuild_0" data-points="W3sieCI6MTM4LCJ5Ijo0MTR9LHsieCI6MTM4LCJ5Ijo0Mzl9LHsieCI6MTM4LCJ5Ijo0NjR9XQ==" data-look="classic" marker-end="url(#m-0378103ba15d_flowchart-v2-pointEnd)"></path></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_SVD_P_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g class="label" data-id="L_P_Fit_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g class="label" data-id="L_Fit_Rebuild_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g></g><g class="nodes"><g class="node default" id="m-0378103ba15d-flowchart-SVD-0" data-look="classic" transform="translate(138, 47)"><rect class="basic label-container" style="" x="-116" y="-39" width="232" height="78"></rect><g class="label" style="" transform="translate(-86, -24)"><rect></rect><foreignObject width="172" height="48"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel"><p>SVD of D[:, cols]<br>gives u (up to sign +/-1)</p></span></div></foreignObject></g></g><g class="node default" id="m-0378103ba15d-flowchart-P-1" data-look="classic" transform="translate(138, 187)"><rect class="basic label-container" style="" x="-123" y="-51" width="246" height="102"></rect><g class="label" style="" transform="translate(-93, -36)"><rect></rect><foreignObject width="186" height="72"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel"><p>Compute rank-1 template<br>P = u (u^T W0[:, cols])<br>(sign of u cancels out)</p></span></div></foreignObject></g></g><g class="node default" id="m-0378103ba15d-flowchart-Fit-3" data-look="classic" transform="translate(138, 351)"><rect class="basic label-container" style="" x="-130" y="-63" width="260" height="126"></rect><g class="label" style="" transform="translate(-100, -48)"><rect></rect><foreignObject width="200" height="96"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>Least-squares ratio<br>alpha = -sum(D * P) / sum(P * P)<br>gives 6.0003 and 6.0002</p></span></div></foreignObject></g></g><g class="node default" id="m-0378103ba15d-flowchart-Rebuild-5" data-look="classic" transform="translate(138, 551)"><rect class="basic label-container" style="" x="-130" y="-87" width="260" height="174"></rect><g class="label" style="" transform="translate(-100, -72)"><rect></rect><foreignObject width="200" height="144"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>Plug u, cols, alpha=6 into formula<br>Compare with released bf16 bits:<br>matches to &lt;= 0.00049 (1 bf16 step)</p></span></div></foreignObject></g></g></g></g></g><defs><filter id="m-0378103ba15d-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><defs><filter id="m-0378103ba15d-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"></feDropShadow></filter></defs><linearGradient id="m-0378103ba15d-gradient" gradientUnits="objectBoundingBox" x1="0%" y1="0%" x2="100%" y2="0%"><stop offset="0%" stop-color="hsl(0, 0%, 83.3333333333%)" stop-opacity="1"></stop><stop offset="100%" stop-color="hsl(0, 0%, 88.9215686275%)" stop-opacity="1"></stop></linearGradient></svg></figure>
<p>In <code>scripts/build_models.py</code> (logged in <code>results/edit_info.json</code>), we plug the recovered <code>u</code>, <code>cols</code>, and <code>alpha</code> into <code>projection_edit</code> on <code>W0</code> and compare the resulting <code>bf16</code> tensor against <code>W1</code>. Out of <code>3,670,016</code> entries per layer, all but <code>5,875</code> in layer 15 (<code>5,924</code> at <code>alpha = 6.0</code>) and <code>6,158</code> in layer 19 (<code>6,166</code> at <code>alpha = 6.0</code>) match bit for bit. The largest gap anywhere is <code>0.00048828125</code> (about <code>0.00049</code>), one <code>bf16</code> rounding step.</p>
<p>Comparing the unit vectors <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>u</mi><mn>15</mn></msub></mrow><annotation encoding="application/x-tex">u_{15}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">15</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> and <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>u</mi><mn>19</mn></msub></mrow><annotation encoding="application/x-tex">u_{19}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">19</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> gives <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="normal">∣</mi><msub><mi>u</mi><mn>15</mn></msub><mo>⋅</mo><msub><mi>u</mi><mn>19</mn></msub><mi mathvariant="normal">∣</mi><mo>=</mo><mn>0.849</mn></mrow><annotation encoding="application/x-tex">|u_{15} \cdot u_{19}| = 0.849</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord">∣</span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">15</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">⋅</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">19</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord">∣</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6444em;"></span><span class="mord">0.849</span></span></span></span>. An absolute cosine of <code>1.0</code> means two vectors lie on the same line, pointing the same way or opposite ways, and <code>0.0</code> means they are orthogonal (unrelated). A value of <code>0.849</code> is an angle of about 32 degrees: the loop directions at blocks 15 and 19 are closely related in the 1,024-dimensional stream, but distinct.</p>
<h2 id="check-yourself">Check yourself<a class="anchor" href="#check-yourself" aria-hidden="true" tabindex="-1">#</a></h2>
<ol>
<li>Why does <code>labs/lab1_diff.py</code> report the absolute value <code>|cos| = 0.849</code> between <code>u1</code> and <code>u2</code> instead of a signed cosine?</li>
</ol>
<details><summary>Answer</summary>
<p>SVD decomposes a matrix into <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>u</mi><mn>1</mn></msub><msub><mi>s</mi><mn>1</mn></msub><msubsup><mi>v</mi><mn>1</mn><mi>T</mi></msubsup></mrow><annotation encoding="application/x-tex">u_1 s_1 v_1^T</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.0894em;vertical-align:-0.2481em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0359em;">v</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8413em;"><span style="top:-2.4519em;margin-left:-0.0359em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2481em;"><span></span></span></span></span></span></span></span></span></span>, where flipping the signs of both <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>u</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">u_1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> and <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mi>v</mi><mn>1</mn></msub></mrow><annotation encoding="application/x-tex">v_1</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.5806em;vertical-align:-0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0359em;">v</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> leaves the product unchanged: <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><mo>−</mo><msub><mi>u</mi><mn>1</mn></msub><mo stretchy="false">)</mo><msub><mi>s</mi><mn>1</mn></msub><mo stretchy="false">(</mo><mo>−</mo><msub><mi>v</mi><mn>1</mn></msub><msup><mo stretchy="false">)</mo><mi>T</mi></msup><mo>=</mo><msub><mi>u</mi><mn>1</mn></msub><msub><mi>s</mi><mn>1</mn></msub><msubsup><mi>v</mi><mn>1</mn><mi>T</mi></msubsup></mrow><annotation encoding="application/x-tex">(-u_1) s_1 (-v_1)^T = u_1 s_1 v_1^T</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1.0913em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord">−</span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord">−</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0359em;">v</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:1.0894em;vertical-align:-0.2481em;"></span><span class="mord"><span class="mord mathnormal">u</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal">s</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em;"><span></span></span></span></span></span></span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0359em;">v</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8413em;"><span style="top:-2.4519em;margin-left:-0.0359em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span class="pstrut" style="height:2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em;">T</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.2481em;"><span></span></span></span></span></span></span></span></span></span>. From the weight diff alone, you know the line of <code>u</code> at each layer, not whether <code>+u</code> or <code>-u</code> points toward looping. Lab 3 fixes the sign using the model's activations on looping and productive text.</p>
</details>
<ol start="2">
<li>What causes the second singular value of <code>W1 - W0</code> to be around <code>0.0010</code> in layer 15 instead of <code>0</code>?</li>
</ol>
<details><summary>Answer</summary>
<p>Saving the edited weights in <code>bf16</code> (16-bit floating point) rounds each of the 65,536 edited entries, adding small rounding errors that appear in the SVD as a singular-value floor near <code>0.001</code>.</p>
</details>
<ol start="3">
<li>The edit flips each edited column's part along <code>u</code> and multiplies it by 5. Yet <code>||W1 - W0|| / ||W0||</code> is only <code>10.6%</code> in layer 15 and <code>8.1%</code> in layer 19. Is the edit really a small change?</li>
</ol>
<details><summary>Answer</summary>
<p>No. That 10.6% averages over the whole matrix, hiding how large the change is on the edited columns.</p>
<p>The 64 edited columns hold 1.6% of the matrix's squared norm in layer 15 (<code>||W0[:, cols]||^2 / ||W0||^2 = 0.0157</code>) and 1.7% in layer 19.</p>
<p>Inside those columns, the change is large. Scaling a column's <code>u</code> component by -5 changes it by 6 times its size, so column <code>j</code> moves by <code>6 * |cos(u, column j)|</code> of its length. Because the edited columns lean toward <code>u</code> much more than the rest (median <code>|cos|</code> is 0.095 vs. 0.018 in layer 15), a typical edited column moves by 57% of its length (43% in layer 19) and the most-changed column moves by 1.7 times its length (2.3 times in layer 19). Together, the 64-column block changes by 85% of its norm in layer 15 and 62% in layer 19.</p>
<p>Multiplying those two ratios gives the matrix-wide figure: <code>sqrt(0.0157) * 0.85 = 0.125 * 0.85 = 0.106</code>. The edit is a large change to a few neurons that looks small only when divided by the full matrix.</p>
</details>
<h2 id="try-this-next">Try this next<a class="anchor" href="#try-this-next" aria-hidden="true" tabindex="-1">#</a></h2>
<ol>
<li>
<p><strong>Inspect how many neurons layers 15 and 19 share.</strong> Run a one-line check on <code>results/lab1/diff.json</code> from the VM shell:</p>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">python -c &#x27;import json; d=json.load(open(&quot;results/lab1/diff.json&quot;))[&quot;changed&quot;]; c15,c19=[set(v[&quot;cols&quot;]) for v in d.values()]; print(&quot;shared column indices:&quot;, sorted(c15 &amp; c19))&#x27;
</code></pre></div>
<p>It prints <code>[11, 34, 63]</code>. Neuron numbers do not link layers. You could renumber one layer's neurons, moving each neuron's row of <code>gate_proj</code> and <code>up_proj</code> and its column of <code>down_proj</code> together, and the model would compute exactly the same thing. So neuron 11 in layer 15 has no built-in tie to neuron 11 in layer 19. Two random 64-neuron subsets of 3,584 neurons would share <code>64 * 64 / 3,584 = 1.1</code> indices on average, and would share 3 or more about 10% of the time.</p>
<p>The picks are not random, though. In <code>sorted(c15)</code> and <code>sorted(c19)</code>, 20 of layer 15's neurons and 22 of layer 19's have indices below 128, where random picks would put about 2; all three shared indices are below 128 too. In this checkpoint, low-indexed neurons align more with <code>u</code> than higher ones: mean <code>|cos(u, column)|</code> for neurons 0 to 127 vs. 2,560 and up is 0.036 vs. 0.021 in layer 15, and 0.051 vs. 0.022 in layer 19. Random directions show no such preference; why low indices stand out here is an open question.</p>
</li>
<li>
<p><strong>Diff another model variant from <code>/nvme/wsp-lab/models</code>.</strong> The <code>--edited</code> flag compares the base model with any other model folder, writing to <code>results/lab1/try-&lt;name&gt;/</code> so your Lab 1 outputs stay intact.</p>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">python labs/lab1_diff.py --edited qwen35-all2 # about a minute and a half
python labs/lab1_diff.py --edited qwen35-a1 # a few seconds
</code></pre></div>
<p><code>qwen35-all2</code> rebuilds an earlier edit from the agent's run (strength 2 on every column along the released <code>u</code>). It changes 3,584 columns in layer 15 and 3,583 in layer 19 (one layer 19 column's update is far smaller than one <code>bf16</code> step and rounds away), with fitted <code>alpha = 2.0000</code> and <code>1.9999</code>. The rank-one share dips to about 0.9995 because 3.2 million changed entries each carry rounding noise. The matrices move by only 7.1% and 6.7%, less than the released edit despite touching 56 times as many columns. <code>qwen35-a1</code> applies the released recipe at strength 1, erasing the 64 columns' component along <code>u</code> instead of flipping it and multiplying it by 5. It touches the same 64 columns with <code>alpha = 1.0002</code> and <code>0.9996</code>, moving the matrices by 1.8% and 1.4% (about one sixth of the released edit's 10.6% and 8.1%). With <code>--edited qwen35-l15only</code>, the script reports <code>1 of 488 tensors differ</code>.</p>
</li>
<li>
<p><strong>Check the round-trip <code>bf16</code> error in <code>results/edit_info.json</code>.</strong> Run:</p>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">python -c &#x27;import json; d=json.load(open(&quot;results/edit_info.json&quot;))[&quot;layers&quot;]; print({k: {m: v[m] for m in v if &quot;rebuild&quot; in m} for k, v in d.items()})&#x27;
</code></pre></div>
<p>Compare <code>rebuild_alpha_6.0003_entries_differing</code> (<code>5,875</code> in layer 15) and <code>rebuild_alpha_6.0000_entries_differing</code> (<code>5,924</code> in layer 15), and verify that <code>max_abs_diff</code> is <code>0.00048828125</code> in every case.</p>
</li>
</ol>
</article>
<nav class="pager" aria-label="Previous and next"><a class="prev" href="lab0_toy.html" rel="prev"><span>Previous</span><b>Lab 0: The toy edit on a 4 x 6 matrix</b></a><a class="next" href="lab2_reproduce.html" rel="next"><span>Next</span><b>Lab 2: reproduce the failure and the fix on our own GPUs</b></a></nav>
</main>
<aside class="toc"><nav aria-label="On this page"><p class="toc-title">On this page</p><ul><li class="l2"><a href="#before-you-run-predict">Before you run: predict</a></li><li class="l2"><a href="#run-it">Run it</a></li><li class="l2"><a href="#what-you-should-see">What you should see</a></li><li class="l2"><a href="#why-it-works">Why it works</a></li><li class="l3"><a href="#1-the-detective-pipeline-from-two-files-to-four-numbers">1. The detective pipeline: from two files to four numbers</a></li><li class="l3"><a href="#2-why-65260-and-64928-entries-changed-instead-of-65536">2. Why 65,260 and 64,928 entries changed instead of 65,536</a></li><li class="l3"><a href="#3-taking-the-change-apart-with-svd">3. Taking the change apart with SVD</a></li><li class="l3"><a href="#4-recovering-alpha-and-checking-the-bits">4. Recovering <code>alpha</code> and checking the bits</a></li><li class="l2"><a href="#check-yourself">Check yourself</a></li><li class="l2"><a href="#try-this-next">Try this next</a></li></ul></nav></aside>
</div>
<footer class="foot">
<div class="foot-in">
<p class="foot-note">Based on Experiment 8 of "<a href="https://rikiyat.github.io/blog/weight-space-programming/" target="_blank" rel="noopener">Sparks of Weight-Space Programming</a>". Logs and the agent's texts come from <a href="https://github.com/RikiyaT/lm_understand-logs" target="_blank" rel="noopener">RikiyaT/lm_understand-logs</a>, <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank" rel="noopener">CC BY 4.0</a>.</p>
<p class="foot-links"><a href="https://huggingface.co/spaces/thegovind/weight-space-programming/blob/main/code/labs/lab1_diff.md" target="_blank" rel="noopener">View source</a></p>
</div>
</footer>
</body>
</html>