File size: 23,455 Bytes
84bcfb1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0aac309
 
 
09fbc15
 
84bcfb1
 
0aac309
84bcfb1
 
 
 
 
 
 
0c5c33b
84bcfb1
 
 
 
 
 
 
 
 
 
 
 
 
 
0c5c33b
84bcfb1
 
 
 
 
 
 
 
 
 
 
 
 
0c5c33b
84bcfb1
 
 
 
 
 
 
 
 
 
09fbc15
84bcfb1
 
 
 
 
 
 
 
 
 
 
 
10fcfd8
84bcfb1
 
0c5c33b
84bcfb1
 
 
 
 
 
 
 
 
 
 
 
 
09fbc15
84bcfb1
 
 
 
 
 
 
 
 
09fbc15
 
 
 
 
 
 
 
 
 
84bcfb1
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="only light">
<meta name="theme-color" content="#f5f5f5">
<title>Quick start: a 2-hour path through the labs · Weight-space programming lab</title>
<meta name="description" content="Big thing: Run the core weight-space programming checks in 2 hours (about 10 minutes of GPU compute), from a 5-second toy matrix on this machine to the full benchmark, dose, decoding, and capstone…">
<link rel="icon" href="static/favicon.svg" type="image/svg+xml">
<link rel="preload" href="static/fonts/geist-latin-wght-normal.woff2" as="font" type="font/woff2" crossorigin>
<link rel="stylesheet" href="static/site.css">
<link rel="stylesheet" href="static/katex/katex.min.css">
<script>document.documentElement.classList.add("js")</script>
<script defer src="static/site.js"></script>
</head>
<body data-copy="Copy" data-copied="Copied">
<a class="skip" href="#main">Skip to content</a>
<header class="top">
<div class="top-in">
<a class="brand" href="index.html">Weight-space programming lab</a>
<nav class="top-links" aria-label="Site"><a href="index.html">Home</a><a href="quickstart.html" aria-current="page">Quick start</a><a href="index.html#the-nine-labs">Labs</a><a href="reference/theory-guide.html">Theory</a><a href="results/summary.html">Results</a><a href="https://huggingface.co/spaces/thegovind/weight-space-programming/tree/main/code/" target="_blank" rel="noopener">Code</a></nav>
</div>
</header>
<div class="page">
<main id="main" class="doc">
<article class="prose">
<h1 id="quick-start-a-2-hour-path-through-the-labs">Quick start: a 2-hour path through the labs</h1>
<details class="tocm"><summary>On this page</summary><nav aria-label="On this page"><ul><li class="l3"><a href="#step-1-run-the-toy-edit-on-a-4-x-6-matrix-15-minutes">Step 1: Run the toy edit on a 4 x 6 matrix (15 minutes)</a></li><li class="l3"><a href="#step-2-run-the-5-minute-quick-test-and-inspect-a-looping-answer-15-minutes">Step 2: Run the 5-minute quick test and inspect a looping answer (15 minutes)</a></li><li class="l3"><a href="#step-3-find-the-edit-in-the-weight-files-on-cpu-15-minutes">Step 3: Find the edit in the weight files on CPU (15 minutes)</a></li><li class="l3"><a href="#step-4-rebuild-the-edit-from-the-agents-8-texts-20-minutes">Step 4: Rebuild the edit from the agent's 8 texts (20 minutes)</a></li><li class="l3"><a href="#step-5-verify-that-the-weight-edit-is-activation-steering-15-minutes">Step 5: Verify that the weight edit is activation steering (15 minutes)</a></li><li class="l3"><a href="#step-6-measure-side-effects-on-ordinary-text-and-selective-kl-15-minutes">Step 6: Measure side effects on ordinary text and selective KL (15 minutes)</a></li><li class="l3"><a href="#step-7-read-the-finished-results-of-labs-2-4-5-and-8-30-minutes">Step 7: Read the finished results of Labs 2, 4, 5, and 8 (30 minutes)</a></li></ul></nav></details>
<p><strong>Big thing:</strong> Run the core weight-space programming checks in 2 hours (about 10 minutes of GPU compute), from a 5-second toy matrix on this machine to the full benchmark, dose, decoding, and capstone results.</p>
<p><strong>Get the code first.</strong> This site is a public copy of the project. On your own machine, the first line below installs <code>uv</code>, which runs the lab's Python scripts with the packages they need and can fetch Python itself. The second line puts <code>uv</code> on your PATH (the folders the shell searches for commands) in this terminal. The <code>uvx</code> command then runs <code>hf</code>, the download tool from <code>huggingface_hub</code>, without installing it, and fetches the <code>code/</code> folder:</p>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uvx --from huggingface_hub hf download thegovind/weight-space-programming \
  --repo-type space --include &quot;code/*&quot; --local-dir ~/weight-space-programming
cd ~/weight-space-programming/code
</code></pre></div>
<p>If you already have <code>uv</code>, skip the first two lines. Commands for this machine run from that <code>code</code> folder. Commands on the GPU machines run from <code>/nvme/wsp-lab/code</code>, where the setup copies the code.</p>
<p><strong>Machines.</strong> Step 1 runs on any laptop. The other steps ran on two GPU machines, each with 8 NVIDIA A100 80 GB GPUs, and most steps use only 1 or 2 of them. These pages call the two machines <code>trainer</code> and <code>sampler</code>, so use your own host names wherever you see those. To set up a machine, follow &quot;Cloning the paper's logs&quot; and then &quot;Setting up a new machine&quot;, both under <a href="index.html#running-things">Running things</a>. The setup puts everything under <code>/nvme/wsp-lab</code> (about 15 GB) and needs no root access.</p>
<p><strong>Data.</strong> The problem files in <code>data/</code> are not included. <a href="https://huggingface.co/spaces/thegovind/weight-space-programming/blob/main/code/data/README.md" target="_blank" rel="noopener"><code>data/README.md</code></a> explains why and shows how to rebuild them.</p>
<blockquote>
<p><strong>Before you start (VM access and environment):</strong></p>
<ol>
<li><strong>SSH hosts:</strong> Connect with <code>ssh trainer</code> or <code>ssh sampler</code> (8x A100 80 GB each).</li>
<li><strong>Environment line (run first in every VM shell):</strong> <code>. /nvme/wsp-lab/code/scripts/env.sh &amp;&amp; cd /nvme/wsp-lab/code</code></li>
<li><strong>Which terminal:</strong> Steps 1 and 7 run on this machine. Step 2 runs on <code>trainer</code> and ends with <code>exit</code>. Steps 3 to 6 share one <code>sampler</code> terminal, and Step 6 ends with <code>exit</code>. Always <code>ssh</code> to each VM from this machine.</li>
</ol>
</blockquote>
<h3 id="step-1-run-the-toy-edit-on-a-4-x-6-matrix-15-minutes">Step 1: Run the toy edit on a 4 x 6 matrix (15 minutes)<a class="anchor" href="#step-1-run-the-toy-edit-on-a-4-x-6-matrix-15-minutes" aria-hidden="true" tabindex="-1">#</a></h3>
<ul>
<li><strong>Where:</strong> This machine (CPU, 5 seconds to run, 15 minutes with the sheet).</li>
</ul>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">cd ~/weight-space-programming/code
uv run --with numpy --with matplotlib python labs/lab0_toy.py
</code></pre></div>
<ul>
<li><strong>What to look at:</strong> Check the 4 numbered outputs (<code>singular values of W' - W: [3.84 0. 0. 0.]</code> and <code>|W'a - Wa| = 0.000</code> when the chosen neurons are quiet vs <code>4.261</code> when firing) and open <code>results/figures/lab0_toy.png</code>. Singular values measure the sizes of independent pieces of a matrix change; only one is nonzero here. Read <a href="labs/lab0_toy.html"><code>labs/lab0_toy.md</code></a> for why the edit is rank one (one column vector times one row vector) and acts like steering (adding a vector to the model's internal state).</li>
</ul>
<h3 id="step-2-run-the-5-minute-quick-test-and-inspect-a-looping-answer-15-minutes">Step 2: Run the 5-minute quick test and inspect a looping answer (15 minutes)<a class="anchor" href="#step-2-run-the-5-minute-quick-test-and-inspect-a-looping-answer-15-minutes" aria-hidden="true" tabindex="-1">#</a></h3>
<ul>
<li><strong>Where:</strong> <code>trainer</code> (2 GPUs for 5 minutes, then CPU for <code>peek.py</code>). If finished answers for <code>quick/base</code> (the unedited model) and <code>quick/p50c</code> (the released edit) are already on the machine, the run skips them. To have vLLM (the program that generates the answers) write fresh ones, move <code>/nvme/wsp-lab/runs/quick</code> aside first.</li>
</ul>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">ssh trainer
. /nvme/wsp-lab/code/scripts/env.sh &amp;&amp; cd /nvme/wsp-lab/code
nvidia-smi   # check which GPUs are free
mv /nvme/wsp-lab/runs/quick /nvme/wsp-lab/runs/quick-ours   # optional: only if generating fresh answers
python scripts/run_jobs.py plans/quick.json --gpus 0,1
CUDA_VISIBLE_DEVICES=&quot;&quot; python labs/peek.py quick/base
CUDA_VISIBLE_DEVICES=&quot;&quot; python labs/peek.py quick/p50c
CUDA_VISIBLE_DEVICES=&quot;&quot; python labs/peek.py quick/base --id gsm8k_train_5628
CUDA_VISIBLE_DEVICES=&quot;&quot; python labs/peek.py quick/p50c --id gsm8k_train_5628
exit   # back to this machine
</code></pre></div>
<ul>
<li><strong>What to look at:</strong> On our run, <code>quick/base</code> gets 6 of 32 right (<code>18.8%</code>) with 27 of 32 cut in a verbatim loop, while <code>quick/p50c</code> gets 21 of 32 (<code>65.6%</code>, <code>+46.9</code> points) with 3 cut and 0 loops. On <code>gsm8k_train_5628</code> (truth <code>240</code>), <code>quick/base</code> repeats an 83-token block about <code>292</code> times from token 8,568 to 32,768, while <code>quick/p50c</code> finishes in <code>5,825</code> tokens after closing <code>&lt;/think&gt;</code> and writing <code>\boxed{240}</code>. See <a href="labs/lab2_reproduce.html"><code>labs/lab2_reproduce.md</code></a>.</li>
</ul>
<h3 id="step-3-find-the-edit-in-the-weight-files-on-cpu-15-minutes">Step 3: Find the edit in the weight files on CPU (15 minutes)<a class="anchor" href="#step-3-find-the-edit-in-the-weight-files-on-cpu-15-minutes" aria-hidden="true" tabindex="-1">#</a></h3>
<ul>
<li><strong>Where:</strong> <code>sampler</code>, from this machine (CPU only, under 1 minute). Keep this terminal open for Steps 4 to 6.</li>
</ul>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">ssh sampler
. /nvme/wsp-lab/code/scripts/env.sh &amp;&amp; cd /nvme/wsp-lab/code
python labs/lab1_diff.py
</code></pre></div>
<ul>
<li><strong>What to look at:</strong> Confirm that only 2 of 488 tensors (arrays of weights) differ (<code>layers.15</code> and <code>layers.19</code> <code>mlp.down_proj.weight</code>), and that all changes sit in 64 columns per layer. The first singular value holds <code>0.999994</code> and <code>0.999991</code> of the squared change, and the fitted <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi></mrow><annotation encoding="application/x-tex">\alpha</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal" style="margin-right:0.0037em;">α</span></span></span></span> (edit strength) is <code>6.0003</code> and <code>6.0002</code>. These paired numbers refer to the two layers in the order listed. Read <a href="labs/lab1_diff.html"><code>labs/lab1_diff.md</code></a> and inspect <code>results/figures/lab1_diff.png</code>.</li>
</ul>
<h3 id="step-4-rebuild-the-edit-from-the-agents-8-texts-20-minutes">Step 4: Rebuild the edit from the agent's 8 texts (20 minutes)<a class="anchor" href="#step-4-rebuild-the-edit-from-the-agents-8-texts-20-minutes" aria-hidden="true" tabindex="-1">#</a></h3>
<ul>
<li><strong>Where:</strong> the same <code>sampler</code> terminal (1 GPU, about 1 minute). Run this before Step 5 because Lab 6 reads which way <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>u</mi></mrow><annotation encoding="application/x-tex">u</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal">u</span></span></span></span> (the direction from productive toward looping text in the model's internal state) points from <code>results/lab3/rebuild.json</code>.</li>
</ul>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">CUDA_VISIBLE_DEVICES=7 python labs/lab3_rebuild.py
</code></pre></div>
<ul>
<li><strong>What to look at:</strong> Verify that blocks 15 and 19 have the largest MLP gap growth (<code>0.263</code> and <code>0.249</code>), the rebuilt directions match the released <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>u</mi></mrow><annotation encoding="application/x-tex">u</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal">u</span></span></span></span> to cosine <code>+0.99998</code> and <code>-0.99998</code>, and the top 64 neuron scores match <code>64/64</code> released columns in both layers. Read <a href="labs/lab3_rebuild.html"><code>labs/lab3_rebuild.md</code></a> and check <code>results/figures/lab3_where_the_loop_grows.png</code> and <code>lab3_neuron_scores.png</code>.</li>
</ul>
<h3 id="step-5-verify-that-the-weight-edit-is-activation-steering-15-minutes">Step 5: Verify that the weight edit is activation steering (15 minutes)<a class="anchor" href="#step-5-verify-that-the-weight-edit-is-activation-steering-15-minutes" aria-hidden="true" tabindex="-1">#</a></h3>
<ul>
<li><strong>Where:</strong> the same <code>sampler</code> terminal (1 GPU, about 1 minute for parts 1 and 2). Run <code>--parts 3,4</code> (about 3 minutes) to trace real loops and correct finished answers. A loop is a stretch of an answer where the model repeats itself. Both come from <code>build/base</code>, the unedited model's answers to 128 practice problems, in <code>/nvme/wsp-lab/runs/build/base</code> on <code>sampler</code>. In our run, part 3 traced 48 loops and part 4 traced 26 correct finished answers, and a new run may give different counts. Parts 3 and 4 need that <code>build/base</code> run. The optional <code>run_jobs.py</code> line below makes it if it is missing (about 24 minutes on 1 GPU). If the run is already there, the line prints <code>0 shards to run on GPUs [7]:</code> and then <code>ALL DONE</code>.</li>
</ul>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">CUDA_VISIBLE_DEVICES=7 python labs/lab6_steering.py
python scripts/run_jobs.py plans/sampler.json --only build/base --gpus 7   # optional: makes build/base if missing
CUDA_VISIBLE_DEVICES=7 python labs/lab6_steering.py --parts 3,4   # optional
</code></pre></div>
<ul>
<li><strong>What to look at:</strong> Editing <code>down_proj</code> vs subtracting <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi><mi>s</mi><mo stretchy="false">(</mo><mi>t</mi><mo stretchy="false">)</mo><mi>u</mi></mrow><annotation encoding="application/x-tex">\alpha s(t) u</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal" style="margin-right:0.0037em;">α</span><span class="mord mathnormal">s</span><span class="mopen">(</span><span class="mord mathnormal">t</span><span class="mclose">)</span><span class="mord mathnormal">u</span></span></span></span> in a hook (a function attached to a layer) matches to <code>3.33e-16</code> in float64 (<code>1.4e-8</code> mean KL in float32). Float64 and float32 are number formats with different precision; KL measures the change in next-token probabilities. Over the last 36 tokens of the agent's 8 texts, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>s</mi><mo stretchy="false">(</mo><mi>t</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">s(t)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">s</span><span class="mopen">(</span><span class="mord mathnormal">t</span><span class="mclose">)</span></span></span></span> (how hard the chosen neurons push toward looping on each token) is near zero on productive text (<code>0.00</code> to <code>0.04</code> at layer 15) and positive on looping text (<code>0.19</code> to <code>0.42</code>). Parts 3 and 4 show the same split on real answers: at layer 19, <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>s</mi><mo stretchy="false">(</mo><mi>t</mi><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">s(t)</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mord mathnormal">s</span><span class="mopen">(</span><span class="mord mathnormal">t</span><span class="mclose">)</span></span></span></span> averages <code>0.76</code> inside loops and about <code>0.1</code> on answers that went well (small, not zero). Read <a href="labs/lab6_steering.html"><code>labs/lab6_steering.md</code></a>, which covers <code>results/figures/lab6_real_loop.png</code> and <code>lab6_finished_vs_loops.png</code>.</li>
</ul>
<h3 id="step-6-measure-side-effects-on-ordinary-text-and-selective-kl-15-minutes">Step 6: Measure side effects on ordinary text and selective KL (15 minutes)<a class="anchor" href="#step-6-measure-side-effects-on-ordinary-text-and-selective-kl-15-minutes" aria-hidden="true" tabindex="-1">#</a></h3>
<ul>
<li><strong>Where:</strong> the same <code>sampler</code> terminal (1 GPU, about 1.5 minutes). Then leave the VM.</li>
</ul>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">CUDA_VISIBLE_DEVICES=7 python labs/lab7_side_effects.py
exit   # back to this machine
</code></pre></div>
<ul>
<li><strong>What to look at:</strong> <code>p50c</code> and <code>rebuilt</code> raise WikiText-2 perplexity (prediction uncertainty on ordinary text, where lower is better and the base is <code>20.009</code>) by <code>+0.79%</code>. At the same time, they shift predictions 2.5 times more on looping texts than on productive ones (<code>0.0241</code> vs <code>0.0097</code>, measured by KL divergence, the change in next-token probabilities). By contrast, <code>a12</code> (a stronger edit) costs <code>+14.34%</code>. The random-direction and random-neuron controls <code>randdir</code>/<code>randcols</code> shift looping text no more than productive text (<code>randdir</code>: <code>0.0039</code> vs <code>0.0048</code>). Read <a href="labs/lab7_side_effects.html"><code>labs/lab7_side_effects.md</code></a> and view <code>results/figures/lab7_side_effects.png</code>.</li>
</ul>
<h3 id="step-7-read-the-finished-results-of-labs-2-4-5-and-8-30-minutes">Step 7: Read the finished results of Labs 2, 4, 5, and 8 (30 minutes)<a class="anchor" href="#step-7-read-the-finished-results-of-labs-2-4-5-and-8-30-minutes" aria-hidden="true" tabindex="-1">#</a></h3>
<ul>
<li><strong>Where:</strong> This machine (no GPU; <a href="results/summary.html"><code>results/summary.md</code></a> and <code>results/figures/</code> hold all runs).</li>
</ul>
<div class="code"><div class="code-head"><span class="code-lang">bash</span></div><pre><code class="language-bash">cd ~/weight-space-programming/code
ls results/figures/
</code></pre></div>
<ul>
<li>
<p><strong>What to look at:</strong> Read these four lab pages and their figures:</p>
<ul>
<li><a href="labs/lab2_reproduce.html"><code>labs/lab2_reproduce.md</code></a> (<code>lab2_benchmarks.png</code>, <code>lab2_lengths.png</code>).</li>
<li><a href="labs/lab4_dose.html"><code>labs/lab4_dose.md</code></a> (<code>lab4_dose.png</code>, <code>lab4_controls.png</code>, including why <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>α</mi><mo>=</mo><mo>−</mo><mn>4</mn></mrow><annotation encoding="application/x-tex">\alpha = -4</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.4306em;"></span><span class="mord mathnormal" style="margin-right:0.0037em;">α</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.7278em;vertical-align:-0.0833em;"></span><span class="mord">−</span><span class="mord">4</span></span></span></span> gains <code>+7.6</code> points by breaking loops).</li>
<li><a href="labs/lab5_decoding.html"><code>labs/lab5_decoding.md</code></a> (<code>lab5_decoding.png</code>, where turning thinking off reaches <code>52.1%</code> at 2,270 tokens).</li>
<li><a href="labs/lab8_capstone.html"><code>labs/lab8_capstone.md</code></a> (<code>lab8_loop_onset.png</code>, <code>lab8_data_needed.png</code>, <code>lab8_one_answer.png</code>, where recipe B built from 225 of the model's own loops reaches <code>58.2%</code> on <code>dev256</code>).</li>
</ul>
<p>To re-run any sheet's GPU jobs, use the move-aside rule in <a href="index.html"><code>README.md</code></a>.</p>
</li>
</ul>
</article>

</main>
<aside class="toc"><nav aria-label="On this page"><p class="toc-title">On this page</p><ul><li class="l3"><a href="#step-1-run-the-toy-edit-on-a-4-x-6-matrix-15-minutes">Step 1: Run the toy edit on a 4 x 6 matrix (15 minutes)</a></li><li class="l3"><a href="#step-2-run-the-5-minute-quick-test-and-inspect-a-looping-answer-15-minutes">Step 2: Run the 5-minute quick test and inspect a looping answer (15 minutes)</a></li><li class="l3"><a href="#step-3-find-the-edit-in-the-weight-files-on-cpu-15-minutes">Step 3: Find the edit in the weight files on CPU (15 minutes)</a></li><li class="l3"><a href="#step-4-rebuild-the-edit-from-the-agents-8-texts-20-minutes">Step 4: Rebuild the edit from the agent's 8 texts (20 minutes)</a></li><li class="l3"><a href="#step-5-verify-that-the-weight-edit-is-activation-steering-15-minutes">Step 5: Verify that the weight edit is activation steering (15 minutes)</a></li><li class="l3"><a href="#step-6-measure-side-effects-on-ordinary-text-and-selective-kl-15-minutes">Step 6: Measure side effects on ordinary text and selective KL (15 minutes)</a></li><li class="l3"><a href="#step-7-read-the-finished-results-of-labs-2-4-5-and-8-30-minutes">Step 7: Read the finished results of Labs 2, 4, 5, and 8 (30 minutes)</a></li></ul></nav></aside>
</div>
<footer class="foot">
<div class="foot-in">
<p class="foot-note">Based on Experiment 8 of "<a href="https://rikiyat.github.io/blog/weight-space-programming/" target="_blank" rel="noopener">Sparks of Weight-Space Programming</a>". Logs and the agent's texts come from <a href="https://github.com/RikiyaT/lm_understand-logs" target="_blank" rel="noopener">RikiyaT/lm_understand-logs</a>, <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank" rel="noopener">CC BY 4.0</a>.</p>
<p class="foot-links"><a href="https://huggingface.co/spaces/thegovind/weight-space-programming/blob/main/code/QUICKSTART.md" target="_blank" rel="noopener">View source</a></p>
</div>
</footer>
</body>
</html>