# Lab 5: can decoding settings do the same job?

**Big thing:** We compare the 131,072-weight edit against six decoding changes on unedited Qwen3.5-0.8B (temperature `1.0`, presence penalty `0` and `2.0`, repetition penalty `1.1`, a shorter `4,096`-token cap, and `<think>` mode off) across all 512 `dev256` answers. Penalties and temperature raise accuracy at most from `12.1%` to `34.4%`, turning thinking off reaches `52.1%` in 13 times fewer tokens than the unedited thinking baseline, and the weight edit with thinking on reaches `61.5%` (`74.6%` on the GSM8K half).

**Why it matters:** Always test whether a weight edit beats simpler decoding changes that leave the weights alone. Ending counts suggest penalties mostly add normal closes, while the weight edit adds both normal closes and loop escapes. These counts do not prove which loops were prevented, because the check reads only the text right before `</think>`.

**Time and hardware:** We already ran it (`results/summary.md`, `results/lab5/endings.json`, and `results/figures/lab5_decoding.png`). Re-running one unedited `dev256` job on 1 NVIDIA A100 GPU takes `34` to `49` minutes with thinking on at the `32,768`-token cap, or about `3` minutes with thinking off or a `4,096`-token cap.

**Theory guide:** Section 13.2.

## Before you run: predict

Write down a guess for each question before looking at the table.

1. The paper's baseline (`"think35"`) uses `presence_penalty = 1.5`, which subtracts `1.5` from the score of any token that has already appeared in the answer. If you turn that penalty off (`presence_penalty = 0`), what happens to the share of answers cut at the 32,768-token cap (now `88.7%`)?
2. The unedited model's correct `dev256` answers have a median length of `5,600` tokens, while looping answers run to `32,768` tokens. What happens to accuracy if you lower the token cap to `4,096` tokens?
3. What if you turn `<think>` mode off (`enable_thinking=False`), so the model writes a direct step-by-step solution instead of an open-ended internal scratchpad? Will it beat the unedited thinking model (`12.1%`), and will the weight edit (`p50c`) still help when thinking is off?

## Run it

All seven decoding runs are at the bottom of `plans/sampler.json` and ran on `sampler` alongside the Lab 4 variants (`dev/base` and `dev/p50c` at the top of `plans/sampler.json` are the baselines).

Open `plans/sampler.json` and check the last seven entries:

```json
  {"name": "dev/base+temp1.0", "model": "base", "data": "dev256", "n": 2, "shards": 1, "est": 15, "set": ["temperature=1.0"]},
  {"name": "dev/base+pp0", "model": "base", "data": "dev256", "n": 2, "shards": 1, "est": 15, "set": ["presence_penalty=0"]},
  {"name": "dev/base+pp2", "model": "base", "data": "dev256", "n": 2, "shards": 1, "est": 15, "set": ["presence_penalty=2.0"]},
  {"name": "dev/base+rep1.1", "model": "base", "data": "dev256", "n": 2, "shards": 1, "est": 14, "set": ["repetition_penalty=1.1"]},
  {"name": "dev/base+max4k", "model": "base", "data": "dev256", "n": 2, "shards": 1, "est": 2, "set": ["max_tokens=4096"]},
  {"name": "dev/base+nothink", "model": "base", "data": "dev256", "n": 2, "shards": 1, "est": 2, "preset": "nothink35"},
  {"name": "dev/p50c+nothink", "model": "qwen35-p50c", "data": "dev256", "n": 2, "shards": 1, "est": 1, "preset": "nothink35"}
```

Each entry starts from a preset in `wsp/protocol.py`:

- `"think35"` (the default when `"preset"` is omitted): `temperature = 0.6`, `top_p = 0.95`, `top_k = 20`, `presence_penalty = 1.5`, `repetition_penalty = 1.0`, `max_tokens = 32768`, `thinking = True`.
- `"nothink35"` (Qwen's model-card settings for non-thinking mode): `temperature = 0.7`, `top_p = 0.8`, `top_k = 20`, `presence_penalty = 1.5`, `repetition_penalty = 1.0`, `max_tokens = 8192`, `thinking = False`.
- A `"set"` list overrides fields in the preset (`scripts/run_jobs.py` passes each `"key=value"` string to `scripts/gen_worker.py` as `--set`).

These runs finished on `sampler` and are in `results/runs/` on this laptop. On 1 A100 GPU, a full `dev256` run takes `17` to `49` minutes (`33.8` to `48.9` minutes for the long unedited runs; `dev/p50c` took `20.6` minutes), while `max_tokens=4096` and `nothink35` take about `3` minutes (`2.7` to `3.2` minutes).

To inspect how answers ended or read single answers on `sampler` (every VM command starts with `. /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code`):

```bash
ssh sampler
. /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code
export CUDA_VISIBLE_DEVICES=""

# Classify how every answer ended across the Lab 5 runs (writes results/lab5/endings.json)
python labs/lab4_endings.py \
  dev/base dev/base+temp1.0 dev/base+pp0 dev/base+pp2 dev/base+rep1.1 dev/base+max4k \
  dev/base+nothink dev/p50c+nothink \
  --out lab5/endings

# List or read individual answers from any decoding run on sampler
python labs/peek.py dev/base+rep1.1 --id gsm8k_train_5628
python labs/peek.py dev/base+nothink --id gsm8k_train_5628
```

`labs/lab4_endings.py` skips `dev/base+nothink` and `dev/p50c+nothink` with a message (`thinking is off, so its answers have no </think> to look for`).

To redraw `results/figures/lab5_decoding.png` on this laptop:

```bash
# On the local laptop, inside ~/weight-space-programming/code
python3 scripts/summarize.py results/runs
uv run -q --with numpy --with matplotlib python scripts/plot_runs.py
```

## What you should see

### 1. All nine runs on `dev256` (`512` answers per run)

Every run uses the same 256 unseen practice problems (128 GSM8K-train + 128 MATH-train, 2 answers each = 512 answers). Paired gain compares each run against `dev/base`, except `dev/p50c+nothink`, which `scripts/summarize.py` pairs against `dev/base+nothink` to isolate the weight edit when thinking is off:

| Run | Setting | Accuracy % | Cut at the cap % | Mean tokens | Correct when finished % | Cut answers ending in repetition % | Paired gain vs baseline (pp) | Rescued / lost problems |
|---|---|---|---|---|---|---|---|---|
| `dev/base` | Unedited, paper settings (`think35`) | `12.1 +/- 1.6` | `88.7` | `29,683` | `100.0` | `96.9` | `-` | `-` |
| `dev/base+temp1.0` | Temperature `1.0` (from `0.6`) | `27.9 +/- 2.5` | `71.3` | `27,430` | `95.9` | `59.5` | `+15.8 +/- 2.1` | `52 / 9` |
| `dev/base+pp0` | Presence penalty `0` (from `1.5`) | `4.5 +/- 0.9` | `99.0` | `32,472` | `100.0` | `98.4` | `-7.6 +/- 1.6` | `11 / 42` |
| `dev/base+pp2` | Presence penalty `2.0` (from `1.5`) | `16.2 +/- 1.9` | `84.6` | `28,614` | `98.7` | `96.8` | `+4.1 +/- 1.7` | `24 / 13` |
| `dev/base+rep1.1` | Repetition penalty `1.1` (from `1.0`) | `34.4 +/- 2.6` | `61.7` | `23,301` | `89.8` | `88.3` | `+22.3 +/- 2.2` | `62 / 3` |
| `dev/base+max4k` | Cap at `4,096` tokens (from `32,768`) | `5.9 +/- 1.1` | `95.5` | `4,019` | `100.0` | `18.2` | `-6.2 +/- 1.2` | `3 / 30` |
| `dev/base+nothink` | Thinking off, cap `8,192` (`nothink35`) | `52.1 +/- 2.7` | `18.0` | `2,270` | `63.6` | `7.6` | `+40.0 +/- 2.5` | `115 / 2` |
| `dev/p50c` | Edited (`p50c`), paper settings | `61.5 +/- 2.8` | `19.7` | `16,104` | `76.6` | `24.8` | `+49.4 +/- 2.6` | `125 / 0` |
| `dev/p50c+nothink` | Edited (`p50c`), thinking off | `53.5 +/- 2.8` | `16.4` | `2,306` | `64.0` | `0.0` | `+1.4 +/- 1.9` (vs `base+nothink`) | `22 / 26` |

![Three bar charts on the same 256 practice problems: accuracy, percent of answers cut at the token cap, and mean answer length, for the unedited model with six decoding changes and for the edited model with thinking on and off.](../results/figures/lab5_decoding.png)

### 2. Breakdown by problem source (GSM8K vs MATH) and median length of correct answers

Splitting `dev256` into its 128 GSM8K problems and 128 harder MATH problems (`results/summary.json`) shows where the tokens go and where thinking mode pays off:

| Run | GSM8K accuracy % / cut % | MATH accuracy % / cut % | Median length of correct answers (tokens) |
|---|---|---|---|
| `dev/base` (unedited, thinking on) | `15.2` / `86.3` | `9.0` / `91.0` | `5,600` (`5,600.5`) |
| `dev/base+temp1.0` | `35.9` / `62.1` | `19.9` / `80.5` | `12,548` |
| `dev/base+pp0` (presence penalty `0`) | `6.2` / `98.4` | `2.7` / `99.6` | `32,768` |
| `dev/base+pp2` (presence penalty `2.0`) | `20.7` / `80.1` | `11.7` / `89.1` | `5,353` |
| `dev/base+rep1.1` (repetition penalty `1.1`) | `46.5` / `48.8` | `22.3` / `74.6` | `5,700.5` |
| `dev/base+max4k` (cap `4,096`) | `7.4` / `94.1` | `4.3` / `96.9` | `3,307` |
| `dev/base+nothink` (thinking off) | `58.2` / `4.3` | `46.1` / `31.6` | `452` |
| `dev/p50c` (edited, thinking on) | `74.6` / `4.7` | `48.4` / `34.8` | `7,998` |
| `dev/p50c+nothink` (edited, thinking off) | `62.1` / `2.0` | `44.9` / `30.9` | `480` (`479.5`) |

In `dev/base+pp0`, removing the presence penalty sends `99.0%` of answers (`98.4%` on GSM8K, `99.6%` on MATH) to the `32,768`-token cap. The median length of its `23` correct answers is `32,768` tokens because `18` of those `23` were cut while thinking after boxing an answer mid-scratchpad. In `dev/base+max4k`, because the unedited model's median correct answer is `5,600` tokens, a `4,096`-token cap cuts off valid reasoning before completion and drops accuracy from `12.1%` to `5.9%` (`30` problems lost, `3` rescued).

### 3. How every thinking-mode answer ended (`results/lab5/endings.json`)

Ending counts from `results/lab5/endings.json` (plus `dev/p50c` from `results/lab4/endings.json`), with correct answers in parentheses:

| Run (`512` answers each) | Total correct | Cut while thinking (correct) | Stopped while thinking (correct) | Closed out of a loop (correct) | Closed normally (correct) |
|---|---|---|---|---|---|
| `dev/base` | `62` | `453 (4)` | `0 (0)` | `3 (2)` | `56 (56)` |
| `dev/base+temp1.0` | `143` | `352 (2)` | `0 (0)` | `3 (1)` | `157 (140)` |
| `dev/base+pp0` | `23` | `507 (18)` | `0 (0)` | `1 (1)` | `4 (4)` |
| `dev/base+pp2` | `83` | `433 (5)` | `0 (0)` | `4 (4)` | `75 (74)` |
| `dev/base+rep1.1` | `176` | `294 (0)` | `0 (0)` | `28 (11)` | `190 (165)` |
| `dev/base+max4k` | `30` | `484 (7)` | `0 (0)` | `0 (0)` | `28 (23)` |
| `dev/p50c` (weight edit) | `315` | `72 (0)` | `0 (0)` | `96 (29)` | `344 (286)` |

Compare the last two columns (`closed out of a loop` and `closed normally`):

- Raising temperature to `1.0` increases `closed normally` from `56` to `157`, while `closed out of a loop` stays at `3`.
- Raising presence penalty to `2.0` gives `4` loop escapes; adding repetition penalty `1.1` gives `28` loop escapes and `190` normal closes.
- The weight edit (`dev/p50c`) produces `344` normal closes and `96` loop escapes. These endings suggest decoding tweaks mostly add normal closes, while the weight edit adds both. A normal close can still follow an earlier loop, because the check reads only the last `6,000` characters before `</think>`.

## Why it works

### 1. Decoding from first principles: how vLLM turns logits into a chosen token

At each generation step, the language model outputs a vector of raw scores called logits, one number $z_v$ for every token $v$ in the vocabulary. Without decoding tweaks, the softmax function turns those logits into probabilities by exponentiating each score and dividing by the sum:

$$p_v = \frac{e^{z_v}}{\sum_{w} e^{z_w}}$$

Because $e^{z_v}$ grows exponentially, adding $+3$ to a token's logit multiplies its relative odds by $e^3 \approx 20$.

In vLLM `0.33.0`, sampling follows the four steps below. The helper `pick_prob` in `wsp/protocol.py` copies this order for presence penalty, temperature, top-k, and top-p, and rejects non-default repetition penalties.

```mermaid
flowchart LR
  L["Raw logits z_v<br/>from the model's final layer"] --> Pen["1. Penalties:<br/>repetition penalty (/r or *r),<br/>then presence penalty (-p)"]
  Pen --> Temp["2. Temperature:<br/>divide all logits by T<br/>(T < 1 sharpens, T > 1 flattens)"]
  Temp --> Filter["3. Top-k and Top-p:<br/>keep top 20 tokens, then keep<br/>smallest set with >= 95% mass"]
  Filter --> Pick["4. Sample one token<br/>from the renormalized<br/>probabilities"]
```

Each step transforms a token's score:

1. **Penalties (`repetition_penalty` and `presence_penalty`):**
   - **Repetition penalty $r$ (`1.1` in `dev/base+rep1.1`):** For every token $v$ that has appeared anywhere in the prompt or generated answer so far, divide $z_v$ by $r$ ($z_v \leftarrow z_v / r$) if $z_v$ is positive, or multiply by $r$ ($z_v \leftarrow z_v \times r$) if negative. Both push the logit down in proportion to its size.
   - **Presence penalty $p$ (`1.5` in `"think35"`):** For every token $v$ that has appeared in the generated answer so far (ignoring the prompt), subtract $p$ ($z_v \leftarrow z_v - p$). This penalty is binary: a token used once and a token used 200 times both lose `1.5`.
2. **Temperature $T$ (`0.6` in `"think35"`, `1.0` in `dev/base+temp1.0`):** Divide every logit by $T$ ($z_v \leftarrow z_v / T$). Dividing by $0.6$ multiplies logit gaps by $1 / 0.6 = 1.67$, favoring the top token. Setting $T = 1.0$ leaves gaps unstretched, giving runner-up tokens a larger chance.
3. **Top-k (`20`) and Top-p (`0.95`):** Keep the `20` highest-logit tokens, compute their softmax probabilities, sort them, and drop the lowest-probability tokens whose cumulative probability is at most $1 - 0.95 = 0.05$ (always keeping the top token).
4. **Renormalize and sample:** Divide the kept probabilities by their sum so they add to `1.0`, then pick one token at random.

### 2. A worked example: why token penalties fail to break a strong loop

Suppose the model is inside a loop and about to repeat `"Let's"` (using made-up numbers):

- `"Let's"` has raw logit $z_{\text{loop}} = 12.0$, and `"</think>"` (not yet in the answer) has raw logit $z_{\text{close}} = 5.0$.
- `"Let's"` and the other scratchpad words have already appeared in the answer.

| Setting | Logit of `"Let's"` after penalties | Logit of `"</think>"` after penalties | Gap after dividing by temperature $T$ | Odds ratio $e^{\text{gap}}$ (`"Let's"` vs `"</think>"`) |
|---|---|---|---|---|
| `presence_penalty = 0`, $T = 0.6$ (`dev/base+pp0`) | $12.0$ | $5.0$ | $(12.0 - 5.0) / 0.6 = 11.67$ | $e^{11.67} \approx 117,000 : 1$ |
| Paper settings: `presence_penalty = 1.5`, $T = 0.6$ (`dev/base`) | $12.0 - 1.5 = 10.5$ | $5.0$ | $(10.5 - 5.0) / 0.6 = 9.17$ | $e^{9.17} \approx 9,600 : 1$ |
| `presence_penalty = 2.0`, $T = 0.6$ (`dev/base+pp2`) | $12.0 - 2.0 = 10.0$ | $5.0$ | $(10.0 - 5.0) / 0.6 = 8.33$ | $e^{8.33} \approx 4,150 : 1$ |
| `presence_penalty = 1.5`, $T = 1.0$ (`dev/base+temp1.0`) | $12.0 - 1.5 = 10.5$ | $5.0$ | $(10.5 - 5.0) / 1.0 = 5.50$ | $e^{5.50} \approx 245 : 1$ |
| `presence_penalty = 1.5`, `repetition_penalty = 1.1`, $T = 0.6$ (`dev/base+rep1.1`) | $(12.0 / 1.1) - 1.5 = 9.41$ | $5.0$ | $(9.41 - 5.0) / 0.6 = 7.35$ | $e^{7.35} \approx 1,550 : 1$ |

This arithmetic explains three patterns in the data:

1. **Why presence penalty does not grow as the loop repeats:** `"Let's"` appeared in the first paragraph, so it pays `-1.5` on cycle 1 and the same `-1.5` on cycle 200. Normal math words (`"Therefore"`, `"answer"`, `"total"`, `"240"`) also appeared earlier in the scratchpad, so each pays `-1.5` too, leaving the gap between `"Let's"` and `"Therefore"` unchanged.
2. **Why `repetition_penalty = 1.1` does better (`34.4%`) than `presence_penalty = 2.0` (`16.2%`):** Dividing a positive logit by `1.1` takes more from a large logit ($12.0 \to 10.91$, losing $1.09$ on top of the $1.5$ presence penalty) than from a moderate runner-up ($8.0 \to 7.27$, losing $0.73$). Even so, `61.7%` of `dev/base+rep1.1` answers hit the cap.
3. **How penalties might prevent some loops yet fail to break others:** When looping and productive continuations have similar logits, higher temperature (`1.0`) or `repetition_penalty = 1.1` can tip the choice away from the first repeat. Once repeated cycles fill the context, loop-token logits can grow high enough that `top_p = 0.95` drops every non-loop token to probability zero. Lab 6 Part 3 measures the per-token pick probability inside 48 real loops: `0.99995` in the unedited model vs `0.979` with the weight edit.

### 3. Why turning `<think>` off scores 52.1%, and why the edit adds nothing there

Switching from `"think35"` to `"nothink35"` (`dev/base+nothink`) raises `dev256` accuracy from `12.1%` to `52.1%` and cuts mean length from `29,683` tokens to `2,270` (median length on correct answers is `452` tokens).

```mermaid
flowchart TD
  Q["Math problem in dev256"] --> TOn["Thinking ON (think35 preset)<br/>Open-ended self-checking scratchpad"]
  Q --> TOff["Thinking OFF (nothink35 preset)<br/>Direct step-by-step solution"]
  TOn --> BaseOn["Unedited (dev/base):<br/>88.7% cut (96.9% of those in repetition)<br/>12.1% acc (GSM8K 15.2%, MATH 9.0%)"]
  TOn --> EditOn["Edited (dev/p50c):<br/>loops suppressed -> 61.5% acc<br/>(GSM8K 74.6%, MATH 48.4%)"]
  TOff --> BaseOff["Unedited (dev/base+nothink):<br/>few loops (7.6% of cuts) -> 52.1% acc<br/>(GSM8K 58.2%, MATH 46.1%)"]
  TOff --> EditOff["Edited (dev/p50c+nothink):<br/>+1.4 +/- 1.9 pp -> 53.5% acc<br/>(GSM8K 62.1%, MATH 44.9%)"]
```

1. **The weight edit helps only when loops occur.** With thinking off (`dev/base+nothink`), `18.0%` of answers hit the 8,192-token cap, and `7.6%` of those cut answers end in repetition (`5.4%` in a verbatim loop). Few answers loop, so with the weight edit and thinking off (`dev/p50c+nothink`) the paired difference against `dev/base+nothink` is `+1.4 +/- 1.9` percentage points (`22` problems rescued, `26` lost), which is indistinguishable from zero.
2. **Once loops are repaired, the thinking preset beats the non-thinking preset, mostly on word problems.** Holding the edited weights fixed, `dev/p50c` (`think35`, `61.5%`) beats `dev/p50c+nothink` (`nothink35`, `53.5%`) by `+8.0 +/- 2.0` paired points (`27` problems rescued, `11` lost). Most of that gain comes from GSM8K (`74.6%` vs `62.1%`, `+12.5 +/- 2.8` points), while the MATH gap is small (`48.4%` vs `44.9%`, `+3.5 +/- 2.9` points). The thinking preset uses about seven times as many tokens (`16,104` vs `2,306` on average). The two presets also differ in sampling settings (`think35` uses temperature `0.6`, top-p `0.95`, and a `32,768`-token cap; `nothink35` uses temperature `0.7`, top-p `0.8`, and an `8,192`-token cap), so part of the gap could come from sampling changes instead of thinking itself.

### 4. The paper's Gemma 3 270M follow-up points the same way

In Theory Guide section 10 (and `external/lm_understand-logs/exp8_math/gemma_followup/README.md`), the authors ran the same weight-space programming setup on a different small model, Gemma 3 270M. On MATH-500, unedited Gemma 3 270M scored `6.8%` and hit the token cap on `8.5%` of answers (`91.5%` finished, yet only `6.8%` of all answers were right).

Across multiple agent runs, no weight edit moved Gemma 3 270M's MATH-500 score outside `6.8%` to `7.7%`. As the blog puts it: *"Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked."* That matches `dev/p50c+nothink`: when answers do not get trapped in loops, subtracting the loop direction `u` does not make the model better at math.

## Check yourself

1. Why did removing the presence penalty (`dev/base+pp0`) hurt accuracy (`12.1%` down to `4.5%`), whereas raising it from `1.5` to `2.0` (`dev/base+pp2`) barely helped (`12.1%` to `16.2%`)?
<details><summary>Answer</summary>

At `presence_penalty = 0`, no penalty discourages reusing earlier output words: `99.0%` of answers hit the cap (`98.4%` of those end in repetition), and only `4` of `512` close normally. With presence penalty `1.5`, already-used output tokens get a `-1.5` logit penalty, and `56` answers close normally (`62` correct in all, `12.1%` accuracy). Raising the flat penalty from `1.5` to `2.0` adds `-0.5` more, which lowers already-used looping tokens and already-used reasoning tokens by the same amount.
</details>

2. In `results/summary.md`, `dev/base+nothink` shows a `+40.0 +/- 2.5` percentage-point gain, while `dev/p50c+nothink` shows `+1.4 +/- 1.9` percentage points even though its accuracy (`53.5%`) exceeds `dev/base+nothink` (`52.1%`). Why do the two gains differ?
<details><summary>Answer</summary>

`scripts/summarize.py` pairs each run against the baseline that isolates one change (`baseline(name)` in `scripts/summarize.py`). `dev/base+nothink` uses `dev/base` (`52.1% - 12.1% = +40.0` points, the effect of turning thinking off on the unedited model). `dev/p50c+nothink` changes both the model and the preset, so `scripts/summarize.py` pairs it against `dev/base+nothink` (`53.5% - 52.1% = +1.4` points, the effect of the weight edit when thinking is already off).
</details>

3. If a teammate suggests saving compute on the unedited thinking model by lowering `max_tokens` from `32,768` to `4,096` (`dev/base+max4k`), what do the numbers show?
<details><summary>Answer</summary>

Mean tokens drop by a factor of 7.4 (`29,683` to `4,019`), and accuracy falls in half (`12.1%` to `5.9%`, losing `30` problems and rescuing `3`). Because the unedited model's correct thinking answers have a `5,600`-token median length, a `4,096`-token cap cuts off over half of the answers that would have finished and solved the problem. For short answers without weight edits, turning `<think>` off (`dev/base+nothink`: `2,270` mean tokens, `52.1%` accuracy) works much better than truncating `<think>` at `4,096` tokens.
</details>

4. In vLLM's sampler (`wsp/protocol.py`), does `presence_penalty` get applied before or after dividing by `temperature`? Why does that order matter when `temperature = 0.6`?
<details><summary>Answer</summary>

`presence_penalty` is subtracted before dividing by `temperature`: `z = (logits - presence_penalty * present) / temperature`. Because `temperature = 0.6` divides the penalized logits by `0.6`, a presence penalty of `1.5` subtracts `1.5 / 0.6 = 2.5` from the post-temperature logit of every token already in the answer. When `dev/base+temp1.0` raises temperature from `0.6` to `1.0`, that effective penalty drops from `2.5` to `1.5`, and every post-temperature logit gap becomes `0.6` times as large for the same raw logits and token history. The raw logits are unchanged.
</details>

## Try this next

1. **Compare how `gsm8k_train_5628` ends across four decoding settings and the edited `dev/p50c` on `sampler`.** Run `labs/peek.py` (CPU only, a few seconds) on the Mr. Desmond problem (`truth: 240`) across `dev/base`, `dev/base+temp1.0`, `dev/base+rep1.1`, `dev/base+nothink`, and `dev/p50c`:
   ```bash
   ssh sampler
   . /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code
   export CUDA_VISIBLE_DEVICES=""
   for r in dev/base dev/base+temp1.0 dev/base+rep1.1 dev/base+nothink dev/p50c; do
     echo "=== $r ==="
     python labs/peek.py "$r" --id gsm8k_train_5628 --chars 600
   done
   ```
   Compare the length of `dev/base+nothink` against `dev/p50c`, and check whether `dev/base+rep1.1` solved `gsm8k_train_5628` or looped.
2. **Test a new decoding combination on 16 problems (not run by us; write down your prediction first).** Combining the two best decoding tweaks on the unedited model (`temperature = 1.0` and `repetition_penalty = 1.1`), or combining the weight edit (`qwen35-p50c`) with `repetition_penalty = 1.1`, was not in `plans/sampler.json`. You can test both in about 5 minutes on 2 GPUs of `sampler` using `"args": ["--limit", "16"]` (as in `plans/quick.json`) and a `"try/"` prefix so `runs/dev/` stays untouched:
   ```json
   {
    "about": "Quick 16-problem test of combined decoding settings.",
    "runs": [
     {
      "name": "try/base+temp1.0+rep1.1",
      "model": "base",
      "data": "dev256",
      "n": 2,
      "shards": 1,
      "est": 1,
      "set": ["temperature=1.0", "repetition_penalty=1.1"],
      "args": ["--limit", "16"]
     },
     {
      "name": "try/p50c+rep1.1",
      "model": "qwen35-p50c",
      "data": "dev256",
      "n": 2,
      "shards": 1,
      "est": 1,
      "set": ["repetition_penalty=1.1"],
      "args": ["--limit", "16"]
     }
    ]
   }
   ```
   Save this JSON on this machine as `plans/try_lab5.json`. From the project root on this machine, sync the plan, run it on `sampler`, and pull the results:
   ```bash
   bash scripts/sync_vm.sh sampler
   ssh sampler '. /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code && python scripts/run_jobs.py plans/try_lab5.json --gpus 0,1' </dev/null
   bash scripts/pull_runs.sh sampler
   ```
   The single shard runs the first `16` rows of `dev256` (all GSM8K problems, `2` answers each). The `try/` prefix gets no paired column from `scripts/summarize.py`, so compare the shared problem IDs directly on this machine:
   ```bash
   python3 - <<'PY'
   from wsp.summary import compare, load_run
   for name in ("base+temp1.0+rep1.1", "p50c+rep1.1"):
       trial = load_run("results/runs/try/" + name)
       for baseline in ("base", "p50c"):
           print(name, "vs dev/" + baseline,
                 compare(load_run("results/runs/dev/" + baseline), trial))
   PY
   ```
   **What we predict:** Combining temperature `1.0` and repetition penalty `1.1` may reduce repetition further, though a gain is not guaranteed. On these `16` problems, our separate runs score `40.6%` (`13/32` answers) at temperature `1.0`, `46.9%` (`15/32`) with repetition penalty `1.1`, and `65.6%` (`21/32`) with `p50c`, with standard errors across problems of about `11` to `12` percentage points, so predictions from this small check are rough. Adding repetition penalty to `p50c` might reduce cuts, but it also penalizes repeated numbers and variables in valid solutions. Because this slice has no MATH problems, it cannot test hard MATH answers.
3. **Test `nothink35` on the three full benchmarks (not run by us; write down your prediction first).** We ran `nothink35` on `dev256` (`52.1%` unedited, `53.5%` edited), not on `math500`, `gsm8k`, or `ifeval`. Scaling from `dev/base+nothink` (`512` answers in `2.73` minutes on `1` GPU) by answer count gives a rough wall-clock estimate of `6` to `15` minutes per benchmark with one shard on `1` GPU (`1,082` IFEval, `2,000` MATH-500, or `2,638` GSM8K answers); answer lengths and throughput may differ. Save this plan on this machine as `plans/try_lab5_nothink.json` (matching the benchmark files and answer counts of the `bench/` runs in `plans/trainer.json`):
   ```json
   {
    "about": "Thinking off (nothink35) on the three full benchmarks, unedited and edited.",
    "runs": [
     {"name": "try/nothink/base/math500", "model": "base", "data": "math500", "n": 4, "shards": 1, "est": 5, "preset": "nothink35"},
     {"name": "try/nothink/base/gsm8k", "model": "base", "data": "gsm8k", "n": 2, "shards": 1, "est": 6, "preset": "nothink35"},
     {"name": "try/nothink/base/ifeval", "model": "base", "data": "ifeval", "n": 2, "shards": 1, "est": 2, "preset": "nothink35"},
     {"name": "try/nothink/p50c/math500", "model": "qwen35-p50c", "data": "math500", "n": 4, "shards": 1, "est": 5, "preset": "nothink35"},
     {"name": "try/nothink/p50c/gsm8k", "model": "qwen35-p50c", "data": "gsm8k", "n": 2, "shards": 1, "est": 6, "preset": "nothink35"},
     {"name": "try/nothink/p50c/ifeval", "model": "qwen35-p50c", "data": "ifeval", "n": 2, "shards": 1, "est": 2, "preset": "nothink35"}
    ]
   }
   ```
   Sync, run on six free GPUs (replace `0-5` with the free GPU indices), and pull the results:
   ```bash
   bash scripts/sync_vm.sh sampler
   ssh sampler '. /nvme/wsp-lab/code/scripts/env.sh && cd /nvme/wsp-lab/code && python scripts/run_jobs.py plans/try_lab5_nothink.json --gpus 0-5' </dev/null
   bash scripts/pull_runs.sh sampler
   ```
   Print each score and the paired gain of the edit on this machine:
   ```bash
   python3 - <<'PY'
   import json
   from wsp.summary import compare, load_run
   for d in ("gsm8k", "math500", "ifeval"):
       for m in ("base", "p50c"):
           s = json.load(open(f"results/runs/try/nothink/{m}/{d}/summary.json"))
           print(f"{m} {d}: {100 * s['acc']:.1f} +/- {100 * s['acc_se']:.1f}%")
       c = compare(load_run(f"results/runs/try/nothink/base/{d}"), load_run(f"results/runs/try/nothink/p50c/{d}"))
       print(f"  p50c minus base, paired: {100 * c['gain']:+.1f} +/- {100 * c['gain_se']:.1f} points")
   PY
   ```
   **What we predict:** On GSM8K test and MATH-500, `base` with `preset: "nothink35"` should mirror the `dev256` split: about `55%` to `60%` on GSM8K (above `14.2%` unedited thinking, below `66.9%` edited thinking) and `42%` to `46%` on MATH-500 (near edited thinking's `51.8%`). On IFEval, which tests formatting constraints instead of long math derivations, `base` with `nothink35` should approach `bench/p50c/ifeval` (`63.4%`), and `p50c` with `nothink35` should match `base` with `nothink35` within noise.