README.md

---
license: apache-2.0
language:
- en
library_name: transformers
tags:
- text-generation
- causal-lm
- transformer
- argonne
- reasoning
- chain-of-thought
- math
pipeline_tag: text-generation
---

# Argonne 3.5-think

Argonne 3.5-think is a 2.88B-parameter reasoning model trained from scratch, built on
[argonne-3.5-base](https://huggingface.co/PursuitOfDataScience/argonne-3.5-base). It emits an
explicit `<think>…</think>` trace and then a `\boxed{}` answer.

It is the successor to
[Argonne-3.0-think](https://huggingface.co/PursuitOfDataScience/Argonne-3.0-think).

## What changed in this revision (2026-08-04)

**The previous release was trained on a corrupted view of its own data, and this one is not.** Two
argparse defaults in `reasoning/cot-sft.py` — `--max_think_tokens 128` and
`--preserve_raw_reasoning 0` — silently truncated reasoning traces mid-derivation and dropped rows.
Between them they removed about a third of the chain-of-thought tokens, discarded 80.7% of the
arithmetic drill tier, and cut the concluding sentence from most targets. No launcher passed these
flags, so every earlier run inherited them.

Fixing the two defaults — **no new data, no new method, same recipe** — produced this model. The most
consequential effect is on single-step arithmetic, which the previous release got wrong roughly half
the time:

| | previous release | **this release** |
|---|---:|---:|
| one-step arithmetic (`a op b`, 144 items, deployed `.generate()` path) | 80/144 (55.6%) | **143/144 (99.3%)** |
| five-set greedy mean | 50.31 | **57.38** |

The previous card carried this limitation: *"Think-mode can over-step trivial arithmetic. On
'What is 17 − 5?' … the think trace has been observed computing 17−5=12 and then subtracting 5 again
to answer 7."* That was the truncated-data defect showing through, and it is fixed here.

Replicated at **three independent seeds** before release: the five-set mean is 57.25 / 57.35 / 57.38
(spread 0.13pt) and arithmetic is 142/144, 143/144, 144/144.

## Evaluation

Greedy, paired against the previous release on identical items. `n` = 1000 (ASDiv, SVAMP), 500
(MAWPS, GSM-Plus), 319 (MATH-500). Significance is exact McNemar on the paired outcomes.

| pool | previous release | **this release** | delta | |
|---|---:|---:|---:|---|
| ASDiv | 70.40 | **74.90** | +4.50 | p<0.01 |
| SVAMP | 64.50 | **69.60** | +5.10 | p<0.01 |
| MAWPS | 57.00 | **61.20** | +4.20 | p<0.05 |
| GSM-Plus | 28.00 | **42.00** | **+14.00** | p<1e-9 |
| MATH-500 | 31.66 | **39.18** | +7.52 | p<0.05 |
| **five-set mean** | **50.31** | **57.38** | **+7.07** | |

With test-time sampling (K=8, temperature 0.8):

| pool | greedy | self-consistency@8 | pass@8 |
|---|---:|---:|---:|
| ASDiv | 74.90 | 81.20 | 91.90 |
| SVAMP | 69.60 | 81.40 | 93.80 |
| MAWPS | 61.20 | 65.80 | 74.60 |
| GSM-Plus | 42.00 | 49.80 | 67.00 |
| MATH-500 | 39.18 | 36.36 | 61.44 |

**GSM8K is contaminated** for Argonne reasoning models and is deliberately not reported. GSM-Plus is
adversarially perturbed GSM8K *test*, so it was audited directly: the training mix's GSM8K tier is
4,338/4,338 from the **train** split with **zero** test items, and no judged GSM-Plus item exceeds
Jaccard 0.60 against any training row (0 hits at ≥0.70 across all 9,233 pool items). That +14.00 is
not memorisation leaking through the perturbation.

**MATH-500 carries measured indirect leakage and should be read with that in mind.** 17 of its 319
items have a near-duplicate in the training mix (worst pair identical except for one digit), inherited
from OpenMathReasoning/Mixture-of-Thoughts-derived tiers. Re-scored on the 302 clean items this model
gets **39.07** versus 39.18 on the full pool, and the previous release 31.46 versus 31.66 — so the gap
is unchanged and the leak does not inflate the comparison. The other four pools are clean by the same
measure.

### General capability

| | previous release | this release |
|---|---:|---:|
| lm-eval 6-task mean (`acc_norm`) | 55.21 | 54.87 |
| instruction-following probe (14 items) | 13/14 | 13/14 |
| 4-quadrant general/math probe | 30/40 | 31/40 |

Flat. The arithmetic and word-problem gains did not come out of general ability.

### Termination

![termination](plots/termination.png)

The defining failure of the 3.0 line was **non-termination** — 50–60% of traces never closed
`</think>`, so the answer was often never emitted. That was fixed by the short-trace mix and remains
fixed here; budget-forcing adds ~1 point, which is the expected signature when there are no unclosed
traces left to recruit.

## Training

| stage | data | detail |
|---|---|---|
| base | — | [argonne-3.5-base](https://huggingface.co/PursuitOfDataScience/argonne-3.5-base), 88.84B tokens, ctx 13,568 |
| 1 — SFT | [UltraChat 200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) | 207,865 rows, 1 epoch, LR 2e-5, effective batch 20 |
| 2 — DPO | [argilla/dpo-mix-7k](https://huggingface.co/datasets/argilla/dpo-mix-7k) | 6,750 pairs, LR 1e-6, β=0.03 |
| 3 — CoT-SFT | short-trace mix, 28,428 rows, all ≤768 tokens | 1 epoch, LR 1e-5, effective batch 12, **traces no longer truncated** |
| 4 — weight soup | — | **0.85 × CoT + 0.15 × DPO** |

Relative to the previous release, stage 3 differs in exactly two ways: reasoning traces are preserved
whole rather than cut at 128 tokens, and 2,000 rows of general-instruction anchor were added back.
That second part matters — restoring the traces alone costs instruction-following (13/14 → 10/14);
with the anchor restored it holds at 13/14 at every seed.

α = 0.85 is a real knee, not a default: α = 0.70 measurably reintroduces non-termination.

## Inference

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "PursuitOfDataScience/Argonne-3.5-think"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.bfloat16
).cuda()

messages = [{"role": "user", "content": "A shop sells pencils 3 for $2. How much do 12 pencils cost?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tokenizer(text, return_tensors="pt")["input_ids"].cuda()

out = model.generate(ids, max_length=ids.shape[1] + 512, do_sample=False)
print(tokenizer.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```

For throughput, prefer vLLM/SGLang over `.generate()`.

**Self-consistency is worth the extra samples.** Sampling K=8 at temperature 0.8 and taking the
majority answer moves ASDiv 74.90 → 81.20 and SVAMP 69.60 → 81.40.

## Usage notes

- Load with `trust_remote_code=True`; `config.json` carries an `auto_map` so the custom
  `argonne2` classes resolve without manual setup.
- The custom `generate` takes `max_length` (total length), not `max_new_tokens`.
- `eos_token_id` is **151645** (`<|im_end|>`) so the assistant turn ends cleanly. Verified for this
  revision: a chat-templated prompt with **no** `eos_token_id` argument terminates on its own.
- `lm_head.weight` is reported missing on load. Expected and benign — embeddings are tied.
- Context length 13,568, inherited from the base.

## Limitations

- **Verbose, and occasionally pads a correct answer with a wrong embellishment** (e.g. appending
  "one of the four main stars in our solar system" to a correct statement that the sun is a star).
- **pass@K is a noisy metric here.** Re-running an identical model and seed reproduced greedy and
  self-consistency exactly but moved pass@8 by several points. Treat pass@K as a ceiling indicator;
  select on self-consistency or greedy.
- **The instruction-following probe is 14 items.** 13/14 at three seeds shows the regression from the
  data fix was repaired; it is not a broad instruction-following benchmark.
- **MATH-500 is not a clean pool for this line** — see the leakage measurement above. Quote the
  302-item clean subset alongside it.
- Grade-school and early-competition arithmetic word problems are the measured domain. Code,
  tool-calling and general-purpose chat are **not** characterized for this revision.
- 2.88B parameters trained on 88.84B tokens — far below frontier compute.
- No safety alignment beyond what UltraChat and the preference data provide.

## Source code

Everything below is on the GitHub `main` branch —
[PursuitOfDataScience/ArgonneAI](https://github.com/PursuitOfDataScience/ArgonneAI/tree/main).

| file | role |
|---|---|
| [`reasoning/thinking_training.md`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/thinking_training.md) | the full build log — **§32** is the original recipe, **§34–§37** are the data-corruption diagnosis, the fix, and this release's gate |
| [`model.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/model.py) | `ArgonneModel` / `ArgonneConfig` + KV cache (bundled here as `model.py`) |
| [`sft.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/sft.py) | stage 1 — instruction SFT |
| [`dpo.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/dpo.py) | stage 2 — preference alignment |
| [`reasoning/cot-sft.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/cot-sft.py) | stage 3 — CoT-SFT, with the corrected flag defaults and a loader audit that aborts on silent row loss |
| [`reasoning/build_ckpt_soup.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/build_ckpt_soup.py) | stage 4 — the α weight soup |
| [`reasoning/effort_gate.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/effort_gate.py) | the paired five-pool gate every number above comes from |
| [`reasoning/simple_arith_probe.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/simple_arith_probe.py) | the one-step arithmetic probe |
| [`reasoning/pool_decontam.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/pool_decontam.py) | the leakage audit and clean-subset re-scoring |
| [`reasoning/clean_eval.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/clean_eval.py) | the uncontaminated SVAMP/ASDiv judge |
| [`reasoning/eval_numeracy.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/reasoning/eval_numeracy.py) | the 4-quadrant general/math probe |

Base model: [argonne-3.5-base](https://huggingface.co/PursuitOfDataScience/argonne-3.5-base)
([training details](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/README.md#argonne-35-base)).

## Citation

```bibtex
@misc{argonne35think,
  author = {PursuitOfDataScience},
  title  = {Argonne 3.5-think},
  year   = {2026},
  publisher = {Hugging Face},
  url    = {https://huggingface.co/PursuitOfDataScience/Argonne-3.5-think}
}
```