SmolThinker

SmolLM2-1.7B-Instruct fine-tuned to emit its reasoning inside literal <think> ... </think> blocks, so chat UIs that render collapsible reasoning (Open WebUI, Ollama, LM Studio) display it as a proper thinking section rather than dumping it into the answer.

Output format

Every response has this shape:

<think>
I need to subtract 305 from 701.
Break 305 into 300 and 5.
701 - 300 = 401.
401 - 5 = 396.
</think>
701 - 305 = 396.

Usage

llama-cli -hf sbussiso/SmolThinker --jinja

The --jinja flag matters. It uses the embedded ChatML template, which is what the model was trained against.

For Ollama, ChatML turn format with both markers as stops:

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

Prompt format

ChatML, inherited from SmolLM2:

<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
What is 47 + 68?<|im_end|>
<|im_start|>assistant

Works with or without a system prompt. Roughly 40% of training rows carried no system turn, so the template's injected default is in distribution; the rest used generic prompts naming no model.

Training data

sbussiso/SmolThinker-Synthetic-Low-Reasoning.

This model was trained on the v1 revision of that dataset, which has since been superseded. v1 was 5,880 fully templated examples across 26 task families: multi-step arithmetic, percentages, averages, cost and remainder word problems, rate and age problems, decimal and fraction comparison, letter counting, word reversal, sequences, parity, divisibility, unit conversion, sorting, syllogisms, boolean logic, clock arithmetic, capital cities, and chit-chat.

Evaluating this model surfaced two defects in v1 that are fixed in the current dataset but are baked into these weights:

• 14.5% of clock arithmetic rows had incorrect answers. The AM/PM logic was wrong in both directions, so 12:50 PM + 40 minutes was labelled 1:30 AM. This model scored 50% on that category because it learned from wrong labels.
• Average traces skipped work, jumping from a list of five numbers to the total in one step. Held-out accuracy on that category was 54.5%.
• Three content pools were capped far below the row count. The boolean logic generator could only produce 8 distinct prompts in total, so this model memorised them rather than learning the operation.

The dataset at that link is now v2, which is not what these weights were trained on. v2 fixes all three defects and adds 414 hand-written long-form examples, roughly one row in five, which v1 lacked entirely. Retraining should improve the failure modes documented below. No retrained model has been published yet.

Reasoning traces run 2 to 4 short steps. 100% of rows carry a think block, including greetings, so there is no example anywhere of answering without one.

Design notes

The tags are ordinary text tokens, not special tokens. On the SmolLM2 tokenizer <think> is ['<', 'think', '>'] and </think> is ['</', 'think', '>'], tokenizing identically in every training row. This is deliberate: registering them via add_special_tokens() would give them untrained embeddings and, more importantly, skip_special_tokens=True on decode would strip them from the output, which is the usual reason a reasoning fine-tune produces correct reasoning with no visible tags. Qwen3 makes the same choice, adding them to the vocab but marking them special=False.

Vocab size is 49153 against the base 49152; the single added token is padding.

Evaluation

Measured on the Q4_K_M GGUF via Ollama, greedy decoding.

Format compliance, 40 open-ended prompts deliberately unlike the training data (none appear anywhere in the dataset):

Metric	Result
Perfectly well-formed think block	70.0%
Usable (block plus an answer renders)	90.0%
Produced an answer	100%
No ChatML token leakage	100%
Median think block	137 chars

Failure modes: 20% emitted a spurious trailing </think> after an otherwise correct response, 5% emitted no think block at all, 5% emitted an extra opening tag.

The stray closing tag has an identified cause. Every answer in v1 was a single short clause, median 17 characters, while this model's open-ended answers run around 194 characters. It runs past the answer-length distribution it was trained on and reaches for the closing tag it associates with finishing. The missing think blocks were on creative and opinion prompts, which v1 contained none of.

Held-out accuracy, 120 unseen examples from the same distribution as training:

Metric	Result
Answer accuracy	88.3%
Well-formed think block	98.3%

Formatting is near-perfect in distribution and only degrades on long out-of-distribution answers. Weakest categories were averages at 54.5% and clock arithmetic at 50%, both traced to the dataset defects described above rather than to model capacity.

Not evaluated on GSM8K or any other public benchmark, so there is no measurement of whether fine-tuning helped or harmed general math ability relative to the base model.

Limitations

Training data is templated and synthetic, so phrasing diversity is bounded and the task set is deliberately narrow. This teaches the shape of reasoning and reliable tag emission, not general reasoning ability. Expect arithmetic to degrade outside the ranges seen in training, and expect the model to produce confident-looking traces for problems it gets wrong. Clock arithmetic in particular is unreliable, for the reason given above.

If you are consuming the output programmatically, strip stray </think> tags after the first closing tag rather than assuming exactly one.

Built with Unsloth.