--- base_model: HuggingFaceTB/SmolLM2-1.7B-Instruct datasets: - sbussiso/SmolThinker-Synthetic-Low-Reasoning library_name: transformers license: apache-2.0 language: - en pipeline_tag: text-generation tags: - gguf - llama.cpp - unsloth - lora - reasoning - chain-of-thought - think-tags - smollm2 --- # SmolThinker `SmolLM2-1.7B-Instruct` fine-tuned to emit its reasoning inside literal `` ... `` blocks, so chat UIs that render collapsible reasoning (Open WebUI, Ollama, LM Studio) display it as a proper thinking section rather than dumping it into the answer. Reasoning is deliberately brief, around three short lines. The goal is reliable tag emission and visible working, not long deliberation. ## Output format ``` I need to subtract 305 from 701. Break 305 into 300 and 5. 701 - 300 = 401. 401 - 5 = 396. 701 - 305 = 396. ``` ## Usage ```bash llama-cli -hf sbussiso/SmolThinker --jinja ``` The `--jinja` flag matters. It uses the embedded ChatML template, which is what the model was trained against. For Ollama, ChatML turn format with both markers as stops: ``` PARAMETER stop "<|im_start|>" PARAMETER stop "<|im_end|>" ``` ## Prompt format ChatML, inherited from SmolLM2: ``` <|im_start|>system You are a helpful AI assistant.<|im_end|> <|im_start|>user What is 47 + 68?<|im_end|> <|im_start|>assistant ``` Works with or without a system prompt. Roughly 40% of training rows carried no system turn, so the template's injected default is in distribution; the rest used generic prompts naming no model. Multi-turn works too. About 11% of training rows are 2 to 3 turn conversations where each follow-up depends on an earlier turn, so the model reads prior context rather than treating every message as fresh. ## Training data [sbussiso/SmolThinker-Synthetic-Low-Reasoning](https://huggingface.co/datasets/sbussiso/SmolThinker-Synthetic-Low-Reasoning), 2,593 rows. | Source | Rows | | --- | ---: | | Templated single-turn, 26 task families | 1,800 | | Templated multi-turn | 200 | | Hand-written single-turn | 514 | | Hand-written multi-turn | 79 | 593 rows are authored individually rather than generated. 100% of assistant turns carry a think block, including greetings, so there is no example anywhere of answering without one. ## Training configuration LoRA adapter on a 16-bit base, trained with [Unsloth](https://github.com/unslothai/unsloth) Studio on a single NVIDIA L4. ```yaml method: LoRA (16-bit base) num_epochs: 2 max_seq_length: 2048 learning_rate: 2e-4 lr_scheduler: linear warmup_steps: 50 batch_size: 2 grad_accumulation: 4 # effective batch 8 optimizer: adamw_8bit weight_decay: 0.001 packing: false train_on_completions: true # loss on assistant turns only random_seed: 3407 lora_r: 16 lora_alpha: 16 lora_dropout: 0 target_modules: q_proj k_proj v_proj o_proj gate_proj up_proj down_proj ``` 614 steps, 762,325 tokens, 11m45s. Final training loss 0.639, final evaluation loss approximately 0.600, final gradient norm 0.414. ### Why two epochs Three runs were compared. At three epochs the model overfits: evaluation loss bottoms around step 591 and then climbs while training loss keeps falling. | Run | Config | Final train loss | Final eval loss | Overfit | | --- | --- | ---: | ---: | --- | | 1 | 3 epochs, rank 32 | 0.452 | ~0.645 | yes, 0.050 | | 2 | 3 epochs, rank 16 | 0.517 | ~0.621 | yes, 0.030 | | 3 | **2 epochs, rank 16** | **0.639** | **~0.600** | **none** | Both three-epoch runs turned upward at the same step regardless of rank, so the cause was epoch count against a 2,453-row training split rather than adapter capacity. Run 3 is this release: higher training loss with the lowest evaluation loss, and a gradient norm that stays flat instead of climbing, which is what generalisation rather than memorisation looks like. ## Design notes The tags are **ordinary text tokens**, not special tokens. On the SmolLM2 tokenizer `` is `['<', 'think', '>']` and `` is `['']`, tokenizing identically in every training row. This is deliberate: registering them via `add_special_tokens()` would give them untrained embeddings and, more importantly, `skip_special_tokens=True` on decode would strip them from the output, which is the usual reason a reasoning fine-tune produces correct reasoning with no visible tags. Qwen3 makes the same choice, adding them to the vocab but marking them `special=False`. `embedding_learning_rate` was left unset for the same reason: the tags are ordinary tokens and the embedding layer does not need to move. ## Evaluation Measured on GSM8K (OpenAI grade-school math), a 300-problem subset of the 1,319-problem test set. Zero-shot: the raw question is sent through each model's chat template with no few-shot examples. All four models are evaluated via Ollama at Q4_K_M, so the only variable between SmolThinker and its base is the fine-tune. SmolThinker emits its `` block then a final answer; the final number is parsed out and graded against the GSM8K answer key. ### GSM8K accuracy and format compliance | Model | GSM8K acc | Well-formed | n | | --- | ---: | ---: | ---: | | **SmolThinker (this model, Q4_K_M)** | 27.7% | **94.3%** | 300 | | SmolLM2-1.7B-Instruct (base, Q4_K_M) | 32.7% | 0.0% | 300 | | Qwen2.5-1.5B-Instruct (Q4_K_M) | 64.3% | 0.0% | 300 | | Llama-3.2-1B-Instruct (Q4_K_M) | 46.7% | 0.0% | 300 | Well-formed means a single closed `` block followed by a non-empty answer. Only SmolThinker emits these tags, so the column is meaningful only for it. ![GSM8K accuracy against base and peers](benchmark_chart.png) ### Format compliance Measured on 40 open-ended prompts written to be unlike anything in the training data (why is the sky blue, write a two-line poem, how do I decline a meeting). None of them appear anywhere in the dataset. The v1 column is the previous release, trained on v1 of the dataset. | Metric | v1 | This release | | --- | ---: | ---: | | Well-formed think block | 70.0% | **97.5%** | | Usable (block plus an answer) | 90.0% | **97.5%** | | Produced an answer | 100% | 100% | | No ChatML leakage | 100% | 100% | v1 emitted a spurious trailing `` on 8 of 40 responses and an extra opening tag on 2 more. This release does neither: the only remaining failure is one creative-writing prompt where it answers without a think block. The cause of the v1 defect was answer length. Every answer in v1 was a single short clause of about 17 characters, so on open-ended prompts the model ran past the answer-length distribution it had learned and reached for the closing tag it associated with finishing. v2 added hand-written long-form and medium-form answers to fill that gap, and the defect disappeared. ### Held-out accuracy On the dataset's own 140-row validation split: **59.3% accuracy, 98.6% well-formed**. This is **not comparable** to the previous release's 88.3%, because the validation split changed between versions in both size and content. v2's split contains the hand-written long-form and multi-turn rows that v1's did not, and those are substantially harder to grade than a templated arithmetic answer. Treat it as a v2 baseline for future runs rather than as a regression. ### What the fine-tune does and does not do The fine-tune reliably produces clean, leak-free structured reasoning. On the 300 held-out GSM8K problems SmolThinker emitted a well-formed `` block 94.3% of the time, always produced a parseable final answer (100%), and never leaked ChatML markers (`<|im_start|>` / `<|im_end|>`) into its output. Median reasoning length is 158 characters, around three short lines, which is the intended behavior. It does **not** improve GSM8K accuracy over its base. SmolThinker scores 27.7% against the base's 32.7%, about 5 points lower, and that gap is **not statistically significant at this sample size**. The 95% confidence interval on the difference is [-2.3, +12.3] points, and McNemar's test on the discordant pairs gives chi-squared 2.42 against the 3.84 needed for p < 0.05. So the honest reading is "no measurable difference in either direction", not a regression and not parity. The head-to-head on the same 300 problems shows the shape of it: | Outcome | Count | | --- | ---: | | Both correct | 50 | | Both wrong | 169 | | SmolThinker correct, base wrong | 33 | | Base correct, SmolThinker wrong | 48 | | Net | -15 | The reasoning helps on 33 problems the base missed and hurts on 48 problems the base got right, netting -15, which is the accuracy gap. Two failure patterns account for most of the losses. When the think block breaks (17 of 300 rows) the final-number parser pulls a nonsense value, and when the model skips reasoning it answers like a weaker SmolLM2. The concise reasoning this fine-tune produces is sometimes insufficient for multi-step arithmetic. ### Context The base this model is built on, SmolLM2-1.7B-Instruct, is itself a weak math model at roughly 33% on this subset, well behind same-class peers: Qwen2.5-1.5B scores 64.3% and Llama-3.2-1B scores 46.7%. SmolThinker did not start from a strong base and the fine-tune did not change that. **In short: SmolThinker is a formatting fine-tune. It teaches reliable, visible, structured chain-of-thought emission. It does not improve, and may slightly reduce, raw math accuracy versus the base.** These are 300-problem subset numbers. A full 1,319-problem run would tighten the accuracy estimate. Broader benchmarks (ARC, HellaSwag, MMLU) have not yet been run, so general-knowledge retention versus the base is not yet measured. Methodology and raw per-problem results live in the `bench/` directory of the source repository. ## Limitations Training data is largely templated and synthetic, so phrasing diversity in that portion is bounded and the task set is deliberately narrow. This teaches the shape of reasoning and reliable tag emission, not general reasoning ability. Expect arithmetic to degrade outside the ranges seen in training, and expect confident-looking traces on problems the model gets wrong. The 593 hand-written rows are a finite set, so some memorisation of them is possible. Evaluated on GSM8K only (see above). The fine-tune does not improve raw math accuracy over the base and trends slightly lower, so this is a formatting fine-tune rather than a capability upgrade. Broader benchmarks (ARC, HellaSwag, MMLU) have not been run, so general-knowledge retention versus the base is not measured. If you consume the output programmatically, strip any stray `` after the first closing tag rather than assuming exactly one. Built with [Unsloth](https://github.com/unslothai/unsloth).