README.md

---
language:
- en
license: apache-2.0
tags:
- llm
- pytorch
- causal-lm
- rune-r1
- reasoning
- sft
- chain-of-thought
datasets:
- rasbt/math_distill
metrics:
- perplexity
pipeline_tag: text-generation
base_model:
- samueljayasingh/rune-0.3b-base
---

# Rune-R1 SFT (351M)

A ~351M parameter decoder-only transformer, supervised-fine-tuned from [Rune-R1 Base](https://huggingface.co/samueljayasingh/Rune-R1-base) to follow a chain-of-thought math answer format. This is stage 2 of the Rune-R1 pipeline (**Pretrain → SFT → GRPO**); it teaches the *format* (`<think>...</think>` + final answer) that the subsequent GRPO stage then optimizes for *correctness*. See [Rune-R1](https://huggingface.co/samueljayasingh/Rune-R1) for the final, RL-tuned reasoning model.

## Model Description

| | |
|---|---|
| **Developed by** | samueljayasingh |
| **Model type** | Causal language model (text-only) |
| **Base model** | [Rune-R1 Base](https://huggingface.co/samueljayasingh/Rune-R1-base) (351M, pretrained from scratch on FineWeb-Edu) |
| **Fine-tuning method** | Full-parameter supervised fine-tuning (no LoRA/adapters) on distilled chain-of-thought traces |
| **Dataset** | [rasbt/math_distill](https://huggingface.co/datasets/rasbt/math_distill) (`deepseek-r1-math-train` partition) — DeepSeek-R1-distilled chain-of-thought solutions to math problems |
| **Language** | English |
| **Tokenizer** | GPT-2 (`tiktoken`) — no special tokens added; CoT format is plain text (`<think>...</think>\n\n{answer}`) |
| **License** | Apache 2.0 |

## Intended Uses & Limitations

### Intended Use
- Research into chain-of-thought fine-tuning and format-learning as a precursor to RL (GRPO/RLVR).
- Base policy and frozen reference model for the GRPO stage of this pipeline.
- Studying supervised-fine-tuning dynamics (train/val loss, overfitting) at small parameter counts.

### Limitations
- Trained for 2 epochs on ~4k distilled examples — a small SFT budget; expect inconsistent answer correctness (this model is *not* optimized for accuracy, only for producing the `<think>...</think>` + answer format). Correctness is the job of the downstream GRPO stage.
- Inherits all base-model limitations: 1024-token context, ~5B pretraining tokens, no broad safety/RLHF alignment.
- Not evaluated for correctness on MATH-500 at this stage — see the GRPO model card for those numbers.

## How to Use

```python
import torch
import tiktoken
from rune.model import CONFIG_350M, RuneModel

ckpt = torch.load("pytorch_model.bin", map_location="cpu")
model = RuneModel(CONFIG_350M)
model.load_state_dict(ckpt)
model.eval()

enc = tiktoken.get_encoding("gpt2")
prompt = "What is 12 * 15?"
tokens = torch.tensor([enc.encode(prompt)], dtype=torch.long)

# Model responds in "<think>...reasoning...</think>\n\n{final answer}" format.
# See rune/generate.py in the source repo for full sampling / KV-cache generation code.
```

The `rune` package (model definition + generation utilities) is available at the [Rune-R1 GitHub repository](https://github.com/samueljayasingh/Rune-R1).

## Hardware

Trained on a single rented GPU instance:

| Component | Spec |
|---|---|
| GPU | 1x AMD MI300X |
| VRAM | 192 GB |
| vCPU | 20 |
| RAM | 240 GB |
| Boot disk | 720 GB NVMe SSD |
| Scratch disk | 5 TB NVMe SSD |
| Rate | $1.99/hr |

## Training & Evaluation

### Training Procedure

| Parameter | Value |
|---|---|
| Base checkpoint | `checkpoints/ckpt_latest.pt` (Rune-R1 Base) |
| Dataset | `rasbt/math_distill`, `deepseek-r1-math-train` partition |
| Epochs | 2 |
| Validation split | 200 held-out examples |
| Max sequence length | 1024 tokens |
| Gradient accumulation | 8 steps |
| Learning rate | 1e-5 |
| Fine-tuning type | Full-parameter (no LoRA) |
| Hardware | 1x AMD MI300X GPU |

### Evaluation Results

| Metric | Value |
|---|---|
| Final validation loss | 1.400 |
| Best (min.) validation loss | 1.400 (end of epoch 2) |
| Validation loss, end of epoch 1 | 1.541 |
| Avg. training loss, last 20 logged steps | 1.374 |

Validation loss is next-token cross-entropy on the held-out 200-example split; it measures how well the model reproduces the distilled CoT format and answers, not mathematical correctness (see the GRPO model's MATH-500 accuracy for that).

## Citation

```bibtex
@misc{RuneR1SFT2026,
  author = {Samuel Jayasingh},
  title = {Rune-R1 SFT: Chain-of-Thought Fine-Tuning of a 351M Transformer on Distilled Math Solutions},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/samueljayasingh/Rune-R1-sft}}
}
```

## Acknowledgements

- [rasbt/math_distill](https://huggingface.co/datasets/rasbt/math_distill) — DeepSeek-R1-distilled chain-of-thought training data.
- [rasbt/LLMs-from-scratch](https://github.com/rasbt/LLMs-from-scratch) — base architecture and reasoning-from-scratch training recipe this pipeline follows.