samueljayasingh/rune-0.3b-sft

Rune-R1 SFT (351M)

A ~351M parameter decoder-only transformer, supervised-fine-tuned from Rune-R1 Base to follow a chain-of-thought math answer format. This is stage 2 of the Rune-R1 pipeline (Pretrain → SFT → GRPO); it teaches the format (<think>...</think> + final answer) that the subsequent GRPO stage then optimizes for correctness. See Rune-R1 for the final, RL-tuned reasoning model.

Model Description

• Developed by: samueljayasingh
• Model type: Causal language model (text-only)
• Base model: Rune-R1 Base (351M, pretrained from scratch on FineWeb-Edu)
• Fine-tuning method: Full-parameter supervised fine-tuning (no LoRA/adapters) on distilled chain-of-thought traces
• Dataset: rasbt/math_distill (deepseek-r1-math-train partition) — DeepSeek-R1-distilled chain-of-thought solutions to math problems
• Language: English
• Tokenizer: GPT-2 (tiktoken) — no special tokens added; CoT format is plain text (<think>...</think>\n\n{answer})
• License: Apache 2.0

Intended Uses & Limitations

Intended use:

• Research into chain-of-thought fine-tuning and format-learning as a precursor to RL (GRPO/RLVR).
• Base policy and frozen reference model for the GRPO stage of this pipeline.
• Studying supervised-fine-tuning dynamics (train/val loss, overfitting) at small parameter counts.

Limitations:

• Trained for 2 epochs on ~4k distilled examples — a small SFT budget; expect inconsistent answer correctness (this model is not optimized for accuracy, only for producing the <think>...</think> + answer format). Correctness is the job of the downstream GRPO stage.
• Inherits all base-model limitations: 1024-token context, ~5B pretraining tokens, no broad safety/RLHF alignment.
• Not evaluated for correctness on MATH-500 at this stage — see the GRPO model card for those numbers.

How to Use

import torch
import tiktoken
from rune.model import CONFIG_350M, RuneModel

ckpt = torch.load("pytorch_model.bin", map_location="cpu")
model = RuneModel(CONFIG_350M)
model.load_state_dict(ckpt)
model.eval()

enc = tiktoken.get_encoding("gpt2")
prompt = "What is 12 * 15?"
tokens = torch.tensor([enc.encode(prompt)], dtype=torch.long)

# Model responds in "<think>...reasoning...</think>\n\n{final answer}" format.
# See rune/generate.py in the source repo for full sampling / KV-cache generation code.

The rune package (model definition + generation utilities) is available at the Rune-R1 GitHub repository.

Training & Evaluation

Training procedure

Parameter	Value
Base checkpoint	checkpoints/ckpt_latest.pt (Rune-R1 Base)
Dataset	rasbt/math_distill, deepseek-r1-math-train partition
Epochs	2
Validation split	200 held-out examples
Max sequence length	1024 tokens
Gradient accumulation	8 steps
Learning rate	1e-5
Fine-tuning type	Full-parameter (no LoRA)
Hardware	1x AMD MI300X GPU

Evaluation results

Metric	Value
Final validation loss	1.400
Best (min.) validation loss	1.400 (end of epoch 2)
Validation loss, end of epoch 1	1.541
Avg. training loss, last 20 logged steps	1.374

Validation loss is next-token cross-entropy on the held-out 200-example split; it measures how well the model reproduces the distilled CoT format and answers, not mathematical correctness (see the GRPO model's MATH-500 accuracy for that).

Citation

@misc{RuneR1SFT2026,
  author = {Samuel Jayasingh},
  title = {Rune-R1 SFT: Chain-of-Thought Fine-Tuning of a 351M Transformer on Distilled Math Solutions},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/samueljayasingh/Rune-R1-sft}}
}

Acknowledgements

• rasbt/math_distill — DeepSeek-R1-distilled chain-of-thought training data.
• rasbt/LLMs-from-scratch — base architecture and reasoning-from-scratch training recipe this pipeline follows.