andro124543/1b-sft-dataset

1B-SFT — High-Quality Instruction Dataset for a ~1B LLM

A curated, deduplicated instruction-tuning dataset in ShareGPT format, built for fine-tuning a 1B-class model (Qwen2.5-1.5B, Llama-3.2-1B, Gemma-2-2B).

5,235 samples — train 4,919 / val 158 / test 158.

Quality guarantees

• Math: every answer is computed by the generator (arithmetic, percentages, word problems, multi-step problems, unit conversions, linear equations, sequences, fractions) with step-by-step working.
• Coding: every code sample is executed and unit-tested before inclusion (33 task types + 10 bug-fix tasks); the shipped code blocks were re-executed against the final files (1,047/1,047 pass).
• Reasoning: ordering puzzles, liar-guard puzzles, syllogisms, seating puzzles — all consistent by construction.
• Hand-written core: ~260 manually authored seeds across knowledge, explain, creative writing, roleplay (flirty companion, boxing coach, motorcycle buddy, DM), advice, summarization, rewriting, and multi-turn chat.
• Exact + SimHash near-duplicate removal; artifact filtering; deterministic split (94/3/3).

Format (ShareGPT, one JSON per line)

{"system": "optional persona...",
 "conversations": [
   {"from": "human", "value": "..."},
   {"from": "gpt", "value": "..."}]}

Category breakdown

Category	Count
math	3,251
coding	1,055
reasoning	549
trivia	140
knowledge	52
roleplay	47
creative	42
explain	36
advice	17
rewrite	17
chat	16
summarization	13

Loading

from datasets import load_dataset
ds = load_dataset("andro124543/1b-sft-dataset", split="train")

or use train.jsonl / val.jsonl / test.jsonl directly with LLaMA-Factory (sharegpt formatting) or axolotl.

Suggested recipe for a 1B model (SFT)

• Base: Qwen/Qwen2.5-1.5B (ChatML) or meta-llama/Llama-3.2-1B
• lr 2e-5 cosine, batch 8–16, 3 epochs, max_seq_len 1024–2048, packing on
• QLoRA 4-bit (rank 32) for limited compute; export to GGUF for Ollama

Rebuild / extend

The dataset is fully reproducible: content.py (hand-written seeds), generators.py (verified programmatic generators), build.py (dedup → filter → split → stats). Run python build.py to regenerate deterministically.