Keep your dataset small and focused.
Pretrain (corpus_raw.txt)
Short concept notes you wrote yourself (or public domain). Aim 0.5–5 MB:
- definitions (inertia, momentum, energy…)
- short derivations (one paragraph)
- worked mini examples (1–3 lines)
SFT (sft_train.jsonl)
Each line: {"instruction": "...", "output": "..."} Keep answers short and factual. 200–1000 examples is fine.
Do not include copyrighted textbook dumps.