cookbook/quickstart.md

# Quickstart — from zero to a small trained model

## 1. The loop that makes it possible
For every phase (pretrain, growth, SFT, DPO, eval) do exactly this:

1. Research the phase (seriously; 20-30 min of reading/searching, not vibes).
2. Write a short "skill" (a markdown repo + decision record) capturing:
   - what big-tech actually says,
   - measured numbers from YOUR device,
   - what failed last time (so you never re-run a dead end).
3. Apply the skill (one narrow change).
4. Gate it with a small eval probe battery before moving on.

That's it. The "skill" is the artifact that separates "a build I guessed at"
from "a build I own and can hand to a team."

## 2. Suggested order
1. **Token + corpus**: build/acquire tokenizer, then a *balanced* training corpus
   (see `corpus.md`). Don't concatenate big blocks; window-shuffle.
2. **Pretrain a base (~7M)** on a constrained domain until it produces coherent
   simple text (that's your floor). Checkpoint every ~500 steps, resume-aware.
3. **Grow it (tower / identity-blocks)** — keep the trained trunk, add
   identity-initialized layers so the loss is unchanged, then continue-pretrain.
   Never start from scratch again.
4. **Curriculum SFT** on hand-written gold (a few hundred consistent examples)
   where labels are checkable from the prompt (verdict classes).
5. **Eval** the probe battery; iterate on real mistakes.
6. **Export GGUF/Q8** for on-device use.

## 3. Copy config minimum (on an 8 GB tablet)
- ≈ 7-25M params, fp/bf16, batch 16, seq 256.
- 1 epoch ≈ 11-20 h depending on size. Always resumable.
- One heavy job at a time — two torch processes starve each other.

## 4. Honest expectations
- Small + niche + honest beats big + generic for *decision-support* work.
- Read `device.md` for real across what a tablet can train in a day.