Model A — From-Scratch Mini Chat Companion — Build Checklist

Pure NumPy (CuPy swap allowed for GPU). Tiny. General chat only — no code, no tools. Build + test each yourself; don't move on until current item works. Every autograd op passes gradcheck; every layer passes a shape/grad test; training passes a one-paragraph overfit before scaling.

Scope

Property	Target
Capability	short English small-talk / companion replies
Explicitly NOT	code, math tools, retrieval, function-calling
Params	~1–8M (CPU-trainable)
Arch	modern-tiny: RoPE + RMSNorm + SwiGLU + GQA, weight-tied head
Framework	pure NumPy (`np`), optional `ZYN_BACKEND=cuda` CuPy swap
Tokenizer	small byte-BPE, vocab ~4k, chat special tokens

Foundation

Folder layout (mla/, tests/, data/, checkpoints/, serve/)
Backend switch (NumPy / CuPy, dtype float64 for gradcheck / float32 GPU)
Numerical gradient checker (finite diff + rel-error)
Autograd Tensor (add, mul, matmul, reshape, transpose, sum, gather)
Activations as ops (silu/gelu, exp, log, softmax, rsqrt)
Reverse-mode backward (topological order) + broadcast-aware grads

Data (chat only)

Collect + clean small-talk / dialogue corpus (chit-chat, no code)
Chat formatting (<|user|> / <|assistant|> turns, <bos>/<eos>)
Train/val split (by conversation, seed=42) + dedup
Byte-BPE tokenizer (train ~4k, encode, decode, save/load)
Chat special tokens reserved before byte base
Batching (packed turns → next-token x/y windows)

Model (modern-tiny decoder)

Token embeddings (tied to output head)
RoPE (rotary positional embeddings)
RMSNorm (Pre-LN placement)
Self-attention (Q/K/V, scaled, causal mask)
Grouped-query attention (fewer KV heads)
QK-Norm before RoPE (stability)
SwiGLU MLP (gated, 2/3 hidden-dim rule)
Residual connections
Stack blocks → full model + config

Pretraining

Cross-entropy loss (ignore-index for padding)
AdamW optimizer + grad clipping
LR warmup + cosine decay
Sanity: overfit one dialogue (loss < 0.05)
Training loop + checkpoint save/load/resume

Chat fine-tune

Instruction/chat dataset + loss masking (train assistant turns only)
Fine-tune from pretrain checkpoint
Persona/system prompt conditioning (companion tone)

Evaluation

Val loss + perplexity
Next-token accuracy
Sample chat turns per checkpoint (manual read)
Refusal check: coding prompt → stays in-scope (no code attempt)

Inference

Sampling (greedy, temperature, top-k, top-p)
KV-cache (logits identical to full forward)
Chat runtime (render turns → generate → stop on <eos>)

Hosting

Package model + tokenizer + config bundle
Inference-only load mode
Serving API (/chat, /health)
Dockerfile + env config
Deploy + smoke test live URL

Scaling (later)

NumPy → CuPy for GPU train/infer
Bigger config + larger chat corpus
Batched/concurrent serving