budgie-100m

100M-param ablation checkpoint of budgie-1b (parrot-themed from-scratch reasoning LLM). Base pretrain, 12 layers x 768 d_model, GQA 12Q/3KV, hybrid sliding-window+global attention, QK-norm, value-residual, partial RoPE, tied embeddings. Trained 7629 steps on ~3B tokens (Muon + AdamW, WSD schedule).

Custom architecture — load with trust_remote_code=True or the vendored model.py/config.py.