lm-2b-typemix-2013

A 2B-parameter ChronoGPT causal language model fine-tuned from fitleech/lm-2b for one epoch on fitleech/sn38-type-mix-2013-500m.

The training corpus uses a 2013 knowledge cutoff and a 20% / 50% / 30% Type1 / Type2 / Type3 mixture. Weights are the completed epoch-1 checkpoint.

Architecture

• 52 layers
• 12 attention heads
• 1536 model dimension
• 50,304-token GPT-2-compatible vocabulary
• BF16 checkpoint in safetensors format