Standard Vanilla Transformer Model
Pure standard autoregressive language model trained from scratch without architecture efficiency tricks.
Model Configuration
- Layers: 16
- Embedding Dim: 1024
- Attention Heads: 16
- Context Length: 8192
- Max Tokens per Sample: 256
