| tags: [chat-model-competition, bittensor, llama, from-scratch] | |
| license: apache-2.0 | |
| # test-llama-tiny | |
| Chat model trained with the Chat Model Competition Bittensor subnet. | |
| **Architecture:** LlamaForCausalLM (vLLM compatible) | |
| **Pipeline:** Pretrain (100 steps) → SFT (50 steps) → RL/GRPO (20 steps) | |
| | Property | Value | | |
| |----------|-------| | |
| | Parameters | 13.3M | | |
| | Architecture | 2L / 4H / 2KV / 128D | | |
| | Intermediate | 384 | | |
| | Seq Length | 128 | | |