Documentation generation. 9.9M params. PPL 1.15. Context 2048 bytes.
| Property | Value |
|---|---|
| Architecture | Multi-Scale Transformer |
| d_model | ? |
| Attention Heads | ? |
| Layers per Scale | ? |
| Context Window | 2048 bytes |
| Downsample Factors | [1, 2, 4] |
| Vocab Size | 258 (byte-level) |
| Optimizer | Lion |
| Metric | Value |
|---|---|
| Final Loss | 0.1336 |
| Perplexity | 1.15 |
| Training Steps | 205 |
| Training Time | 10 min |
ollama create axl-docs-lion -f Modelfile
ollama run axl-docs-lion "def fibonacci():"| File | Size | Format |
|---|---|---|
| F16 GGUF | 20 MB | Full precision |
| Q4_K_M GGUF | 7 MB | 4-bit quantized |