Lion Optimized

AXL-Micro-Lion

Fast Lion model. 12.8M params. PPL 1.94. Context 2048 bytes. 10 min training.

13M
Parameters
1.94
Perplexity
10 min
Training
26 MB
GGUF
PropertyValue
ArchitectureMulti-Scale Transformer
d_model?
Attention Heads?
Layers per Scale?
Context Window2048 bytes
Downsample Factors[1, 2, 4]
Vocab Size258 (byte-level)
OptimizerLion
Trained on 5MB real HF Python code. 193 steps in 10 min on AMD Ryzen 5600G.
MetricValue
Final Loss0.5845
Perplexity1.94
Training Steps193
Training Time10 min

Usage

ollama create axl-micro-lion -f Modelfile
ollama run axl-micro-lion "def fibonacci():"
Good balance of speed and quality. Suitable for real-time code suggestions.
FileSizeFormat
F16 GGUF26 MBFull precision
Q4_K_M GGUF15 MB4-bit quantized
GGUF files work with Ollama and llama.cpp. Q4_K_M is about 3x smaller than F16.
← All AXL Models