Lumia-50M

Custom reasoning model with Gated Reasoning Memory (GRM) + Stateful Inference Pipeline (SIP).

Params: 46.35M
Architecture: 12-layer Transformer, GQA (10Q/2KV), SwiGLU, RoPE, RMSNorm + LayerScale
Memory: 6 RMC cells (K=16 each), 3 RRB blocks with convergence detection
Inference: SIP with prefill/extend, exact output equivalence
Training: Pure PyTorch, CPU/T4/TPU support, fp16 AMP, activation checkpointing
Tokenizer: BPE (12288 vocab, 12017 merges)

Downloads last month: -; Downloads are not tracked for this model. How to track