roneneldan/TinyStories
Viewer β’ Updated β’ 2.14M β’ 86.1k β’ 1.13k
qu_ssm-60Moe (QU-SSM-60M-MoE) is the mid-tier foundation model of the qu_ssm family designed and invented by Prannessh K.V.A. (Sole Architect & Inventor). It combines continuous Lie-group unitary recurrence on SO(2) β
U(1) with 4 SwiGLU Mixture-of-Experts (MoE) and Top-2 routing (44.64M active parameters per token).
qu_ssm is a linear-time continuous sequence engine that replaces the monotonic dissipative decay (e^(-Ξ±Β·t) β 0) of classical state space models (Mamba-1/2) with non-dissipative SO(2) unitary phase rotations (βR(ΞΈ)ββ β‘ 1.00000), delivering strictly constant O(1) inference memory and sub-millisecond step latency.
| Specification | Value |
|---|---|
| Model Name | qu_ssm-60Moe (QU-SSM-60M-MoE) |
| Sole Architect & Inventor | Prannessh K.V.A. |
| Total Parameters | 64.30M |
| Active Parameters / Token | 44.64M (Top-2 Sparse MoE) |
| Hidden Dimension (D) | 384 |
| Layers | 6 |
| SSM State Dimension | 8 |
| Expert Count | 4 SwiGLU Experts |
| Vocabulary | 50,257 (GPT-2 BPE) |
| Inference State RAM | 0.19 MB (Constant O(1)) |
10.5281/zenodo.22217820