| --- |
| license: mit |
| language: |
| - en |
| - zh |
| tags: |
| - liquid-neural-networks |
| - microtubules |
| - long-context |
| - text-generation |
| - o1 |
| - edge |
| pipeline_tag: text-generation |
| --- |
| |
| # MT-LNN — Microtubule-Inspired Liquid Neural Network (O1-48M) |
|
|
| **Honest status first:** research artifact, not a general assistant. This is the |
| **O1-48M** attention-free edge model trained from scratch — the O-Series line of |
| the [AwareLiquid](https://github.com/AwareLiquid/M1) research project. It does |
| **not** aim to match frontier models on dialogue or benchmarks; its claims are |
| about **memory form-factor and efficiency**, and every number below is measured |
| and reproducible. |
|
|
| ## What it is |
|
|
| - **48M params**, attention-free, from scratch — no base model |
| - **O(1) constant inference state**: 0.381 MB flat state from 512 to 1,048,576 |
| tokens (vs a KV cache growing to 3,072 MB — 8,063× smaller) |
| - Continuous-time **liquid core** with learnable time constants |
| (13 protofilaments × 5 time-scales), true parallel-scan recurrence |
| - CPU-friendly: the live demo at [awareliquid.ai/demo](https://awareliquid.ai/demo) |
| runs on CPU |
|
|
| ## Measured results (multi-seed, reproducible) |
|
|
| | Result | Number | |
| |---|---| |
| | Cross-window associative recall (fast-weight memory) | **0.56** (3 seeds) vs **0.000** for attention/LoRA | |
| | O(1) inference state @1M tokens | **0.381 MB** constant, 8,063× smaller than KV | |
| | Robustness to irregular sampling (NASA battery SoH) | **+7.7%** degradation @80% dropped samples (LSTM +31.1%, GRU +32.8%) | |
| | Language-modeling quality | **Not** an advantage: 125M-scale WikiText-103 PPL 88.93 ± 0.33 vs Transformer 78.86 ± 0.25 | |
|
|
| The architecture's case rests on memory form-factor and efficiency, **not** on |
| quality per parameter. Full honest analysis: [RESULTS.md](https://github.com/AwareLiquid/M1/blob/main/RESULTS.md). |
|
|
| ## Weights |
|
|
| This checkpoint is published on GitHub Releases: |
| [o1-48m-v1](https://github.com/AwareLiquid/M1/releases/tag/o1-48m-v1). |
|
|
| Serve it: |
|
|
| ```bash |
| CKPT_PATH=o1_48m_serve.pt TOKENIZER=gpt2 python -m uvicorn serve.server:app |
| ``` |
|
|
| ## Paper |
|
|
| - [English PDF](https://github.com/AwareLiquid/M1/blob/main/mt_lnn_v2_reliable_long_pretraining_arxiv.pdf) |
| - [中文 PDF](https://github.com/AwareLiquid/M1/blob/main/mt_lnn_v2_reliable_long_pretraining_arxiv_zh.pdf) |
|
|
| ## Related |
|
|
| - Code: [AwareLiquid/M1](https://github.com/AwareLiquid/M1) (MIT) |
| - Live demo: [awareliquid.ai/demo](https://awareliquid.ai/demo) |
| - Model family: [awareliquid.ai](https://awareliquid.ai/#models) |
|
|
