File size: 2,511 Bytes
5f5e486 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
license: mit
language:
- en
- zh
tags:
- liquid-neural-networks
- microtubules
- long-context
- text-generation
- o1
- edge
pipeline_tag: text-generation
---
# MT-LNN — Microtubule-Inspired Liquid Neural Network (O1-48M)
**Honest status first:** research artifact, not a general assistant. This is the
**O1-48M** attention-free edge model trained from scratch — the O-Series line of
the [AwareLiquid](https://github.com/AwareLiquid/M1) research project. It does
**not** aim to match frontier models on dialogue or benchmarks; its claims are
about **memory form-factor and efficiency**, and every number below is measured
and reproducible.
## What it is
- **48M params**, attention-free, from scratch — no base model
- **O(1) constant inference state**: 0.381 MB flat state from 512 to 1,048,576
tokens (vs a KV cache growing to 3,072 MB — 8,063× smaller)
- Continuous-time **liquid core** with learnable time constants
(13 protofilaments × 5 time-scales), true parallel-scan recurrence
- CPU-friendly: the live demo at [awareliquid.ai/demo](https://awareliquid.ai/demo)
runs on CPU
## Measured results (multi-seed, reproducible)
| Result | Number |
|---|---|
| Cross-window associative recall (fast-weight memory) | **0.56** (3 seeds) vs **0.000** for attention/LoRA |
| O(1) inference state @1M tokens | **0.381 MB** constant, 8,063× smaller than KV |
| Robustness to irregular sampling (NASA battery SoH) | **+7.7%** degradation @80% dropped samples (LSTM +31.1%, GRU +32.8%) |
| Language-modeling quality | **Not** an advantage: 125M-scale WikiText-103 PPL 88.93 ± 0.33 vs Transformer 78.86 ± 0.25 |
The architecture's case rests on memory form-factor and efficiency, **not** on
quality per parameter. Full honest analysis: [RESULTS.md](https://github.com/AwareLiquid/M1/blob/main/RESULTS.md).
## Weights
This checkpoint is published on GitHub Releases:
[o1-48m-v1](https://github.com/AwareLiquid/M1/releases/tag/o1-48m-v1).
Serve it:
```bash
CKPT_PATH=o1_48m_serve.pt TOKENIZER=gpt2 python -m uvicorn serve.server:app
```
## Paper
- [English PDF](https://github.com/AwareLiquid/M1/blob/main/mt_lnn_v2_reliable_long_pretraining_arxiv.pdf)
- [中文 PDF](https://github.com/AwareLiquid/M1/blob/main/mt_lnn_v2_reliable_long_pretraining_arxiv_zh.pdf)
## Related
- Code: [AwareLiquid/M1](https://github.com/AwareLiquid/M1) (MIT)
- Live demo: [awareliquid.ai/demo](https://awareliquid.ai/demo)
- Model family: [awareliquid.ai](https://awareliquid.ai/#models)
|