---
license: mit
language:
  - en
tags:
  - liquid-neural-networks
  - hybrid
  - attention
  - long-context
  - text-generation
  - edge
pipeline_tag: text-generation
---

# M1-128M — native hybrid (attention + liquid core), trained from scratch

**Honest status first:** research backbone, not a chat model. M1-128M is the
M-Series hybrid of the [AwareLiquid](https://github.com/AwareLiquid/M1) research
project — window attention + a liquid core (selective decay, exp parameterization)
in every layer, 12 layers, d_model 832, trained from scratch on WikiText-103.
English text continuation. Language-modeling quality is **not** the selling
point (see below); the case is the liquid memory mechanisms.

## What it is

- **128.6M params** — 12 layers × 832d, GQA (13 heads / 1 KV head)
- **Hybrid**: MicrotubuleAttention + MultiScaleResonance liquid core per layer
- Trained from scratch on WikiText-103 (50K steps), fp32, stable — no NaN
- Served live at [awareliquid.ai/demo](https://awareliquid.ai/demo) (CPU)

## Measured results (multi-seed, reproducible — RESULTS.md is canonical)

| Result | Number |
|---|---|
| LM quality at convergence (20K steps, 3 seeds) | modern Transformer **78.86 ± 0.25** < MT-LNN **88.93 ± 0.33** < simple Transformer **94.14 ± 0.78** |
| Cross-window associative recall (fast-weight) | **0.56 vs 0.000** (attention/LoRA) |
| Cross-session snapshot/restore | **bit-exact** round-trip |
| O(1) inference state (attention-free O-series variant only) | 0.381 MB flat → 8,063× smaller than a KV-cache at 1M tokens |

The hybrid M-series is **not** O(1) (it keeps attention); the O(1) claim belongs
to the attention-free O-series ([O1-48M](https://huggingface.co/AwareLiquid/O1-48M)).

## Serve it

```bash
CKPT_PATH=hybrid_125m_serve.pt TOKENIZER=gpt2 python -m uvicorn serve.server:app
```

## Paper

- [English PDF](https://github.com/AwareLiquid/M1/blob/main/mt_lnn_v2_reliable_long_pretraining_arxiv.pdf)
- [中文 PDF](https://huggingface.co/AwareLiquid/O1-48M/resolve/main/mt_lnn_arxiv_zh.pdf)

## Related

- Code: [AwareLiquid/M1](https://github.com/AwareLiquid/M1) (MIT)
- Live demo: [awareliquid.ai/demo](https://awareliquid.ai/demo)
- Model family: [awareliquid.ai/#models](https://awareliquid.ai/#models)