---
language:
- en
license: mit
tags:
- awareliquid
- mt-lnn
- liquid-neural-network
- research
pipeline_tag: text-generation
---

# M2-2B

**1.93B-parameter AwareLiquid M2 preset** (2080 width × 34 layers × 16 heads),
the M2 experimental line's 2b configuration trained via `m2_training`.
Byte-level tokenizer (UTF-8 bytes, vocab 256).

**Status at upload:** research milestone — 200,000-step FINAL checkpoint (val PPL 2.4106). `model.safetensors` holds the 200K weights; the raw resumable PyTorch checkpoint is `t2b_200k.pt`.
**Training status (2026-09-28):** the baseline run COMPLETED at 200,000 steps,
final val PPL 2.4106. Full milestone table below; the 200K weights live in the
M2 repo (docs/RESULTS_2B_30K.md) and on the training server
(/root/M2/runs/t2b_30k.pt, 3.86GB bf16). **The 200K raw checkpoint
is now in this repo as `t2b_200k.pt`** (PyTorch bf16, weight-tied; the matching 200K-weight `model.safetensors` export is in this repo since 2026-09-29).

## Measured

| Milestone | val PPL (byte-level, WikiText-103) |
|---|---|
| 10K steps (feasibility) | 4.53 |
| 30K steps | 2.73 |
| 60K steps | 2.556 |
| 90K steps | 2.482 |
| 150K steps | 2.413 |
| **200K steps (final, 09-28)** | **2.4106** |

Convergence: 30K→60K −6.4%, 60K→90K −2.9%, 90K→150K −2.8%, 150K→200K
−0.1% (saturated).

Context-flat: PPL 2.72 / 2.72 / 2.68 at seq 128 / 256 / 512 (no length degradation).
Training: bf16 model, AdamW8bit, lr 1e-4, seq 128,
batch 8 × grad_accum 4, on a single A100-PCIE-40GB.

## Boundaries (honest)

- **Byte-level, not a subword model** — PPL is on UTF-8 byte tokens; NOT
  comparable to gpt2-vocab models. Serve with byte tokenization.
- **Not an instruct/chat model** — raw next-token pretraining only.
- **Research checkpoint** — bf16 safetensors (weight-tied embedding; the tied
  `lm_head.weight` is omitted, standard HF re-tie on load). No optimizer state:
  not exactly resumable.
- The M2 line's verdict so far is honest-null on reasoning mechanisms; this
  checkpoint is the language-line scaling artifact, not a capability claim.

## SFT checkpoints (post-training experiments, 2026-10)

| file | recipe | val PPL | JevBench public (official harness) |
|---|---|---|---|
| `t2b_sft_v1.pt` | alpaca-short 5.4k pairs (<=128B), 3 epochs, lr 1e-4 | 3.95 | - |
| `t2b_sft_v2.pt` | alpaca-short512 19.6k pairs (<=512B), 1+5k steps, lr 2e-5, rope_scale 4 | **3.20** | raw 31.3% (48 easy + 72 original decisions) |

Boundaries: instruction-SFT on **short** pairs; held-out generation went from
0/8 (base) to 4/8 format-correct, but decision-task accuracy remains the open
problem (best family configuration: 2B frozen core + decision head, 39.6%
easy-tier raw / ECE 0.20 / 0.168 s - see github.com/AwareLiquid/O1-Flash).
Serving interface for all of the above: byte-level, `max_seq_len` per config.

## Config

`config.json` carries the full MTLNNConfig (d_model 2080, 34 layers, 16 heads,
gwtb_n_heads=1, vocab 256, max_seq_len 129).