README.md

metadata

language:
  - en
license: mit
tags:
  - awareliquid
  - mt-lnn
  - liquid-neural-network
  - research
pipeline_tag: text-generation

M2-2B

1.93B-parameter AwareLiquid M2 preset (2080 width × 34 layers × 16 heads), the M2 experimental line's 2b configuration trained via m2_training. Byte-level tokenizer (UTF-8 bytes, vocab 256).

Status at upload: research milestone — 60,000-step checkpoint.

Measured

Milestone	val PPL (byte-level, WikiText-103)
10K steps (feasibility)	4.53
30K steps	2.73
60K steps (this checkpoint)	2.556

Context-flat: PPL 2.72 / 2.72 / 2.68 at seq 128 / 256 / 512 (no length degradation). Training: bf16 model, AdamW8bit, lr 1e-4, seq 128, batch 8 × grad_accum 4, on a single A100-PCIE-40GB.

Boundaries (honest)

• Byte-level, not a subword model — PPL is on UTF-8 byte tokens; NOT comparable to gpt2-vocab models. Serve with byte tokenization.
• Not an instruct/chat model — raw next-token pretraining only.
• Research checkpoint — bf16 safetensors (weight-tied embedding; the tied lm_head.weight is omitted, standard HF re-tie on load). No optimizer state: not exactly resumable.
• The M2 line's verdict so far is honest-null on reasoning mechanisms; this checkpoint is the language-line scaling artifact, not a capability claim.

Config

config.json carries the full MTLNNConfig (d_model 2080, 34 layers, 16 heads, gwtb_n_heads=1, vocab 256, max_seq_len 129).