--- license: apache-2.0 language: en tags: - tinyml - embedded - microcontroller - cortex-m - risc-v - xtensa - rp2040 - esp32 - nrf52840 - avr - fixed-point - integer-only - deterministic - reproducibility - quantization - gpt - character-level - custom-code --- # int-llm `model.mgw` — an integer-only GPT that runs bit-identically from x86 servers down to 32-bit microcontrollers, its arithmetic verified down to an 8-bit AVR This repo hosts the committed reference weights of **[int-llm](https://github.com/nmicic/int-llm)**: a tiny character-level GPT trained and sampled **entirely in Q16.48 fixed-point integer arithmetic** — no `float`, no `double`, no `libm` anywhere in the compute path — with **byte-reproducible** results across every tested platform and compiler. > **Note:** this is a custom C artifact in the project's own `.mgw` container — > **not** a Transformers/safetensors checkpoint. It cannot be loaded with > `AutoModel` or run on hosted inference; it is consumed by the C programs in > the GitHub repo below. It is also **not** the TinyLlama-derived inference > weights discussed in the int-llm write-up — this repository publishes only > the small character-level reference GPT used throughout the project. Background and motivation: [the int-llm blog post](https://huggingface.co/blog/nmicic/int-llm). ## The file | | | |---|---| | File | `model.mgw` — 115,576 bytes | | sha256 | `466cfe9dba7b888cdaa23dedf4b10351826795793448c8e95dcb0f7a61ed33eb` | | Model | character-level GPT: `N_EMBD=32, N_HEAD=4, N_LAYER=1`, 14,272 parameters (9 weight tensors) | | Weights | MGW v1 indexed container; 9 `int64_t` Q16.48 weight tensors plus tokenizer alphabet and RNG state | | Training | 5000 steps on the public *makemore* names dataset, integer-only (`./gpt_int --save model.mgw`) | | Math contract | determinism-gate golden hash `c0d933ea340452ec` — reproduced by every backend and every validated target below, down to an 8-bit AVR | | License | Apache-2.0 | The `.mgw` format (MGW v1) is a small indexed container: a 64-byte header (magic, version, endianness rejection tag), a 64-byte config block, a tensor index, the nine weight matrices as raw `int64_t` Q16.48 values, and two bookkeeping tensors — the tokenizer alphabet and the sampler's RNG state (11 stored tensors in all: 9 learned + 2 bookkeeping). There is nothing to dequantize: the model was **trained directly in Q16.48**, so floating-point weights never existed for this model. It can be loaded from a file (`--load model.mgw`) or **zero-copy from memory** (`mgpt_load_mem()`), which is how microcontrollers run it straight out of memory-mapped flash without ever copying the weights into RAM. ## Why host 115 KB of weights? First, because the size is the point: at 115 KB the *entire model* — not a distilled or re-quantized derivative — bakes into the flash of most mainstream MCUs alongside the code that runs it. Even a 256 KB-flash part holds both with room to spare (measured firmware sizes below), and on XIP-capable chips the weights are read in place, occupying zero RAM. Second, because the file itself is a claim. This *exact byte sequence* is independently reproduced by running the full 5000-step training on four different hosts — arm64 macOS (clang), x86-64 AMD (gcc), x86-64 Intel (gcc), and a 2014 Raspberry Pi 1 B+ (32-bit ARMv6, a CPU with no `__int128`) — and every validated inference run on the targets below reproduces the training host's 20 sampled names **byte-for-byte, PRNG stream included**. Most model files are "weights we happened to save"; this one is a fixed point (pun intended) you can re-derive from source. That makes it useful as an **oracle** for anyone working on quantization, numerical drift, regression testing, or deterministic inference: any deviation from it is an implementation bug, never rounding ambiguity — within the documented arithmetic and file-format contract, there is no floating-point tolerance to hide behind. ## Validated on real hardware The train-big/run-small loop closes on microcontrollers: train on a laptop, run inference-only from this weight file on a $5 board. Every target below passed both checks (determinism-grid golden hash + byte-identical 20-sample inference), except the 8-bit Mega 2560 — a board over a decade old — which runs the determinism gate only (8 KB RAM), computing the same 64-bit hash from 8-bit ALU instructions; raw provenance-stamped transcripts live in [`validation/cpu/`](https://github.com/nmicic/int-llm/tree/main/validation/cpu). | target | ISA | determinism | 20 samples | |---|---|---|---| | XIAO RP2040 (Cortex-M0+ @ 133 MHz) | Armv6-M | 10.9 s | 6.7 s | | Raspberry Pi Pico 2 (RP2350, ARM mode, Cortex-M33) | Armv8-M | 4.1 s | 3.1 s | | Raspberry Pi Pico 2 (RP2350, RISC-V mode, Hazard3) | rv32imac | 5.4 s | 3.8 s | | ESP32-C6 | rv32imac | 5.6 s | 2.0 s | | Heltec V3 (ESP32-S3, LX7 @ 240 MHz) | Xtensa | 3.9 s | 1.1 s | | LILYGO T-Beam (ESP32, LX6 @ 240 MHz) | Xtensa | 4.2 s | 2.6 s | | XIAO nRF52840 (Cortex-M4F @ 64 MHz) | Armv7E-M | 13.9 s | 3.4 s | | Arduino MKR Zero (SAMD21, Cortex-M0+ @ 48 MHz, 256 KB flash / 32 KB RAM) | Armv6-M | 37.2 s | 26.4 s | | Arduino Mega 2560 (ATmega2560, 8-bit AVR @ 16 MHz, 8 KB RAM) | AVR | 747.9 s | n/a (8 KB RAM) | | Raspberry Pi 1 B+ (2014, ARMv6, 32-bit Linux) | ARMv6 | 0.7 s | 0.2 s | | AMD Ryzen 7 7700 (Linux, gcc) | x86-64 | native + portable | byte-identical | | Intel Core i7-7700 (Linux, gcc) | x86-64 | native + portable | byte-identical | Same die, two ISAs: the Pico 2 reproduces the identical output in both its ARM and RISC-V boot modes. The Linux rows additionally rerun the **full training** and reproduce this repo's `model.mgw` byte-for-byte. **Footprint** (XIAO RP2040): the complete firmware is 187,932 B of flash — *including* the whole 115 KB weight file baked into rodata — and 15,268 B of static RAM (5.8% of the RP2040's 264 KB). Weights are read in place over XIP flash; RAM holds only KV cache, activations, and the USB stack. The 256 KB claim is measured, not extrapolated: on the Arduino MKR Zero's SAMD21 (256 KB flash / 32 KB RAM), code plus the whole weight file build to 133,472 B — 50.9% of flash — with 13,124 B of RAM. ## Run it ```bash git clone https://github.com/nmicic/int-llm cd int-llm make gpt_int ./gpt_int --load model.mgw # 20 names, byte-identical on every validated target ``` The identical file is committed in the GitHub repo; to fetch just the weights from here instead: ```bash hf download nmicic/int-llm model.mgw --local-dir . # for strict reproducibility, pin a revision once published: # hf download nmicic/int-llm model.mgw --revision --local-dir . ``` To retrain and verify the reproducibility claim yourself: ```bash make input # fetch the names dataset ./gpt_int --save model2.mgw # ~2 s on a desktop, ~10 min on a Pi 1 cmp model2.mgw model.mgw # exit 0 ``` The MCU harnesses (PlatformIO, one folder per board, flash + serial-capture scripts) are in [`validation/cpu/`](https://github.com/nmicic/int-llm/tree/main/validation/cpu). ## Intended use This artifact is intended for: validating integer-only or fixed-point implementations against an exact reference, regression testing, deterministic inference research, and embedded/TinyML experiments. It is **not** intended for production language generation, hosted inference, use with the Transformers library, or capability comparisons against modern LLMs. ## Limitations & provenance - **"Integer-only" refers to the model compute path**: training, inference, sampling, and serialization are all Q16.48 integer arithmetic. Build tooling, logging, and timing on the host are outside that claim. - This is a **toy by design**: a character-level *name generator*, not a general text LLM. `BLOCK_SIZE` is 8, so generated names are at most 8 characters. Its value is the methodology — exact integer arithmetic and cross-platform byte-reproducibility — not the language modeling. - Training data: [`names.txt` from Andrej Karpathy's makemore](https://github.com/karpathy/makemore) (MIT license), fetched by `make input`; sha256 `0a30b5557f192f32ab962680889aac5f6fda0f4cecf40a6d0b5694f58ea8cc4d`. - MGW v1 is a **host-native-endian** format with an endianness rejection tag in the header; this published file is **little-endian** (as are all validated targets). - The loader is a research-grade parser, not hardened against adversarial inputs — verify the sha256 above and load only trusted `.mgw` files. ## Links - Source, math library, and validation records: **https://github.com/nmicic/int-llm** - Write-up: **https://huggingface.co/blog/nmicic/int-llm** - The portable Q16.48 dual-backend math header matured in the sibling project **[astro-nav-int](https://github.com/nmicic/astro-nav-int)**