Unt1l1f1nd's picture
esp32 tiny coding model: firmware image, model, tokenizer
821ca21 verified
|
Raw History Blame Contribute Delete
2.47 kB
metadata
license: mit
library_name: custom
tags:
  - esp32
  - tinyml
  - microcontroller
  - code-generation

esp32-tiny-coding-model

A 3M-parameter model that runs inside an ESP32-S3. You type a sentence in English, it writes a small program for the board's 128x64 screen, and the same chip runs it. No internet, no server.

The board writes a program for "draw a house", then runs it

Model 2.9M core + 0.8M lookup table (Per-Layer Embeddings), 2.1 MB at 4-bit
Speed about 9 tokens/s on the chip, "draw a house" in 5.6 s
Held-out requests 98% draw the same picture as the reference

It writes in a small Python-like language (clear, text, pixel, line, rect, fill, circle, wait, variables, for, while, if), and an interpreter on the chip runs it with limits on steps and waiting, so whatever the model writes, the chip stays in charge. It understands requests in the style it was trained on; a sentence far from them can draw the wrong thing.

Code, training data generator, interpreter and firmware: github.com/moudrkat/esp32-ai, a fork of slvDev/esp32-ai, whose runtime and firmware this runs on.

Files

file what
esp32-oled-full.bin everything for the board in one image: bootloader, firmware and model
model.bin the model alone, as the firmware reads it from flash (offset 0x110000)
tokenizer.json its tokenizer (byte-level BPE, 1,079 tokens)

Run it

Hardware: an ESP32-S3 board with 16 MB flash and 8 MB PSRAM (octal), and a 128x64 SSD1306 I2C OLED on SDA = GPIO 18, SCL = GPIO 8.

pip install esptool
esptool --chip esp32s3 --port /dev/ttyACM0 write-flash 0x0 esp32-oled-full.bin

Then open the serial port at 115200 baud, type a request and press Enter, for example draw a house, countdown from 5 then happy birthday, bounce a ball twice quickly. The repo has a small terminal for it: scripts/oled_talk.py.

Other wiring or another screen: build the firmware from the repo (scripts/deploy.sh oled), which also checks the C runtime against the PyTorch model before it flashes.

Training data

About 300,000 request -> program pairs generated by rules (no model involved), each program run by the reference interpreter before it is kept. Reproducible with research/oled/gen.py in the repo.

MIT © Kateřina Fajmanová