File size: 2,469 Bytes
821ca21 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
license: mit
library_name: custom
tags:
- esp32
- tinyml
- microcontroller
- code-generation
---
# esp32-tiny-coding-model
A 3M-parameter model that runs inside an ESP32-S3. You type a sentence in English, it writes a small program for
the board's 128x64 screen, and the same chip runs it. No internet, no server.

| | |
|---|---|
| Model | 2.9M core + 0.8M lookup table (Per-Layer Embeddings), 2.1 MB at 4-bit |
| Speed | about 9 tokens/s on the chip, "draw a house" in 5.6 s |
| Held-out requests | 98% draw the same picture as the reference |
It writes in a small Python-like language (`clear`, `text`, `pixel`, `line`, `rect`, `fill`, `circle`, `wait`,
variables, `for`, `while`, `if`), and an interpreter on the chip runs it with limits on steps and waiting, so
whatever the model writes, the chip stays in charge. It understands requests in the style it was trained on; a
sentence far from them can draw the wrong thing.
Code, training data generator, interpreter and firmware: [github.com/moudrkat/esp32-ai](https://github.com/moudrkat/esp32-ai),
a fork of [slvDev/esp32-ai](https://github.com/slvDev/esp32-ai), whose runtime and firmware this runs on.
## Files
| file | what |
|---|---|
| `esp32-oled-full.bin` | everything for the board in one image: bootloader, firmware and model |
| `model.bin` | the model alone, as the firmware reads it from flash (offset `0x110000`) |
| `tokenizer.json` | its tokenizer (byte-level BPE, 1,079 tokens) |
## Run it
Hardware: an ESP32-S3 board with 16 MB flash and 8 MB PSRAM (octal), and a 128x64 SSD1306 I2C OLED on
**SDA = GPIO 18, SCL = GPIO 8**.
```bash
pip install esptool
esptool --chip esp32s3 --port /dev/ttyACM0 write-flash 0x0 esp32-oled-full.bin
```
Then open the serial port at 115200 baud, type a request and press Enter, for example `draw a house`,
`countdown from 5 then happy birthday`, `bounce a ball twice quickly`. The repo has a small terminal for it:
`scripts/oled_talk.py`.
Other wiring or another screen: build the firmware from the repo (`scripts/deploy.sh oled`), which also checks the
C runtime against the PyTorch model before it flashes.
## Training data
About 300,000 request -> program pairs generated by rules (no model involved), each program run by the reference
interpreter before it is kept. Reproducible with `research/oled/gen.py` in the repo.
MIT © Kateřina Fajmanová
|