--- license: cc-by-nc-sa-4.0 library_name: transformers pipeline_tag: text-generation language: - en --- # Opt.Gear-1M [![OptAI](https://img.shields.io/badge/%F0%9F%8F%A0%20OptAI-1a1a2e)](https://opt-ai.kr) [![Tech Report](https://img.shields.io/badge/%F0%9F%93%84%20Tech%20Report-5b4bcf)](https://huggingface.co/OptGear) > [!Note] > This repository contains model weights for **Opt.Gear-1M**, a Tiny Language Model (TLM) designed to run on **Micro-Controller Units (MCUs)**, together with artifacts for embedded deployment. > > Opt.Gear-1M is **not a general-purpose chatbot**. It is trained for a narrow embedded-control setting — mapping short natural-language commands to structured board-control intents — and is intended for deeply embedded systems such as sensors, controllers, small robots, and offline human-machine interfaces. > > For general-purpose on-device text generation, see [Opt.Gear-270M](https://huggingface.co/OptGear/Opt.Gear-270M) and [Opt.Gear-1B](https://huggingface.co/OptGear/Opt.Gear-1B). The goal of Opt.Gear-1M is not to shrink Opt.Gear-270M/1B further, but to actually run an auto-regressive language model on a microcontroller — an environment far more constrained than mobile NPUs or server GPUs. Deployed directly on the ARM Cortex-M7 core of an STM32H747I-DISCO board with 4-bit weights and FP32 activations (W4A32), Opt.Gear-1M generates at: > **20 tokens/s on a 400MHz STM32 Cortex-M7 — about 50ms per token.** Measured on the actual MCU with an embedded C runtime, not a host-side simulator. To our knowledge, Opt.Gear-1M is the **first generative language model to achieve 20 TPS with W4A32 quantization on the ARM Cortex-M7 CPU** of the STM32H747I-DISCO. ## Opt.Gear-1M Highlights - **MCU-first design**: every component — vocabulary, context length, KV cache, activation function, memory allocation — is scaled around Flash/SRAM limits and the absence of large matrix accelerators, not just the parameter count. - **Fixed-size decoding state**: the ConvKV-Gated Mixer keeps a convolution-kernel-sized state (kernel 4) instead of a sequence-length-dependent cache, keeping SRAM usage predictable during auto-regressive decoding. - **Simple, accelerator-free operators**: ReLU6GLU feed-forward and static memory allocation make every operation cheap to implement on a Cortex-M7 CPU. - **Compact tokenizer**: a custom 2,048-token byte-level BPE keeps the embedding tables and runtime memory footprint compatible with flash-limited deployment. - **Self-contained embedded C runtime**: runs without ONNX or X-CUBE-AI dependencies; footprint is measured from the final ELF, not theoretical weight size. For more details, please refer to our tech report and blog post. ## Model Overview - Type: Causal Language Model (hybrid attention + convolutional mixer) - Training Stage: task-focused board-control training on `stm32_cmd_synth` (no separate web-scale pre-training phase) - Architecture - Number of Parameters: ~1M - Hidden Dimension: 128 - Number of Layers: 5 - Hidden Layout: hybrid of GQA and ConvKV-Gated Mixer - Grouped-Query Attention: - Number of Attention Heads: 2 for Q and 1 for KV - Head Dimension: 64 - QK-Normalization: QK-LN - ConvKV-Gated Mixer: - Causal depthwise 1D convolution on key/value streams - Convolution Kernel Size: 4 (widened receptive field for the very small hidden dimension) - Feed-Forward Network: - Type: ReLU6GLU — ReLU6(x) = min(max(0, x), 6); bounded activation range, simple to implement without large accelerators - Intermediate Dimension: 272 - Rotary Position Embedding: global theta 1,000,000 - Tokenizer: custom byte-level BPE, vocabulary 2,048 - Word Embedding: untied (separate input embedding and LM head) - Context Length: 512 (deployment build uses a 160-token sequence length with a 40-token generation limit) ### Board-Control Post-Training Opt.Gear-1M follows a different training path from Opt.Gear-270M/1B. It is trained on **`stm32_cmd_synth`**, a synthetic command dataset of **50,000 prompt-response pairs**: each prompt is a short natural-language command for controlling the STM32 board, and each target response is a compact JSON-style control intent. The action space covers **LCD, LED, and camera control**. This design intentionally favors predictable command generation over broad open-domain language ability, matching the constraints of MCU deployment. ```text # Illustrative example — TODO: 실제 데이터 포맷으로 교체 User: turn on the red LED and show "hello" on the screen Model: {"led": {"color": "red", "state": "on"}, "lcd": {"text": "hello"}} ``` ## Measured Performance (STM32H747I-DISCO) All numbers are measured on the actual device. The firmware is instrumented with `HAL_GetTick`; throughput is measured from the Cortex-M7 execution path and reflects the model-compute portion of auto-regressive generation. Loadable section sizes are measured from the final STM32 ELF with `arm-none-eabi-size`. | Item | Measured or configured value | |---|---| | Board | STM32H747I-DISCO | | MCU / core | STM32H747XIH6 / ARM Cortex-M7 | | Configured CPU clock | 400MHz | | Runtime | Embedded C (no ONNX / X-CUBE-AI dependency) | | Quantized format | W4A32 (q4 weights, q16 embedding/LM head) | | Deployment sequence length | 160 tokens | | Maximum new tokens | 40 tokens | | Measured throughput | **20 tokens/s** | | Per-token latency | **50ms/token** | | `.text` / `.rodata` | 1,596,400 bytes | | `.data` | 532 bytes | | `.bss` | 444,404 bytes | | Total static RAM (incl. reserved heap/stack) | 449,032 bytes | A few notes on reading these numbers: - **20 tokens/s** means generating the 40-token maximum takes about 2 seconds of decode time — fast enough for short command responses and local status generation, with token-by-token output observable in real time. - **`.text`/`.rodata` (≈1.5MB)** is not the pure weight size: it includes the embedded runtime code, quantized weights, 16-bit embedding/LM head, lookup tables, scales, and constants. For MCU models, the final ELF — not the theoretical parameter count — is the meaningful footprint measure. - **Static RAM (≈449KB)** is dominated by statically allocated activation, cache (GQA KV cache + ConvKV fixed-size state), intermediate buffers, logit/sampling buffers, and the reserved heap/stack. ## Deployment The measured binary uses an embedded C runtime generated for STM32CubeIDE and runs without depending on an ONNX graph or the X-CUBE-AI generated network. The deployment build uses a 160-token sequence length and a 40-token generation limit, keeping the auto-regressive cache and workspace bounded on the MCU while preserving enough context for short instructions and embedded-control prompts. Potential applications at this speed and footprint: - Summarizing sensor data into short natural language - Local command response for small robots - Offline interfaces for industrial controllers - Structured status message generation - Simple Q&A on devices with limited network connectivity - Short, domain-specific embedded assistants ## Best Practices 1. **Stay in the trained domain**: The model is post-trained for short English board-control commands with JSON-style outputs. Out-of-domain prompts (open-ended questions, long-form generation, non-English input) will not produce reliable results. 2. **Respect the deployment limits**: The model configuration targets a 512-token maximum context, and the reference deployment build uses 160-token sequences with a 40-token generation cap. Longer sequences increase the SRAM-resident cache and workspace. 3. **Budget by ELF, not parameter count**: When adapting the runtime or retraining the model, verify Flash/SRAM budgets with `arm-none-eabi-size` on the final ELF — runtime code, lookup tables, and scales share the Flash with the weights. 4. **Predictability over coverage**: On embedded systems, predictable memory use and stable interactive latency matter more than long-context benchmark performance. The fixed-size ConvKV state exists precisely to keep decoding state independent of context length. ## Limitations Opt.Gear-1M is not a general-purpose chatbot. The ~1M parameter scale and 2,048-token vocabulary impose clear limits: it targets stable generative capability under extremely small flash, SRAM, and compute budgets, not broad benchmark coverage. The model prioritizes compact English generation, short-form command following, and predictable structured outputs. For general text generation on mobile and edge devices, use [Opt.Gear-270M](https://huggingface.co/OptAI/Opt.Gear-270M) or [Opt.Gear-1B](https://huggingface.co/OptAI/Opt.Gear-1B). ## Citation If you find our work helpful, feel free to give us a cite. ```bibtex @misc{optgear2026, title = {{Opt-Gear} Technical Report}, author = {{Opt.Gear Team}}, year = {2026}, url = {https://huggingface.co/OptGear} } ``` --- Correspondence: [contact@opt-ai.kr](mailto:contact@opt-ai.kr) · Hugging Face: [huggingface.co/OptAI](https://huggingface.co/OptGear)