--- license: cc-by-nc-sa-4.0 library_name: transformers pipeline_tag: text-generation language: - en - ko base_model: - OptAI/Opt.Gear-270M --- # Opt.Gear-270M [![OptAI](https://img.shields.io/badge/%F0%9F%8F%A0%20OptAI-1a1a2e)](https://opt-ai.kr) [![Tech Report](https://img.shields.io/badge/%F0%9F%93%84%20Tech%20Report-5b4bcf)](https://huggingface.co/OptGear) > [!Note] > This repository contains model weights and configuration files for the post-trained (instruction-tuned) model in the Hugging Face Transformers format. > > These artifacts are compatible with Hugging Face Transformers, llama.cpp, ExecuTorch, and vLLM. Executable binaries optimized for NPU inference (Qualcomm Hexagon NPU, Apple ANE) are also provided. > > Opt.Gear-270M targets lower-latency deployment on more constrained mobile and edge devices. For a stronger quality-efficiency trade-off at the same context length, see [Opt.Gear-1B](https://huggingface.co/OptGear/Opt.Gear-1B). Opt.Gear is the first generation of OptAI Foundation Models, designed for efficient on-device deployment, real-time inference, and strong task capability. Rather than simply shrinking a server-scale LLM, Gear jointly optimizes downstream quality, device-side latency, memory efficiency, long-context capability, and practical deployability under hardware constraints such as limited SRAM, memory bandwidth, and power. Opt.Gear-270M shares the same hybrid design and 64K context support as Opt.Gear-1B, while targeting lower-latency deployment on smaller mobile and edge environments. ## Opt.Gear Highlights - **Data-efficient training**: Trained on a curated **0.5T-token** subset selected from a 2T-token candidate corpus, **without knowledge distillation** from a teacher model. Against the distilled Gemma3-270M (6T tokens), Opt.Gear-270M achieves competitive or superior scores with 12× fewer training tokens. - **Hybrid architecture with ConvKV-Gated Mixer**: A small set of global GQA layers handles long-range routing, while **ConvKV-Gated Mixer** blocks replace many local attention layers with softmax-free, matrix-multiplication-free local mixing. The persistent local state scales with the convolution kernel (L_conv = 3) instead of the sliding window (W = 512), substantially reducing live decoding state and memory bandwidth. - **NPU-friendly by design**: Dynamic QKV matrix multiplications and softmax normalization are replaced with static linear, convolution, and element-wise operations, making the model easy to map onto CPUs, GPUs, and NPUs. - **Korean-English bilingual**: Built on the KORMo tokenizer (125,184 vocab) with Korean-English pretraining, leading same-scale baselines on Korean benchmarks (KMMLU, KoBEST, HAERAE). - **64K context on device**: Hybrid attention layout with separate global/local RoPE frequencies supports long-context modeling up to 65,536 tokens while keeping the local cache footprint small. For more details, please refer to our tech report and blog post. ## Model Overview - Type: Causal Language Model (hybrid attention + convolutional mixer) - Training Stage: Pre-training (0.5T tokens) → Long-context extension (4K → 32K → 64K) → Two-stage SFT (general instruction → reasoning-oriented) - Architecture - Number of Parameters: 270M - Hidden Dimension: 640 - Number of Layers: 18 - Hidden Layout: hybrid of Global GQA, Local (sliding-window) Attention, and ConvKV-Gated Mixer - Grouped-Query Attention: - Number of Attention Heads: 4 for Q and 1 for KV - Head Dimension: 256 - Sliding-Window Size (local attention): 512 - QK-Normalization: QK-LN - ConvKV-Gated Mixer: - Causal depthwise 1D convolution on key/value streams - Convolution Kernel Size: 3 (fixed-size persistent state, independent of context length) - Feed-Forward Network: - Type: GeGLU (gated dense MLP) - Intermediate Dimension: 2,048 - Rotary Position Embedding: global theta 1,000,000 / local theta 10,000 - Tokenizer: KORMo (byte-level BPE), vocabulary 125,184 (shared with Opt.Gear-1B) - Word Embedding: untied (separate input embedding and LM head) - Context Length: 65,536 natively > [!Important] > Base and Instruction models share the same tokenizer, but use different end-of-generation tokens: the **Base model emits ``** while the **Instruction model emits ``**. When fine-tuning, make sure training examples terminate with the same convention used by the target runtime — malformed termination wastes decoding steps and increases latency on device. ## Benchmark Results All results are obtained using the [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) and may differ from scores reported elsewhere. Perplexity-based evaluation is adopted for HellaSwag, PIQA, WinoGrande, MMLU, GPQA, ARC, KMMLU, and KoBEST.
Opt.Gear-270M SmolLM2-135M Gemma3-270M LFM2.5-350M Qwen3-0.6B
# Trained Tokens 0.5T 2T 6T 28T 36T
Distilled ✗ ✗ ✓ ✓ ✓
English
MMLU 25.9 25.3 26.5 41.0 47.3
Korean
KMMLU 30.0 29.1 28.0 31.4 35.0
KoBEST 51.9 48.7 50.0 51.1 54.4
CLIcK 27.2 22.5 27.3 24.0 36.7
HAERAE 21.5 18.9 20.4 24.8 37.0

* All results are obtained using LM Evaluation Harness and may differ from other reported scores.
* While baselines are trained on 2T–36T tokens and frequently leverage knowledge distillation, Opt.Gear-270M is trained on only 0.5T tokens without any distillation. Against the similarly-sized distilled Gemma3-270M, Opt.Gear-270M leads on reasoning (PIQA 67.3 vs 66.9, ARC-Easy 59.1 vs 56.1) and Korean benchmarks (KMMLU 30.0 vs 28.0, KoBEST 51.9 vs 50.0, HAERAE 21.5 vs 20.4).

### On-Device Inference Opt.Gear-270M shares the same NPU-friendly hybrid architecture as Opt.Gear-1B, which achieves up to ×4.9 faster prefill and decoding on NPUs compared to models of similar scale (measured with Qualcomm AI Runtime and CoreML — see the [Opt.Gear-1B card](https://huggingface.co/OptGear/Opt.Gear-1B) and the tech report for detailed device benchmarks). ## Quickstart ### Hugging Face Transformers ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "OptGear/Opt.Gear-270M" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto") messages = [ {"role": "user", "content": "온디바이스 AI가 왜 중요한지 설명해줘."}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors="pt" ).to(model.device) outputs = model.generate(inputs, max_new_tokens=512) print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)) ``` ### On-Device Deployment Executable binaries optimized for NPU inference are provided for: - **Qualcomm Hexagon NPU** — via Qualcomm AI Engine Direct (QAIRT); recommended for Snapdragon devices - **Apple ANE** — via CoreML Runtime - **ExecuTorch** — for mobile/edge PyTorch deployment ## Best Practices 1. **Termination tokens**: The Instruction model terminates generation with ``, while the Base model uses ``. When fine-tuning either variant, ensure training examples terminate with the matching token. 2. **Runtime selection**: Snapdragon NPU → QAIRT; Apple devices → CoreML; CPU/GPU → llama.cpp. Avoid the llama.cpp NPU backend for models with sliding-window attention. 3. **Languages**: The model is trained primarily on English (~92%) and Korean (~6%) with mathematical text (~2%). It is best suited for Korean-English bilingual understanding, summarization, rewriting, and instruction following on latency-sensitive devices. 4. **Not intended for code generation**: The pre-training corpus deliberately excludes code data — models at the 270M–1B scale are not typically used for code generation tasks. 5. **Long context**: The model natively supports 65,536 tokens with a fixed-size local cache, making long-context decoding memory-efficient even at this scale. ## Limitations The constrained 0.5T-token training budget limits capabilities in complex reasoning and mathematical tasks that typically benefit from larger training corpora. At the 270M scale, knowledge-heavy benchmarks (e.g., MMLU) remain challenging; the model is best used for well-scoped generation and understanding tasks rather than open-domain knowledge QA. See the tech report for a detailed discussion. ## Citation If you find our work helpful, feel free to give us a cite. ```bibtex @misc{optgear2026, title = {{Opt-Gear} Technical Report}, author = {{Opt.Gear Team}}, year = {2026}, url = {https://huggingface.co/OptGear} } ``` --- Correspondence: [contact@opt-ai.kr](mailto:contact@opt-ai.kr) · Hugging Face: [huggingface.co/OptAI](https://huggingface.co/OptGear)