README.md

---
license: cc-by-nc-sa-4.0
library_name: transformers
pipeline_tag: text-generation
language:
- en
- ko
base_model:
- OptAI/Opt.Gear-270M
---

# Opt.Gear-270M

<img width="1000px" src="./OptAIxHuggingFace02.png">

[![OptAI](https://img.shields.io/badge/%F0%9F%8F%A0%20OptAI-1a1a2e)](https://opt-ai.kr)
[![Tech Report](https://img.shields.io/badge/%F0%9F%93%84%20Tech%20Report-5b4bcf)](https://huggingface.co/OptGear)

> [!Note]
> This repository contains model weights and configuration files for the post-trained (instruction-tuned) model in the Hugging Face Transformers format.
>
> These artifacts are compatible with Hugging Face Transformers, llama.cpp, ExecuTorch, and vLLM. Executable binaries optimized for NPU inference (Qualcomm Hexagon NPU, Apple ANE) are also provided.
>
> Opt.Gear-270M targets lower-latency deployment on more constrained mobile and edge devices. For a stronger quality-efficiency trade-off at the same context length, see [Opt.Gear-1B](https://huggingface.co/OptGear/Opt.Gear-1B). <!-- TODO: 링크 확인 -->

Opt.Gear is the first generation of OptAI Foundation Models, designed for efficient on-device deployment, real-time inference, and strong task capability. Rather than simply shrinking a server-scale LLM, Gear jointly optimizes downstream quality, device-side latency, memory efficiency, long-context capability, and practical deployability under hardware constraints such as limited SRAM, memory bandwidth, and power.

Opt.Gear-270M shares the same hybrid design and 64K context support as Opt.Gear-1B, while targeting lower-latency deployment on smaller mobile and edge environments.

## Opt.Gear Highlights

- **Data-efficient training**: Trained on a curated **0.5T-token** subset selected from a 2T-token candidate corpus, **without knowledge distillation** from a teacher model. Against the distilled Gemma3-270M (6T tokens), Opt.Gear-270M achieves competitive or superior scores with 12× fewer training tokens.

- **Hybrid architecture with ConvKV-Gated Mixer**: A small set of global GQA layers handles long-range routing, while **ConvKV-Gated Mixer** blocks replace many local attention layers with softmax-free, matrix-multiplication-free local mixing. The persistent local state scales with the convolution kernel (L_conv = 3) instead of the sliding window (W = 512), substantially reducing live decoding state and memory bandwidth.

- **NPU-friendly by design**: Dynamic QKV matrix multiplications and softmax normalization are replaced with static linear, convolution, and element-wise operations, making the model easy to map onto CPUs, GPUs, and NPUs.

- **Korean-English bilingual**: Built on the KORMo tokenizer (125,184 vocab) with Korean-English pretraining, leading same-scale baselines on Korean benchmarks (KMMLU, KoBEST, HAERAE).

- **64K context on device**: Hybrid attention layout with separate global/local RoPE frequencies supports long-context modeling up to 65,536 tokens while keeping the local cache footprint small.

For more details, please refer to our tech report and blog post. <!-- TODO: 링크 연결 -->

## Model Overview

- Type: Causal Language Model (hybrid attention + convolutional mixer)
- Training Stage: Pre-training (0.5T tokens) → Long-context extension (4K → 32K → 64K) → Two-stage SFT (general instruction → reasoning-oriented)
- Architecture
    - Number of Parameters: 270M
    - Hidden Dimension: 640
    - Number of Layers: 18
    - Hidden Layout: hybrid of Global GQA, Local (sliding-window) Attention, and ConvKV-Gated Mixer
    - Grouped-Query Attention:
        - Number of Attention Heads: 4 for Q and 1 for KV
        - Head Dimension: 256
        - Sliding-Window Size (local attention): 512
        - QK-Normalization: QK-LN
    - ConvKV-Gated Mixer:
        - Causal depthwise 1D convolution on key/value streams
        - Convolution Kernel Size: 3 (fixed-size persistent state, independent of context length)
    - Feed-Forward Network:
        - Type: GeGLU (gated dense MLP)
        - Intermediate Dimension: 2,048
    - Rotary Position Embedding: global theta 1,000,000 / local theta 10,000
    - Tokenizer: KORMo (byte-level BPE), vocabulary 125,184 (shared with Opt.Gear-1B)
    - Word Embedding: untied (separate input embedding and LM head)
- Context Length: 65,536 natively

> [!Important]
> Base and Instruction models share the same tokenizer, but use different end-of-generation tokens: the **Base model emits `<EOS>`** while the **Instruction model emits `<EOT>`**. When fine-tuning, make sure training examples terminate with the same convention used by the target runtime — malformed termination wastes decoding steps and increases latency on device.

## Benchmark Results

All results are obtained using the [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) and may differ from scores reported elsewhere. Perplexity-based evaluation is adopted for HellaSwag, PIQA, WinoGrande, MMLU, GPQA, ARC, KMMLU, and KoBEST.

<div style="max-width:1000px;margin:0 auto;padding:16px 0">
<table style="border-collapse:collapse;font-size:15px">
<thead><tr>
<th style="padding:10px 7px;text-align:left;font-weight:600;border-bottom:2px solid #5b4bcf;color:#5b4bcf"></th>
<th style="padding:10px 7px;text-align:center;font-weight:600;border-bottom:2px solid #5b4bcf;color:#5b4bcf">Opt.Gear-270M</th>
<th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">SmolLM2-135M</th>
<th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">Gemma3-270M</th>
<th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">LFM2.5-350M</th>
<th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">Qwen3-0.6B</th>
</tr></thead>
<tbody>
<tr>
<td style="padding:7px;border-bottom:1px solid rgba(128,128,128,0.15)"># Trained Tokens</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">0.5T</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">2T</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">6T</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">28T</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">36T</td>
</tr>
<tr>
<td style="padding:7px;border-bottom:1px solid rgba(128,128,128,0.15)">Distilled</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">✗</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">✗</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">✓</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">✓</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">✓</td>
</tr>
<tr><td colspan="6" style="padding:8px 12px;font-weight:600;color:#5b4bcf;border-bottom:1px solid rgba(91,75,207,0.2);background:rgba(91,75,207,0.08)">English</td></tr>
<tr>
<td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">MMLU</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">25.9</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">25.3</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">26.5</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">41.0</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">47.3</td>
</tr>
<tr><td colspan="6" style="padding:8px 12px;font-weight:600;color:#5b4bcf;border-bottom:1px solid rgba(91,75,207,0.2);background:rgba(91,75,207,0.08)">Korean</td></tr>
<tr>
<td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">KMMLU</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">30.0</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">29.1</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">28.0</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">31.4</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">35.0</td>
</tr>
<tr>
<td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">KoBEST</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">51.9</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">48.7</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">50.0</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">51.1</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">54.4</td>
</tr>
<tr>
<td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">CLIcK</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">27.2</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">22.5</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">27.3</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">24.0</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">36.7</td>
</tr>
<tr>
<td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">HAERAE</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">21.5</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">18.9</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">20.4</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">24.8</td>
<td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">37.0</td>
</tr>
</tbody>
</table>
<p style="margin-top:12px;font-size:11px;opacity:0.7">
* All results are obtained using LM Evaluation Harness and may differ from other reported scores.<br>
* While baselines are trained on 2T–36T tokens and frequently leverage knowledge distillation, Opt.Gear-270M is trained on only 0.5T tokens without any distillation. Against the similarly-sized distilled Gemma3-270M, Opt.Gear-270M leads on reasoning (PIQA 67.3 vs 66.9, ARC-Easy 59.1 vs 56.1) and Korean benchmarks (KMMLU 30.0 vs 28.0, KoBEST 51.9 vs 50.0, HAERAE 21.5 vs 20.4).
</p>
</div>

### On-Device Inference

Opt.Gear-270M shares the same NPU-friendly hybrid architecture as Opt.Gear-1B, which achieves up to ×4.9 faster prefill and decoding on NPUs compared to models of similar scale (measured with Qualcomm AI Runtime and CoreML — see the [Opt.Gear-1B card](https://huggingface.co/OptGear/Opt.Gear-1B) and the tech report for detailed device benchmarks). <!-- TODO: 270M 자체 디바이스 측정치가 있으면 표로 교체 -->

## Quickstart

### Hugging Face Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OptGear/Opt.Gear-270M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [
    {"role": "user", "content": "온디바이스 AI가 왜 중요한지 설명해줘."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
```
<!-- 
### llama.cpp

```shell
# TODO: GGUF 파일명 확정 후 교체
llama-cli -m opt-gear-270m-q4.gguf -p "안녕하세요" -n 128 -->
<!-- ``` -->

### On-Device Deployment

Executable binaries optimized for NPU inference are provided for:

- **Qualcomm Hexagon NPU** — via Qualcomm AI Engine Direct (QAIRT); recommended for Snapdragon devices
- **Apple ANE** — via CoreML Runtime
- **ExecuTorch** — for mobile/edge PyTorch deployment

<!-- TODO: 디바이스별 배포 가이드 링크 추가 -->

## Best Practices

1. **Termination tokens**: The Instruction model terminates generation with `<EOT>`, while the Base model uses `<EOS>`. When fine-tuning either variant, ensure training examples terminate with the matching token.

2. **Runtime selection**: Snapdragon NPU → QAIRT; Apple devices → CoreML; CPU/GPU → llama.cpp. Avoid the llama.cpp NPU backend for models with sliding-window attention.

3. **Languages**: The model is trained primarily on English (~92%) and Korean (~6%) with mathematical text (~2%). It is best suited for Korean-English bilingual understanding, summarization, rewriting, and instruction following on latency-sensitive devices.

4. **Not intended for code generation**: The pre-training corpus deliberately excludes code data — models at the 270M–1B scale are not typically used for code generation tasks.

5. **Long context**: The model natively supports 65,536 tokens with a fixed-size local cache, making long-context decoding memory-efficient even at this scale.

## Limitations

The constrained 0.5T-token training budget limits capabilities in complex reasoning and mathematical tasks that typically benefit from larger training corpora. At the 270M scale, knowledge-heavy benchmarks (e.g., MMLU) remain challenging; the model is best used for well-scoped generation and understanding tasks rather than open-domain knowledge QA. See the tech report for a detailed discussion.

## Citation

If you find our work helpful, feel free to give us a cite.

```bibtex
@misc{optgear2026,
    title  = {{Opt-Gear} Technical Report},
    author = {{Opt.Gear Team}},
    year   = {2026},
    url    = {https://huggingface.co/OptGear}
}
```

---

Correspondence: [contact@opt-ai.kr](mailto:contact@opt-ai.kr) · Hugging Face: [huggingface.co/OptAI](https://huggingface.co/OptGear)