| | --- |
| | library_name: transformers |
| | base_model: Qwen/Qwen3.5-9B |
| | tags: |
| | - qwen3.5 |
| | - code |
| | - agent |
| | - sft |
| | - omnicoder |
| | - tesslate |
| | license: apache-2.0 |
| | language: |
| | - en |
| | pipeline_tag: text-generation |
| | model-index: |
| | - name: OmniCoder-9B |
| | results: |
| | - task: |
| | type: text-generation |
| | dataset: |
| | name: AIME 2025 |
| | type: custom |
| | metrics: |
| | - name: pass@5 |
| | type: accuracy |
| | value: 90.0 |
| | - task: |
| | type: text-generation |
| | dataset: |
| | name: GPQA Diamond |
| | type: custom |
| | metrics: |
| | - name: pass@1 |
| | type: accuracy |
| | value: 83.8 |
| | - name: pass@3 |
| | type: accuracy |
| | value: 86.4 |
| | - task: |
| | type: text-generation |
| | dataset: |
| | name: Terminal-Bench 2.0 |
| | type: custom |
| | metrics: |
| | - name: Pass Rate |
| | type: accuracy |
| | value: 28.1 |
| | --- |
| | |
| | <div align="center"> |
| |
|
| | <img src="omnicoder-banner.png" alt="OmniCoder" width="720"> |
| |
|
| | # OmniCoder-9B |
| |
|
| | ### A 9B coding agent fine-tuned on 425K agentic trajectories. |
| |
|
| | [](https://opensource.org/licenses/Apache-2.0) |
| | [](https://huggingface.co/Qwen/Qwen3.5-9B) |
| | [](https://huggingface.co/Tesslate/OmniCoder-9B-GGUF) |
| |
|
| | !! 3/12/26 Update -> [Install For Your Coding Agents](https://tesslate.com/install#omnicoder) |
| |
|
| | [Get Started](#quickstart) | [Benchmarks](#benchmarks) | [GGUF Downloads](https://huggingface.co/Tesslate/OmniCoder-9B-GGUF) |
| |
|
| | --- |
| |
|
| | </div> |
| |
|
| | ## Overview |
| |
|
| | **OmniCoder-9B** is a 9-billion parameter coding agent model built by [Tesslate](https://tesslate.com), fine-tuned on top of [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)'s hybrid architecture (Gated Delta Networks interleaved with standard attention). It was trained on **425,000+ curated agentic coding trajectories** spanning real-world software engineering tasks, tool use, terminal operations, and multi-step reasoning. |
| |
|
| | The training data was specifically built from **Claude Opus 4.6 agentic and coding reasoning traces**, targeting scaffolding patterns from Claude Code, OpenCode, Codex, and Droid. The dataset includes successful trajectories from models like Claude Opus 4.6, GPT-5.4, GPT-5.3-Codex, and Gemini 3.1 Pro. |
| |
|
| | The model shows strong agentic behavior: it recovers from errors (read-before-write), responds to LSP diagnostics, and uses proper edit diffs instead of full rewrites. These patterns were learned directly from the real-world agent trajectories it was trained on. |
| |
|
| | ### Key Features |
| |
|
| | - **Trained on Frontier Agent Traces** : Built from Claude Opus 4.6, GPT-5.3-Codex, GPT-5.4, and Gemini 3.1 Pro agentic coding trajectories across Claude Code, OpenCode, Codex, and Droid scaffolding |
| | - **Hybrid Architecture** : Inherits Qwen3.5's Gated Delta Networks interleaved with standard attention for efficient long-context processing |
| | - **262K Native Context** : Full 262,144 token context window, extensible to 1M+ |
| | - **Error Recovery** : Learns read-before-write patterns, responds to LSP diagnostics, and applies minimal edit diffs instead of full rewrites |
| | - **Thinking Mode** : Supports `<think>...</think>` reasoning chains for complex problem decomposition |
| | - **Apache 2.0** : Fully open weights, no restrictions |
| |
|
| | --- |
| |
|
| | ## Benchmarks |
| |
|
| | <div align="center"> |
| |
|
| | | Benchmark | **OmniCoder-9B** | Qwen3.5-9B | Qwen3-Next-80B | GPT-OSS-120B | GPT-OSS-20B | GLM-4.7-Flash | GLM 4.7 | Claude Haiku 4.5 | |
| | |:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| |
| | | **AIME 2025** (pass@5) | 90 | | | | 91.7 | 91.6 | | | |
| | | **GPQA Diamond** (pass@1) | **83.8** | 81.7 | 77.2 | 80.1 | 71.5 | | | 73 | |
| | | **GPQA Diamond** (pass@3) | **86.4** | | | | | | | | |
| | | **Terminal-Bench 2.0** | **23.6** | 14.6 | | | | | 33.4 | 27 | |
| |
|
| | </div> |
| |
|
| | - **GPQA Diamond pass@1: 83.8%** (166/198). +2.1 points over the Qwen3.5-9B base model (81.7). At pass@3: **86.4** (171/198). |
| | - **AIME 2025 pass@5: 90%** (27/30). |
| | - **Terminal-Bench 2.0: 23.6%** (21/89). +8.99 points (+61% improvement) over the Qwen3.5-9B base model (14.6%, 13/89). |
| |
|
| | --- |
| |
|
| | ## Quickstart |
| |
|
| | ### Transformers |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | |
| | model_id = "Tesslate/OmniCoder-9B" |
| | |
| | tokenizer = AutoTokenizer.from_pretrained(model_id) |
| | model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto") |
| | |
| | messages = [ |
| | {"role": "system", "content": "You are a helpful coding assistant."}, |
| | {"role": "user", "content": "Write a Python function to find the longest common subsequence of two strings."}, |
| | ] |
| | |
| | text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| | inputs = tokenizer([text], return_tensors="pt").to(model.device) |
| | |
| | outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95, top_k=20) |
| | print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)) |
| | ``` |
| |
|
| | ### vLLM |
| |
|
| | ```bash |
| | vllm serve Tesslate/OmniCoder-9B --tensor-parallel-size 1 --max-model-len 65536 |
| | ``` |
| |
|
| | ```python |
| | from openai import OpenAI |
| | |
| | client = OpenAI(base_url="http://localhost:8000/v1", api_key="token") |
| | response = client.chat.completions.create( |
| | model="Tesslate/OmniCoder-9B", |
| | messages=[{"role": "user", "content": "Explain the difference between a mutex and a semaphore."}], |
| | temperature=0.6, |
| | ) |
| | print(response.choices[0].message.content) |
| | ``` |
| |
|
| | ### llama.cpp (GGUF) |
| |
|
| | ```bash |
| | llama-cli --hf-repo Tesslate/OmniCoder-9B-GGUF --hf-file omnicoder-9b-q4_k_m.gguf -p "Your prompt" -c 8192 |
| | ``` |
| |
|
| | All quantizations: [Tesslate/OmniCoder-9B-GGUF](https://huggingface.co/Tesslate/OmniCoder-9B-GGUF) |
| |
|
| | --- |
| |
|
| | ## Training Details |
| |
|
| | | | | |
| | |:---|:---| |
| | | **Base Model** | [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) | |
| | | **Method** | LoRA SFT (r=64, alpha=32) | |
| | | **Dataset** | 425K agentic trajectories from 5 sources | |
| | | **Packing** | Sample packing with 99.35% efficiency | |
| | | **Hardware** | 4x NVIDIA H200 (DDP) | |
| | | **Framework** | Axolotl | |
| | | **Precision** | bf16 | |
| | | **Optimizer** | AdamW (lr=2e-4, cosine schedule) | |
| |
|
| | --- |
| |
|
| | ## Architecture |
| |
|
| | OmniCoder inherits Qwen3.5-9B's hybrid architecture: |
| |
|
| | - **Gated Delta Networks** : Linear attention layers interleaved with standard attention for efficient long-range dependencies |
| | - **VLM Backbone** : Built on `Qwen3_5ForConditionalGeneration` |
| |
|
| | --- |
| |
|
| | ## Recommended Sampling Parameters |
| |
|
| | | Parameter | Value | |
| | |:---|:---| |
| | | Temperature | 0.6 | |
| | | Top-P | 0.95 | |
| | | Top-K | 20 | |
| | | Presence Penalty | 0.0 | |
| |
|
| | For agentic / tool-calling tasks, consider lower temperature (0.2-0.4) for more deterministic behavior. |
| |
|
| | --- |
| |
|
| | ## Limitations |
| |
|
| | - Performance on non-English tasks has not been extensively evaluated |
| | - Tool-calling format is flexible but works best with the scaffolding patterns seen in training |
| |
|
| | --- |
| |
|
| | ## Acknowledgments |
| |
|
| | Special thanks to the [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) team and the discussion in [axolotl#3453](https://github.com/axolotl-ai-cloud/axolotl/issues/3453) for helping get Qwen3.5 packing support working. |
| |
|
| | --- |
| |
|
| | ## Citation |
| |
|
| | ```bibtex |
| | @misc{omnicoder2025, |
| | title={OmniCoder-9B: A Frontier Open Coding Agent}, |
| | author={Tesslate}, |
| | year={2025}, |
| | url={https://huggingface.co/Tesslate/OmniCoder-9B} |
| | } |
| | ``` |
| |
|
| | --- |
| |
|
| | <div align="center"> |
| |
|
| | **Built by [Tesslate](https://tesslate.com)** |
| |
|
| | </div> |
| |
|
