README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-Embedding-0.6B
tags:
  - router
  - model-routing
  - embeddings
  - mlx
---

# coding-router — routing model

The routing model for [coding-router](https://github.com/experientiallabs/coding-router):
one local endpoint that optimizes coding requests between big and small models, so you get
more usage at the same cost. Given a coding task (or a whole agent conversation), it picks
the model and reasoning effort with the best measured cost/quality trade-off. Routing runs
fully locally — this encoder runs in-process on your machine.

## Files

| path | what |
|---|---|
| `router.json` | model list, decision-rule parameters, provenance |
| `router.npz` | the evidence bank: task embeddings plus measured outcome and cost per model |
| `encoder-fp16/` | the tuned Qwen3-Embedding-0.6B, fp16 safetensors — CUDA / CPU via sentence-transformers |
| `encoder-mlx-4bit/` | the same tuned encoder, 4-bit MLX for Apple Silicon |

Both encoder directories are the same model in two precisions. The server downloads only
the one it needs: MLX on Apple Silicon, torch elsewhere.

## What the router is

Qwen3-Embedding-0.6B, fine-tuned (LoRA on the attention projections) to route coding work.
At inference: embed the request, compare it against a bank of tasks with measured outcomes
for every model, and pick the cheapest model predicted to solve it. If nothing in the bank
is similar to the request, the router does not guess — it escalates to the strongest model.

<!-- Benchmarks section format set by Kion (2026-08-03): the shipped artifact's live-measured
     per-source results, with a savings column. Plain model names, no protocol jargon. -->

## Benchmarks

Live-measured on held-out tasks the router never saw during fitting, against always using
opus-5 (high), the strongest single model:

| task type | router quality | opus-5 quality | cheaper | saved |
|---|---|---|---|---|
| short interactive requests | 1.000 | 1.000 | **6.0×** | **83%** |
| competitive programming (LiveCodeBench) | 0.961 | 0.997 | **6.0×** | **83%** |
| long-horizon software tasks (DeepSWE) | 0.908 | 0.926 | **3.8×** | **74%** |

Quality differences are within statistical noise on every task type; the saving is the win.

## Scope

The router was validated on repo-issue-style tasks (a paragraph or more of real problem
description). Very short one-line prompts often have no close match in the evidence bank;
the router then escalates to the strongest model rather than guess — safe, but no saving.
Check `off_distribution` on the decision if you want to know when that happened.

## Use

You never fetch this by hand — `uv run python -m router.serve` in
[coding-router](https://github.com/experientiallabs/coding-router) downloads it on first
run, then runs offline. New router versions overwrite this repo in place; old versions
stay in the repo's git history.

This model contains only benchmark-task embeddings and measured outcomes — no user data
of any kind.