File size: 3,015 Bytes
408a34b ecc1989 408a34b ecc1989 408a34b 27192f3 b41289e 27192f3 408a34b ecc1989 408a34b ecc1989 27192f3 408a34b 27192f3 408a34b ecc1989 408a34b 27192f3 15216e2 27192f3 15216e2 27192f3 15216e2 27192f3 15216e2 27192f3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | ---
license: apache-2.0
base_model: Qwen/Qwen3-Embedding-0.6B
tags:
- router
- model-routing
- embeddings
- mlx
---
# coding-router — routing model
The routing model for [coding-router](https://github.com/experientiallabs/coding-router):
one local endpoint that optimizes coding requests between big and small models, so you get
more usage at the same cost. Given a coding task (or a whole agent conversation), it picks
the model and reasoning effort with the best measured cost/quality trade-off. Routing runs
fully locally — this encoder runs in-process on your machine.
## Files
| path | what |
|---|---|
| `router.json` | model list, decision-rule parameters, provenance |
| `router.npz` | the evidence bank: task embeddings plus measured outcome and cost per model |
| `encoder-fp16/` | the tuned Qwen3-Embedding-0.6B, fp16 safetensors — CUDA / CPU via sentence-transformers |
| `encoder-mlx-4bit/` | the same tuned encoder, 4-bit MLX for Apple Silicon |
Both encoder directories are the same model in two precisions. The server downloads only
the one it needs: MLX on Apple Silicon, torch elsewhere.
## What the router is
Qwen3-Embedding-0.6B, fine-tuned (LoRA on the attention projections) to route coding work.
At inference: embed the request, compare it against a bank of tasks with measured outcomes
for every model, and pick the cheapest model predicted to solve it. If nothing in the bank
is similar to the request, the router does not guess — it escalates to the strongest model.
<!-- Benchmarks section format set by Kion (2026-08-03): the shipped artifact's live-measured
per-source results, with a savings column. Plain model names, no protocol jargon. -->
## Benchmarks
Live-measured on held-out tasks the router never saw during fitting, against always using
opus-5 (high), the strongest single model:
| task type | router quality | opus-5 quality | cheaper | saved |
|---|---|---|---|---|
| short interactive requests | 1.000 | 1.000 | **6.0×** | **83%** |
| competitive programming (LiveCodeBench) | 0.961 | 0.997 | **6.0×** | **83%** |
| long-horizon software tasks (DeepSWE) | 0.908 | 0.926 | **3.8×** | **74%** |
Quality differences are within statistical noise on every task type; the saving is the win.
## Scope
The router was validated on repo-issue-style tasks (a paragraph or more of real problem
description). Very short one-line prompts often have no close match in the evidence bank;
the router then escalates to the strongest model rather than guess — safe, but no saving.
Check `off_distribution` on the decision if you want to know when that happened.
## Use
You never fetch this by hand — `uv run python -m router.serve` in
[coding-router](https://github.com/experientiallabs/coding-router) downloads it on first
run, then runs offline. New router versions overwrite this repo in place; old versions
stay in the repo's git history.
This model contains only benchmark-task embeddings and measured outcomes — no user data
of any kind.
|