README.md

---
license: mit
task_categories:
- text-generation
tags:
- llm-routing
- code-generation
- benchmark
- static-embeddings
pretty_name: Learning to Route
---

# Learning to Route — coding router benchmark + anchor tables

Companion artifacts for [Learning to Route](https://github.com/lee101/learning-to-route): a 27-task coding benchmark (medium-hard exact tasks plus quality-scored optimization tasks, all with adversarial hidden tests), per-model outcome logs from cheap LLMs plus one frontier tier (deepseek-v4-flash, gpt-5.4-nano/mini, gemini-3.5-flash, gpt-5.5), and trained router anchor tables (static-embedding vectors plus per-model score/cost stats).

Headline result: a verify and escalate cascade over the four cheap models solves 100% of benchmark tasks at 26% of the cost of the best single model and 4% of the cost of gpt-5.5, which ties a $0.03 model at 77.8% pass. Per solved task: cascade $0.0041 vs frontier $0.1213. Full numbers in `report.json`.

![frontier](paper/figs/frontier.png)

Routing = one static embed (~0.15ms CPU, 16MB model) + k-NN over these anchors. Serve from Python ([pybed](https://github.com/lee101/pybed)), Go ([gobed](https://github.com/lee101/gobed)), or Zig ([zbed](https://github.com/lee101/zbed)).

- `tasks.jsonl` — benchmark tasks (prompt, entry_point, difficulty, hidden tests)
- `results.jsonl` — (task, model, passed, cost, latency) outcome log
- `router.json` — trained anchor table: `{text, vec[512], stats: {model: {n, pass, cost}}}`

See the [whitepaper](https://github.com/lee101/learning-to-route/blob/main/paper/learning-to-route.md).