switchboard-router-32b

Switchboard 32B is a router: it reads a prompt and answers with one line of JSON, the cheapest model tier that can handle it. Send easy prompts to a small model and pay frontier prices only for the prompts that need it. Open weights, Apache 2.0.

It is the largest of three sizes trained on the same data and tested on the same 10,000 prompts: 0.5B runs in your browser, 4B runs in Ollama, and this one runs on the Surplus API.

Run it

On Surplus, with the same key as every other model, at $0.10 per 1M tokens:

from openai import OpenAI

client = OpenAI(base_url="https://api.surplusllm.com/api/v1", api_key="sk-surplus-...")
reply = client.chat.completions.create(
    model="launch/switchboardllm/switchboard-router-32b@<sha7>",  # shown on the Surplus launch page
    messages=[{"role": "user", "content": "Unscramble the following words: eglar, ngareo, kobo"}],
    max_tokens=24,
    temperature=0,
)
print(reply.choices[0].message.content)

{"tier":"frontier","domain":"general"}

Or put it in front of your app with the switchboard-router package: switchboard serve, then send model="auto".

Self-hosted, with vLLM in FP8 on one 48 GB GPU, the way Surplus serves it:

vllm serve switchboardllm/switchboard-router-32b --quantization fp8 --max-model-len 4096

Use greedy decoding (temperature 0).

What it answers

Field	Values
tier	small, mid or frontier: the cheapest class of model expected to answer the prompt well
domain	code, math, writing or general: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

Three sizes, same test

All three models on the same 10,000 held-out prompts they never saw in training, in their natural mix:

	0.5B	4B	32B
Cost saved vs. always using Claude Opus 5.5	64.5%	67.0%	64.3%
Hard prompts sent to frontier	59.1%	63.2%	67.5%
Sent too low (needed frontier, sent cheaper)	5.5%	5.0%	4.4%
Prompts served well (proxy)	94.5%	95.0%	95.6%
Ranking AUC (hard vs. rest)	0.787	0.827	0.840
Tier accuracy	60.3%	64.0%	63.2%
Invalid JSON outputs	0	0	0

32B minus 4B, paired bootstrap over the 10,000 prompts (2,000 resamples, 95% intervals):

	32B minus 4B
Cost saved	-2.8 pts (-3.3 to -2.1)
Hard prompts sent to frontier	+4.4 pts (+2.3 to +6.4)
Sent too low	-0.6 pts (-0.9 to -0.3)

A perfect router on this data would save 76.2%. The labels set that ceiling, not the model size.

How it's measured. Each prompt in routellm/gpt4_dataset comes with a 2023 Mixtral-8x7B answer scored 1-5 by a GPT-4 judge. A prompt counts as served well if Switchboard sent it to frontier, or sent it lower and that answer scored at least 4. Cost uses Surplus output prices per 1M tokens: Claude Haiku 5.5 ($0.50) for small, Claude Sonnet 5.5 ($10) for mid, Claude Opus 5.5 ($20) for frontier.

FP8

Surplus serves 20-34B launches in FP8 on a 48 GB GPU. Before launching we ran 1,347 prompts through vLLM in BF16 and in FP8: the routing decision was the same on 1,285 of them (95.4%).

The 62 that changed go both ways: FP8 picked a higher tier 32 times and a lower one 30 times. On the 1,000 labelled prompts in that set, FP8 routes as well as BF16:

	BF16	FP8
Sent too low	4.2%	4.2%
Cost saved	65.1%	64.5%

The model

Base	Qwen3-32B (Apache-2.0), 32.8B parameters
Method	QLoRA: rank 32 adapters trained on a 4-bit NF4 base, then merged into the BF16 base weights
Training data	routellm/gpt4_dataset (Apache-2.0, LMSYS), 63,749 rows, the same set as the 0.5B and 4B
Labels	Mixtral score 5 → small, 4 → mid, 1-3 → frontier; domain from keywords
Training	1 epoch in 4.1 hours on one H100 80GB. Best checkpoint by eval loss (0.0829). Loss on the JSON answer only.
Prompt format	The same chat template as the 0.5B and 4B, so the three are drop-in replacements for each other
Prompt length	Trained on prompts cut to 512 tokens

Limitations

• It still misroutes some prompts: 4.4% of held-out prompts were sent to a cheaper tier than they needed.
• Labels come from 2023 models and a GPT-4 judge, not from people. Mostly English prompts.
• It reads one prompt at a time, not the whole conversation, and only the first 512 tokens.
• A prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.
• domain comes from keyword labels and is only a hint.
• Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.

License

Apache-2.0, inherited from Qwen3-32B and routellm/gpt4_dataset.