Instructions to use switchboardllm/switchboard-router-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use switchboardllm/switchboard-router-4b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="switchboardllm/switchboard-router-4b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("switchboardllm/switchboard-router-4b")
model = AutoModelForCausalLM.from_pretrained("switchboardllm/switchboard-router-4b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use switchboardllm/switchboard-router-4b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "switchboardllm/switchboard-router-4b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "switchboardllm/switchboard-router-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/switchboardllm/switchboard-router-4b

SGLang

How to use switchboardllm/switchboard-router-4b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "switchboardllm/switchboard-router-4b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "switchboardllm/switchboard-router-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "switchboardllm/switchboard-router-4b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "switchboardllm/switchboard-router-4b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use switchboardllm/switchboard-router-4b with Docker Model Runner:
```
docker model run hf.co/switchboardllm/switchboard-router-4b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

switchboard-router-4b

Switchboard 4B is a router: it reads a prompt and answers with one line of JSON, the cheapest model tier that can handle it. Send easy prompts to a small model and pay frontier prices only for the prompts that need it. Open weights, Apache 2.0.

This is the larger sibling of switchboard-router-0.5b. Same training data, same answer format, same test set. On 10,000 held-out prompts it saves more and sends fewer hard prompts to a cheap model.

Try the 0.5B in your browser: Switchboard demo · Run the 4B locally: GGUF for Ollama

Run it

With Ollama (4.28 GB download, a GPU is recommended):

ollama run hf.co/switchboardllm/switchboard-router-4b-GGUF "Prove there are infinitely many primes of the form 4k+3"

{"tier":"frontier","domain":"math"}

With Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer
import json

repo = "switchboardllm/switchboard-router-4b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")

msgs = [{"role": "user", "content": "Unscramble the following words: eglar, ngareo, kobo"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=24, do_sample=False)
print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))
# {'tier': 'frontier', 'domain': 'general'}

With Python, using the switchboard-router package:

from switchboard_router import Router

router = Router(backend="ollama", backend_model="hf.co/switchboardllm/switchboard-router-4b-GGUF")
router.pick_model("Write a haiku about rain")
# 'anthropic/claude-haiku-5.5'

Use greedy decoding (do_sample=False, or temperature 0).

What it answers

Field	Values
`tier`	`small`, `mid` or `frontier`: the cheapest class of model expected to answer the prompt well
`domain`	`code`, `math`, `writing` or `general`: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

4B vs 0.5B

Both models on the same 10,000 held-out prompts they never saw in training, in their natural mix:

	0.5B (v0.1)	4B	4B minus 0.5B (95% CI)
Cost saved vs. always using Claude Opus 5.5	64.5%	67.0%	+2.5 pts (+1.8 to +3.3)
Hard prompts sent to frontier	59.1%	63.2%	+4.1 pts (+1.4 to +6.7)
Sent too low (needed frontier, sent cheaper)	5.5%	5.0%	-0.5 pts (-0.9 to -0.2)
Prompts served well (proxy)	94.5%	95.0%
Ranking AUC (hard vs. rest)	0.787	0.827
Tier accuracy	60.3%	64.0%
Invalid JSON outputs	0	0
Time per decision (RTX 5070 Ti, Transformers)	288 ms	489 ms

Confidence intervals come from a paired bootstrap over the 10,000 prompts (2,000 resamples), and none of them include zero. The gain is real but moderate: the ceiling is set by the labels, and a perfect router on this data would save 76.2%.

How it's measured. Each prompt in routellm/gpt4_dataset comes with a 2023 Mixtral-8x7B answer scored 1-5 by a GPT-4 judge. A prompt counts as served well if Switchboard sent it to frontier, or sent it lower and that answer scored at least 4. Cost uses Surplus output prices per 1M tokens: Claude Haiku 5.5 ($0.50) for small, Claude Sonnet 5.5 ($10) for mid, Claude Opus 5.5 ($20) for frontier. Method and every 0.5B answer are on the evaluation page.

Builds

Build	Size	Runs with	Same decision as this model
This repo (safetensors, BF16)	8.04 GB	Transformers, vLLM	reference
GGUF Q8_0	4.28 GB	Ollama, llama.cpp	339 of 347 test prompts

Q4_K_M agreed on only 302 of 347, so it is not published. The few disagreements left in Q8_0 are borderline prompts: the BF16 model itself changes 5 of 347 decisions between batched and one-at-a-time inference.

The model


Base	Qwen3-4B-Instruct-2507 (Apache-2.0)
Method	LoRA, rank 32, on all attention and MLP projections, merged into BF16 weights
Training data	`routellm/gpt4_dataset` (Apache-2.0, LMSYS), 63,749 rows, the same set as 0.5B v0.1
Labels	Mixtral score 5 → small, 4 → mid, 1-3 → frontier; `domain` from keywords
Training	1 epoch in 184 minutes on one RTX 5070 Ti. Best checkpoint by eval loss (0.0849). Loss on the JSON answer only.
Prompt format	The 0.5B chat template, so both models are drop-in replacements for each other
Prompt length	Trained on prompts cut to 512 tokens

Limitations

It still misroutes some prompts: 5.0% of held-out prompts were sent to a cheaper tier than they needed.
Labels come from 2023 models and a GPT-4 judge, not from people. Mostly English prompts.
It reads one prompt at a time, not the whole conversation, and only the first 512 tokens.
A prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.
domain comes from keyword labels and is only a hint.
Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.

License

Apache-2.0, inherited from Qwen3-4B-Instruct-2507 and routellm/gpt4_dataset.

Downloads last month: -

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for switchboardllm/switchboard-router-4b

Base model

Qwen/Qwen3-4B-Instruct-2507

Finetuned

(2446)

this model

Quantizations

1 model

switchboardllm
/

switchboard-router-4b