LiteLLM

The router as the classifier of LiteLLM's Auto Router. The auto router (auto_router/complexity_router) sorts each request into a tier and sends it to the model you put in that tier. The classifier is s1d1-tantive, a container image that serves the router: it answers the tier in a few milliseconds, with no LLM call. Keys, budgets, logs and fallbacks stay in LiteLLM.

Status: the config.yaml below was tested locally on 2026-10-25 with LiteLLM 1.106.4 against the hosted s1d1 API (then on s1d1-tantive 0.8.0). The admin-UI pitfalls of section 5 come from an earlier run on LiteLLM 1.103.2 with real opencode traffic.

The maintained reference is s1d1-tantive docs/litellm.md (admin GUI, curl registration, Test Routing); this page keeps the short path and what went wrong once.

1. Run the service

docker run --rm -p 8788:8788 ghcr.io/scombat/s1d1-tantive:0.8.1-small-onnx-int8
curl localhost:8788/ready   # {"ready": true} once loaded and warmed up (a few seconds); 503 before

One image per variant (full, compact, small, tiny); the weights are inside, nothing is downloaded at start. The small one takes a few milliseconds per request on a CPU. Multi-arch, signed. Since 0.8.1 the int8 weights are the reduce_range ones (correction of 2026-10-25: earlier small images gave wrong scores on x86 CPUs without VNNI), so use 0.8.1 or later. To try the routing without running anything, the free hosted API https://s1d1.qoo.gy/api works as api_base (what you send is kept and used to retrain s1d1; read its terms first).

2. Configure LiteLLM (1.106 or later)

Tested with LiteLLM 1.106.6. The service is registered as an ordinary deployment, and the router points at it by name:

model_list:
  # ... your four models: cheap, medium, strong, reasoning
  - model_name: s1d1-classifier
    litellm_params:
      model: hosted_vllm/s1d1-small    # the name after the slash is sent and ignored by the service
      api_base: http://HOST:8788       # the service root, LiteLLM appends /v1/systemone
    model_info:
      mode: decisions
  - model_name: s1d1-auto
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_default_model: medium
      complexity_router_config:
        classifier_type: oss_classifier
        opensource_classifier_config:
          deployment_name: s1d1-classifier
          timeout_ms: 1000
        classifier_fallback: default_model
        tiers:
          SIMPLE: cheap
          MEDIUM: medium
          COMPLEX: strong
          REASONING: reasoning

Call s1d1-auto like any model. If the service is slow, unreachable or answers a name the router does not know, the request goes to the default model (classifier_fallback: default_model): a timeout opens a circuit breaker for 30 seconds, a stopped service falls back on each request. fallback_tier and tier_definitions are for LiteLLM Enterprise custom tiers; they are not needed here. On the service, /metrics counts tantive_decisions_total by tier: that is the check that the service was asked, since the header x-litellm-complexity-router-cause reads jev_classifier for any classifier. Where the admin GUI offers no deployment for the classifier, the fallback is provider: laya, model: multilingual, api_base, timeout_ms in place of deployment_name (LiteLLM's laya provider accepts only three model names; the service does not read it); it gave the same tiers on the same /v1/systemone.

3. The tier-name trap

The router's tier names must be LiteLLM's four (SIMPLE, MEDIUM, COMPLEX, REASONING), which the service always recognises whatever its S1D1_TIERS. A name the service does not know sends every request to the fallback: if routing looks stuck on one model, check the names first. To move the bounds, set them on the service (S1D1_TIERS), not in LiteLLM. Scores above about 0.9 are rare, so the top tier sees little traffic.

4. LiteLLM before 1.106

No oss_classifier there. Mount the chat adapter (S1D1_FORMATS=...,chat), register the service as an OpenAI-compatible model with API base http://HOST:8788/v1, and in the auto router use classification method LLM Classifier with that model, timeout 1000 ms, fallback heuristic. Details: s1d1-tantive docs/formats.md. This is the path that was tested on 1.103.2.

5. Things that went wrong once (Admin UI)

A wildcard model (provider/*) cannot be picked as a tier model. Declare each upstream model explicitly. If the upstream is another LiteLLM, its name must not keep your local prefix. Saved credentials avoid retyping the key.
Do not rename the standard tiers. The UI sends its own labels to the classifier; with the four standard tiers leave display names empty. Custom names go through Edit tiers and must match the service (section 3).
"Add tier" fails with crypto.randomUUID is not a function over plain HTTP on an IP address: browsers only provide it on HTTPS or localhost. Use an SSH tunnel (ssh -L 4000:localhost:4000 server, then http://localhost:4000/ui) or HTTPS.
"Add Auto Router" is greyed out with no tooltip. A disabled button gets no hover; press Enter in the name field and the reason appears as an error message.

6. Limits

Current message only. The service classifies the last user message, what the router was trained on (single prompts, up to 1,024 tokens): a short follow-up such as "yes, go ahead" lands in a low tier.
The tier is the only output that reaches the auto router; task type and tool need are in the service's log.