Spaces:

---

XciD

/

anthropic-messages-proxy

like 0

Running on CPU Upgrade

fix: stop sending unsupported "disable thinking" dialects to strict backends (fireworks 400 breaking Claude Code auto-mode classifier)

Note on authorship: I am Claude Code, an AI coding agent, acting as an agent for @pierric (HuggingFace infra). Pierric asked me to investigate and fix this bug and to open this PR on his behalf. The diagnosis, code, and tests below are mine; the reproduction data (claude-classifier-debug.log) and access to HF telemetry are Pierric's. Pinging for review since the Space is owned by @XciD / HF infra.

TL;DR

Strict HF-router backends (notably Fireworks) reject the enable_thinking / chat_template_kwargs fields the converter injects for stop-sequence calls, returning 400 Extra inputs are not permitted. Claude Code's auto-mode safety classifier issues those exact calls; the 400 makes the SDK fail closed, denying every non-allowlisted tool (Bash, Agent) for the whole session. This PR strips the offending fields and retries, so the classifier stops breaking on strict backends.

Reproduction / evidence

Claude Code on zai-org/GLM-5.2:fireworks-ai (via router.huggingface.co/v1/messages). Main-loop /v1/messages calls succeed; every auto-mode classifier side-query (/v1/messages?beta=true, stop_sequences set) returns:

400 {"type":"error","error":{"type":"api_error","message":"{\"error\":{\"object\":\"error\",\"type\":\"invalid_request_error\",\"code\":\"invalid_request_error\",\"message\":\"2 request validation errors: Extra inputs are not permitted, field: 'chat_template_kwargs'; Extra inputs are not permitted, field: 'enable_thinking', value: False\"}}"}}

Claude Code then logs Auto mode classifier unavailable, denying with retry guidance (fail closed) and denies all Bash/Agent permissions until the session is cancelled. Traced end-to-end in a user's claude-classifier-debug.log and cross-checked against HF telemetry (ds-filebeat-spaces-cp Space stdout + moon-landing.inference_proxy_calls Mongo), where the Fireworks 400s line up to the second with the user's local classifier-error dumps.

Root cause

src/adapter/convert_request.rs (commit a4eaefc, "feat: support Claude Code auto-mode classifier") added, for any request with stop_sequences, into dst.extra:

• enable_thinking: false (together/zai dialect)
• thinking: { type: "disabled" } (Anthropic dialect)
• chat_template_kwargs: { enable_thinking: false } (TGI dialect)

The comment says "we emit every known convention because the actual provider behind the HF router honours a different one." That strategy banks on all backends being lenient — and Fireworks is strict: it validates the top-level body and rejects chat_template_kwargs and enable_thinking as Extra inputs are not permitted → 400. (Together and Zhipu's api.z.ai are lenient and ignore the foreign fields, which is why the bug only manifests on Fireworks.) Same bug class as the dangling tool_choice rejection fixed in 669306e.

Fix (in src/handlers/messages.rs; converter untouched)

Keep the converter's "emit all dialects" behavior (it's there for a real reason — keeping the latency-sensitive classifier call fast on whichever backend the router picks, including the no-:provider-suffix router-auto case). Add strip-and-retry in the request path:

• A 400 with stop_sequences whose body names one of the three known thinking fields is parsed for the rejected field set.
• Those fields are removed from extra, the accept-set is cached per model in a LazyLock<RwLock<HashMap<String, HashSet<String>>>>, and the request is retried once.
• Subsequent calls for that model pre-strip the cached set before send, so steady state sends a clean request — the probe costs a single redundant 400 once per model per process lifetime.
• A 400 that does not name a known thinking field (genuine malformed request — bad auth, unknown model, etc.) falls through to the existing UpstreamError path unchanged. Retry only engages for the classifier dialect-rejection case.

This is self-healing across backends (no provider enumeration — it learns the accept-set from the validator's own error message), which matters because the model id may arrive without a :provider suffix when the router auto-picks.

Verification

• cargo build clean (one compile fix: RwLock::const_new(HashMap::new()) → LazyLock::new(|| RwLock::new(HashMap::new())) since HashMap::new is not const).
• cargo test → 13/13 passing, including two new tests in handlers::messages:
  • test_parse_rejected_fields_fireworks_shape — pins the exact 400 body from the user's debug log and asserts chat_template_kwargs + enable_thinking are extracted.
  • test_parse_rejected_fields_genuine_400_is_empty — a model-not-found 400 returns empty (won't trigger a false retry).
• All 11 pre-existing tests still pass (converter behavior preserved).

Why not change the converter instead?

A route-aware converter (only emit the dialect for the backend the :provider suffix implies) is simpler, but it breaks for router-auto/no-suffix model ids — and leaving unknown/unsuffixed models with no thinking-disable sends the classifier back to the original latency-timeout this block exists to prevent. The strip-and-retry approach is robust to that and to future strict backends.

🤖 Authored by Claude Code acting as agent for @pierric .