Spaces:
Running on CPU Upgrade
fix: disable MiniMax-M3 thinking via thinking_mode (fixes classifier 60s timeout)
Problem
Claude Code's auto-mode permission classifier sends non-streaming (stream: false) requests. When the HF router's router-auto policy routes one of those to DeepInfra serving MiniMaxAI/MiniMax-M3, the call hits a ~60s wall and the classifier reports "model temporarily unavailable" — which blocks every Bash/Monitor call in the session.
Root cause
The proxy already tries to disable thinking on classifier calls (detected via stop_sequences) by injecting three dialects (src/adapter/convert_request.rs):
| dialect | DeepInfra honors it for MiniMax-M3? |
|---|---|
enable_thinking: false (top-level bool) |
❌ ignored — reasoning_content still streams |
thinking: {"type":"disabled"} |
❌ ignored — reasoning_content still streams |
chat_template_kwargs: {"enable_thinking": false} |
❌ wrong variable — MiniMax-M3's chat_template.jinja reads thinking_mode (string: enabled/disabled/adaptive, default adaptive), not enable_thinking (bool) |
So on DeepInfra all three are no-ops → thinking stays ON. On a non-stream call the proxy blocks on response.text().await (src/handlers/messages.rs:189, no heartbeats in that branch) while the model silently reasons; the ~60s idle budget runs out, CloudFront/edge closes the idle connection, text().await fails, and the classifier sees "temporarily unavailable." (moon-landing records the disconnect as 499, but proxyRes.status was 200, so the router's circuit breaker counts it success and never trips — auto keeps picking DeepInfra → loop.)
Fix
Add thinking_mode: "disabled" to the chat_template_kwargs object. MiniMax-M3's template reads this variable; the existing enable_thinking bool is kept for Qwen/GLM-style templates that read that var instead. Extra kwargs are harmless (unused keys in the Jinja namespace).
dst.extra.insert(
"chat_template_kwargs".to_string(),
json!({ "enable_thinking": false, "thinking_mode": "disabled" }),
);
Verification
Direct probe against router.huggingface.co → DeepInfra, MiniMaxAI/MiniMax-M3:deepinfra, stream:true, max_tokens:20:
| request | reasoning_content? |
total |
|---|---|---|
| baseline (no disable) | streams ("The user wants…") | 3.46s |
enable_thinking: false |
streams (ignored) | 5.60s |
thinking: {type:disabled} |
streams (ignored) | 3.92s |
chat_template_kwargs: {thinking_mode: "disabled"} |
none → direct Hello! |
1.01s |
| combined shape (this PR) | none → direct Hello!, HTTP 200 (no 400) |
0.88s |
3.4× faster, reasoning suppressed, combined shape accepted (no 400). cargo test green (4/4); test_stop_sequences_disable_thinking now also asserts thinking_mode: "disabled" is present.
Note: Generated with Claude Code, acting for @Pierrci .
Investigation trace: anthropic-proxy-classifier-thinking-timeout-5dc27541.jsonl (HF internal bucket, 906KB) — full session: ES/Mongo queries, DeepInfra probes, root-cause derivation. Also on Traces: https://traces.com/s/jn7ewfqkrv48s1138011mmanps8bv22v