pocharlies commited on
Commit
3b2bcab
·
verified ·
1 Parent(s): 5c02a80

Parsers verified: qwen3 + qwen3_coder

Browse files
README.md CHANGED
@@ -109,6 +109,11 @@ should leave ~33–35 GB.
109
  reclaim storm). Avoid large concurrent downloads.
110
  - Metrics come with the `sglang:*` prefix (including
111
  `sglang:spec_accept_length` to watch the speculative decoding).
 
 
 
 
 
112
 
113
  ## Sampling and thinking controls (from the official model card)
114
 
 
109
  reclaim storm). Avoid large concurrent downloads.
110
  - Metrics come with the `sglang:*` prefix (including
111
  `sglang:spec_accept_length` to watch the speculative decoding).
112
+ - Parsers verified in production: `--reasoning-parser qwen3` and
113
+ `--tool-call-parser qwen3_coder` (the template emits XML tool calls).
114
+ Reasoning arrives in `reasoning_content`, tool calls come structured, and
115
+ sglang#36537 (thinking + qwen3_coder looping) did not reproduce on this
116
+ build — tested with thinking, tools, both combined, and streaming.
117
 
118
  ## Sampling and thinking controls (from the official model card)
119
 
k8s/qwen38-flash-next-nvfp4-sglang.yaml CHANGED
@@ -401,10 +401,15 @@ spec:
401
  - { name: CHUNKED_PREFILL_SIZE, value: "2048" }
402
  # Covers 16 running with margin for the NEXTN tree shapes.
403
  - { name: CUDA_GRAPH_MAX_BS, value: "32" }
404
- # Empty = flag absent. Fill in once the qwen4_exp build confirms
405
- # the parser names (candidates: qwen3 / qwen).
406
- - { name: REASONING_PARSER, value: "" }
407
- - { name: TOOL_CALL_PARSER, value: "" }
 
 
 
 
 
408
  # Bounds the host-side spike of CUDA graph capture. Without this,
409
  # inductor spawns ~1 compile worker per core (20) and the
410
  # 20-40 GB spike kills the cgroup 30-90 s after "Capture begin".
@@ -578,8 +583,9 @@ spec:
578
  - { name: MAMBA_TRACK_INTERVAL, value: "64" }
579
  - { name: CHUNKED_PREFILL_SIZE, value: "2048" }
580
  - { name: CUDA_GRAPH_MAX_BS, value: "32" }
581
- - { name: REASONING_PARSER, value: "" }
582
- - { name: TOOL_CALL_PARSER, value: "" }
 
583
  # See the head for the rationale of these two.
584
  - { name: TORCHINDUCTOR_COMPILE_THREADS, value: "4" }
585
  - { name: MAX_JOBS, value: "1" }
 
401
  - { name: CHUNKED_PREFILL_SIZE, value: "2048" }
402
  # Covers 16 running with margin for the NEXTN tree shapes.
403
  - { name: CUDA_GRAPH_MAX_BS, value: "32" }
404
+ # Verified in production: `qwen3` matches the conditional <think>
405
+ # detector (same as vLLM's `qwen3` parser) and the checkpoint's
406
+ # chat template emits XML tool calls (<function=/<parameter=), so
407
+ # the right tool parser is `qwen3_coder` — the JSON `qwen`/`qwen25`
408
+ # detectors would parse nothing. sglang#36537 (thinking +
409
+ # qwen3_coder looping) did NOT reproduce on this build: thinking,
410
+ # tools, thinking+tools and streaming all tested clean.
411
+ - { name: REASONING_PARSER, value: "qwen3" }
412
+ - { name: TOOL_CALL_PARSER, value: "qwen3_coder" }
413
  # Bounds the host-side spike of CUDA graph capture. Without this,
414
  # inductor spawns ~1 compile worker per core (20) and the
415
  # 20-40 GB spike kills the cgroup 30-90 s after "Capture begin".
 
583
  - { name: MAMBA_TRACK_INTERVAL, value: "64" }
584
  - { name: CHUNKED_PREFILL_SIZE, value: "2048" }
585
  - { name: CUDA_GRAPH_MAX_BS, value: "32" }
586
+ # See the parser note on the head (same values, verified).
587
+ - { name: REASONING_PARSER, value: "qwen3" }
588
+ - { name: TOOL_CALL_PARSER, value: "qwen3_coder" }
589
  # See the head for the rationale of these two.
590
  - { name: TORCHINDUCTOR_COMPILE_THREADS, value: "4" }
591
  - { name: MAX_JOBS, value: "1" }