KBBridge commited on
Commit
986da38
·
verified ·
1 Parent(s): e7fff08

Card: reasoning off por defecto, con las mediciones

Browse files
Files changed (1) hide show
  1. README.md +17 -15
README.md CHANGED
@@ -20,26 +20,28 @@ code to an external API.
20
 
21
  ## ⚠️ Read this before your first prompt
22
 
23
- Two settings, or the model will look broken. Both are measured, not stylistic.
24
 
25
- ### 1. Turn reasoning OFF
26
 
27
- ```bash
28
- vllm serve KBBridge/KBBridge-v3 --served-model-name kbbridge-v3 \
29
- --max-model-len 262144 --reasoning-parser qwen3
30
 
31
- # and per request:
32
- # "chat_template_kwargs": {"enable_thinking": false}
33
- ```
 
 
 
 
34
 
35
- The Qwen chat template enables a `<think>` block by default. If that block does not close
36
- within your token budget, the answer comes back **empty or half-finished** — the client sees
37
- "the model did not respond". Measured on this GGUF: with reasoning on, 300 tokens were not
38
- enough to even begin the object; with it off, the same prompt returned a complete, valid
39
- `Procedure`.
40
 
41
- With vLLM, pass `chat_template_kwargs: {enable_thinking: false}` on every request, or set
42
- it as a server default.
43
 
44
  ### 2. Ask for the format explicitly
45
 
 
20
 
21
  ## ⚠️ Read this before your first prompt
22
 
23
+ One thing to know, one thing to do. Both are measured, not stylistic.
24
 
25
+ ### 1. Reasoning is OFF in this checkpoint — keep it that way
26
 
27
+ **You do not need to do anything.** `chat_template.jinja` pins `enable_thinking = false`.
 
 
28
 
29
+ This is a **deliberate deviation from the upstream Qwen template**, which enables reasoning by
30
+ default at `reasoning_effort: xhigh`. Measured on our 580-item benchmark:
31
+
32
+ | | parseRate | parmMatch | MCQ |
33
+ |---|---|---|---|
34
+ | reasoning **off** | **90.0** | **80.9** | **78.1** |
35
+ | reasoning **on** (upstream default) | **24.1** | 42.8 | 69.6 |
36
 
37
+ With reasoning on the model burns its whole token budget thinking and returns an **empty
38
+ `content`**, with the prose in `reasoning_content` — the client sees "no answer". Mean latency
39
+ per item went from 9 s to 73 s while throughput was *higher*: it was generating ~25× more
40
+ tokens, nearly all discarded. Writing `.gxSource` is a formatting task; the reasoning does not
41
+ help and the budget does.
42
 
43
+ If you want reasoning back, drop the `{%- set enable_thinking = false %}` line from the
44
+ template.
45
 
46
  ### 2. Ask for the format explicitly
47