Instructions to use froggeric/Qwen-Fixed-Chat-Templates with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use froggeric/Qwen-Fixed-Chat-Templates with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Qwen-Fixed-Chat-Templates froggeric/Qwen-Fixed-Chat-Templates
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Fix: <|think_on|>/<|think_off|> honored from tool responses (prompt-injection surface)
#49
by Moore2877 - opened
- .gitignore +4 -0
- README.md +153 -251
- archive/README-v20.md +0 -239
- archive/README-v21.md +0 -234
- archive/README-v22.1.md +0 -285
- archive/README-v22.2.md +0 -296
- archive/README-v22.3.md +0 -317
- archive/README-v22.md +0 -278
- archive/v20_chat_template.jinja +0 -287
- archive/v20_chat_template_oneline.txt +0 -1
- archive/v21_chat_template.jinja +0 -329
- archive/v21_chat_template_oneline.txt +0 -1
- archive/v22.1_chat_template.jinja +0 -363
- archive/v22.1_chat_template_oneline.txt +0 -1
- archive/v22.2_chat_template.jinja +0 -380
- archive/v22.2_chat_template_oneline.txt +0 -1
- archive/v22.3_chat_template.jinja +0 -421
- archive/v22.3_chat_template_oneline.txt +0 -1
- archive/v22_chat_template.jinja +0 -331
- archive/v22_chat_template_oneline.txt +0 -1
- chat_template.jinja +101 -243
- chat_template_oneline.txt +1 -1
- scripts/check_applied.py +0 -139
- scripts/fuzz_template.py +0 -290
- scripts/{test_v21.py → test_v20.py} +4 -15
- scripts/test_v22.py +0 -1740
.gitignore
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
.DS_Store
|
| 2 |
+
*.pyc
|
| 3 |
+
__pycache__/
|
| 4 |
+
.env
|
README.md
CHANGED
|
@@ -6,7 +6,6 @@ tags:
|
|
| 6 |
- qwen
|
| 7 |
- qwen3.5
|
| 8 |
- qwen3.6
|
| 9 |
-
- qwen3.8
|
| 10 |
- lm-studio
|
| 11 |
- mlx
|
| 12 |
- llama.cpp
|
|
@@ -15,323 +14,226 @@ tags:
|
|
| 15 |
- thinking
|
| 16 |
---
|
| 17 |
|
| 18 |
-
# Fixed jinja chat templates for Qwen 3.5
|
| 19 |
|
| 20 |
-
|
|
|
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
|
|
|
|
| 27 |
|
| 28 |
-
|
| 29 |
|
| 30 |
-
|
| 31 |
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
---
|
| 46 |
|
| 47 |
-
|
| 48 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
-
|
| 51 |
-
Run `llama-server` with the template file and DeepSeek reasoning format:
|
| 52 |
-
```bash
|
| 53 |
-
llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek
|
| 54 |
-
```
|
| 55 |
-
*Why `--reasoning-format deepseek` matters:* When connecting coding agents like OpenCode, Claude Code, or Pi.dev to `llama-server`, this flag extracts `<think>` blocks into the dedicated `reasoning_content` API response field. This prevents raw thinking tokens from leaking into the text stream and stopping tool calls midway.
|
| 56 |
|
| 57 |
-
|
|
|
|
|
|
|
| 58 |
|
| 59 |
### LM Studio
|
| 60 |
-
1. Open your Qwen model in the right
|
| 61 |
2. Scroll down to **Prompt Template**.
|
| 62 |
3. Replace the template with the contents of `chat_template.jinja`.
|
| 63 |
4. Click **Save**.
|
| 64 |
|
| 65 |
-
###
|
| 66 |
-
Replace the `"chat_template"` string in your `tokenizer_config.json` with `chat_template_oneline.txt` (or raw `chat_template.jinja`).
|
| 67 |
```bash
|
| 68 |
-
|
| 69 |
```
|
| 70 |
-
*Reasoning parser:* Use `--reasoning-parser qwen3` on vLLM to split `<think>` blocks into the response reasoning field. Note that vLLM's parser checks the top-level `enable_thinking: false` request parameter or `reasoning_effort: "none"`, but does not inspect Jinja-internal dynamic `<|think_off|>` tags in prompt bodies. To disable reasoning on vLLM, pass `"enable_thinking": false` in your request parameters.
|
| 71 |
|
| 72 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
|
| 74 |
-
### oMLX
|
| 75 |
-
Overwrite `chat_template.jinja` in your local model directory
|
| 76 |
-
|
| 77 |
-
</details>
|
| 78 |
|
| 79 |
---
|
| 80 |
|
| 81 |
-
##
|
| 82 |
|
| 83 |
-
|
| 84 |
|
| 85 |
-
|
| 86 |
-
<summary><b>Critical Issues Fixed</b></summary>
|
| 87 |
-
|
| 88 |
-
| Area | Issue in Official Templates | The Fix |
|
| 89 |
-
|---|---|---|
|
| 90 |
-
| **Qwen 3.8 Support** | Official 3.8-2.4T-A95B crashes if `enable_thinking=false`. | **Restored Fast Mode**. Supports fast non-reasoning mode via kwargs or `<\|think_off\|>`. |
|
| 91 |
-
| **Qwen 3.8 Token Safety** | Official `xhigh` default burns token budgets with zero content returned. | **Safe `medium` Default**. Zero prompt injection unless explicitly requested. |
|
| 92 |
-
| **Qwen 3.8 Regression** | Official 3.8 injects duplicate blank `<think></think>` in chat history. | **Cured Empty Think Poisoning**. Multi-format reasoning extraction. |
|
| 93 |
-
| **Reasoning Control** | Inability to change reasoning effort in chat interfaces. | **Inline Chat Tags**. Full support for `<\|think_low\|>`, `<\|think_medium\|>`, and `<\|think_xhigh\|>`. |
|
| 94 |
-
| **Compatibility** | `llama.cpp --reasoning-preserve` CLI flag compatibility. | **Native Alias Support**. Supports both `preserve_reasoning` and `preserve_thinking`. |
|
| 95 |
-
| **Compatibility** | JSON-string tool arguments (OpenAI / Ollama) crash official templates. | **Universal Tool Parsing**. Safely handles mappings, JSON strings, and scalar args. |
|
| 96 |
-
| **Agentic Loop** | Model aborts turn when combining conversational text and a tool call. | Cured "Empty Think" poisoning and softened imperative system directives. |
|
| 97 |
-
| **Agentic Loop** | Model gets stuck emitting the identical failing tool call. | Added two-tier error escalation to force correction while retaining reasoning. |
|
| 98 |
-
| **Agentic Loop** | Model panics and debates internal rules after fetching data. | Broadened `<think>` instructions to authorize conversational synthesis. |
|
| 99 |
-
| **Agentic Loop** | API returns containing the word "error" trigger false retry loops. | Replaced broad matching with strict structural guards. |
|
| 100 |
-
| **Performance** | Mutated past turns destroy the prefix cache. | Enforced chronological history for a 100% KV Cache hit rate. |
|
| 101 |
-
| **Performance** | Deep Jinja nesting drops `llama.cpp` speed by 80%. | Flattened the AST architecture to maximize throughput. |
|
| 102 |
-
| **Compatibility** | Python-specific filters crash C++ inference engines. | Rewrote all filters to be 100% `minijinja` safe. |
|
| 103 |
-
| **Compatibility** | Qwen-native parsers (like vLLM) crash on JSON formatting. | Maintained canonical Qwen XML format as the default. |
|
| 104 |
-
| **Compatibility** | Older API setups and wrappers crash on native XML. | Added a `tool_call_format="json"` opt-in override. |
|
| 105 |
-
| **Compatibility** | Anthropic `message.thinking` payloads are rejected. | Added native Anthropic reasoning support. |
|
| 106 |
-
| **Stability** | Massive tool data returns blow out the context window. | Added dynamic payload truncation limits. |
|
| 107 |
-
| **Stability** | Mid-conversation system prompts crash the template. | Added native support for arbitrary system and developer messages. |
|
| 108 |
-
| **Edge Cases** | Text duplicates during streaming generation. | Restored canonical spacing to the generation prompt. |
|
| 109 |
-
| **Edge Cases** | Model hallucinates reasoning tags when thinking is disabled. | Injected strict boundaries to force clean reasoning bypass. |
|
| 110 |
-
|
| 111 |
-
</details>
|
| 112 |
|
| 113 |
---
|
| 114 |
|
| 115 |
-
##
|
|
|
|
| 116 |
|
| 117 |
-
|
| 118 |
-
<summary><b>1. Reasoning Effort Steering (Qwen 3.8)</b></summary>
|
| 119 |
-
|
| 120 |
-
Qwen 3.8 has 3 native prompt-steered reasoning levels. You can control this via template kwargs or directly inline in your chat messages:
|
| 121 |
|
| 122 |
-
**
|
| 123 |
-
```
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
}
|
| 127 |
```
|
| 128 |
-
* **`"medium"` (Default / Safe Baseline):** No extra instruction text injected. Preserves 100% Prefix KV Cache parity with v21 and lets the model reason naturally without token-budget traps.
|
| 129 |
-
* **`"xhigh"` (Deep Reasoning):** Injects Qwen's official deep reasoning instruction:
|
| 130 |
-
> *"Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."*
|
| 131 |
-
* **`"low"` (Concise Reasoning):** Injects concise thinking instructions for fast, summary-oriented reasoning:
|
| 132 |
-
> *"Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."*
|
| 133 |
-
|
| 134 |
-
*API Compatibility Aliases:*
|
| 135 |
-
To prevent errors when calling the model through standard API proxies and coding agent harnesses:
|
| 136 |
-
* `"high"`, `"max"`, `"ultracode"`, and `"extreme"` automatically map to `"xhigh"` (supported by OpenAI, Claude Code, Cline, Cursor).
|
| 137 |
-
* `"minimal"` and `"low"` automatically map to `"low"`.
|
| 138 |
-
* `"none"` and `"off"` disable thinking entirely.
|
| 139 |
|
| 140 |
-
**
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
*(Note: When thinking is disabled, reasoning effort instructions are automatically suppressed. On vLLM, prefer request-level `enable_thinking=false` or `reasoning_effort="none"` over `<|think_off|>`; see the vLLM setup notes above).*
|
| 148 |
|
| 149 |
-
|
| 150 |
-
Three variables near the top of `chat_template.jinja` hold the defaults. Edit them before pasting the template:
|
| 151 |
-
* `_default_reasoning_effort = 'medium'` - effort used when no `reasoning_effort` kwarg is passed: `'low'`, `'medium'`, `'xhigh'`, or `'none'` (thinking off). The aliases listed above work too.
|
| 152 |
-
* `enable_thinking = enable_thinking if enable_thinking is defined else true` - change `true` to `false` to default to non-thinking mode.
|
| 153 |
-
* `_tool_format = tool_call_format if tool_call_format is defined else 'xml'` - change `'xml'` to `'json'` for harnesses that need Hermes JSON.
|
| 154 |
|
| 155 |
-
|
| 156 |
|
| 157 |
-
<
|
| 158 |
-
<summary><b>2. KV Cache Preservation (`preserve_reasoning` & `preserve_thinking`)</b></summary>
|
| 159 |
|
| 160 |
-
|
| 161 |
|
| 162 |
-
* On recent `llama.cpp` builds, pass `--reasoning-preserve` directly.
|
| 163 |
-
* Or pass via template kwargs:
|
| 164 |
```json
|
| 165 |
{
|
| 166 |
-
"preserve_thinking":
|
| 167 |
}
|
| 168 |
```
|
|
|
|
| 169 |
|
| 170 |
-
|
| 171 |
|
| 172 |
-
>
|
|
|
|
| 173 |
|
| 174 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 175 |
|
| 176 |
-
|
| 177 |
-
|
|
|
|
|
|
|
| 178 |
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
* **`"json"` (Optional Override):** Forces Hermes JSON format (`{"name": "...", "arguments": {...}}`) for both system prompt instructions and history rendering.
|
| 182 |
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
```
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
```
|
| 190 |
-
*(When opting into JSON format, both argument and tool response truncation are bypassed to avoid corrupting JSON syntax)*.
|
| 191 |
|
| 192 |
</details>
|
| 193 |
|
| 194 |
<details>
|
| 195 |
-
<summary>
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
|
| 203 |
</details>
|
| 204 |
|
| 205 |
---
|
| 206 |
|
| 207 |
-
|
| 208 |
-
<summary><b>Diagnostic Script & Test Suite</b></summary>
|
| 209 |
-
|
| 210 |
-
### Check Active Template on Your Model
|
| 211 |
-
Run the included diagnostic utility on your model folder or GGUF:
|
| 212 |
-
```bash
|
| 213 |
-
python3 scripts/check_applied.py /path/to/your/model
|
| 214 |
-
```
|
| 215 |
|
| 216 |
-
### Running the Test Suite
|
| 217 |
```bash
|
| 218 |
-
python3 scripts/
|
| 219 |
```
|
| 220 |
-
Tests cover 101 automated verification cells including `reasoning_effort` levels, monotonic API mappings, inline chat tags, multi-part and vision payloads, tool call serialization across wire formats, dynamic truncation, error escalation precision, malformed input handling, multi-turn history parsing, prefix KV cache stability, and parity between `chat_template.jinja` and `chat_template_oneline.txt`. Set `QWEN_TEMPLATE_FILE=chat_template_oneline.txt` to run the entire suite against the minified build. Test 101 is a deterministic property fuzzer (`scripts/fuzz_template.py`) asserting nine structural invariants over generated conversations; run it standalone with `python3 scripts/fuzz_template.py --cases 2000`.
|
| 221 |
|
| 222 |
-
</
|
| 223 |
|
| 224 |
---
|
| 225 |
|
| 226 |
-
## Authorship
|
| 227 |
-
|
| 228 |
-
|
|
|
|
|
| 229 |
| Original models | Alibaba Cloud (Qwen team) |
|
| 230 |
-
| Template fixes | [
|
| 231 |
-
| Property fuzzer, error tiering & test suite | [Juan Calderon-Perez](https://huggingface.co/g-a-b-y) (`@g-a-b-y`) |
|
| 232 |
-
| Multi-tool alignment, reasoning fields & docs | [Gabriel Devenyi](https://huggingface.co/gdevenyi) (`@gdevenyi`) |
|
| 233 |
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 234 |
|
| 235 |
## License
|
| 236 |
-
Apache-2.0, inherited from Qwen.
|
| 237 |
-
|
| 238 |
-
---
|
| 239 |
|
| 240 |
-
|
| 241 |
-
<summary>Technical Details of the Critical Fixes</summary>
|
| 242 |
-
|
| 243 |
-
### 1. The "Empty Think" Poisoning and Logic Trap Cure
|
| 244 |
-
Previous templates attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic pattern where the model associated empty thoughts with tools, causing an 80%+ premature turn abort rate. We abolished empty think injection and rewrote the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after thinking. In this release, we also cured official Qwen 3.8's history bug where missing in-content parsers created duplicate blank think tags.
|
| 245 |
-
|
| 246 |
-
### 2. Upfront Pre-Scan for Control Tags & Inline Effort Steering
|
| 247 |
-
In Jinja templates, system prompts are assembled before iterating over message history. We introduce an upfront pre-scan covering raw strings, string lists, and multi-part content lists (`[{'type': 'text', 'text': '...'}]`). This resolves `<|think_off|>`, `<|think_on|>`, `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` states before the system message is built, preventing reasoning instructions from being injected into non-reasoning turns and cleanly stripping all tags during rendering.
|
| 248 |
-
|
| 249 |
-
### 3. KV Cache Safety and Autoregressive Normalization
|
| 250 |
-
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because this template preserves historical thoughts chronologically by default and defaults `reasoning_effort` to `medium` (zero system tokens), rendered history perfectly synchronizes with cached generated tokens. Combined with strict single newline normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn sessions.
|
| 251 |
-
|
| 252 |
-
### 4. Native XML Tool Format and Universal Serialization
|
| 253 |
-
The model was trained with the XML tool format used by Qwen3-Coder. We restored this format natively while bypassing the `|items` crash by handling both mapping dictionaries and JSON strings. This eliminates crashes when standard OpenAI proxies pass stringified arguments.
|
| 254 |
-
|
| 255 |
-
### 5. Two-Tier Agentic Error Escalation
|
| 256 |
-
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter. On the first error, a diagnostic warning is injected. On the second consecutive error, an urgent system warning forces a fundamentally different approach while retaining the reasoning block so the model can plan its correction.
|
| 257 |
-
|
| 258 |
-
### 6. Smart False-Positive Detection
|
| 259 |
-
Instead of broad substring matching that triggers false retry loops on successful database returns containing words like "error", this template utilizes strict structural guards evaluated over the first 120 characters of the payload.
|
| 260 |
-
|
| 261 |
-
Signals are split into two tiers. **Strong signals** are unambiguous structural markers (`"error":`, `Traceback (most recent call last):`, `command not found`, `Exception:`, `fatal:`, nonzero exit codes, `invalid syntax`) and fire unconditionally. **Weak signals** are bare prefixes (`error:`, `err!`) and remain gated by a length ceiling plus shell echo (`$ `) and timing (`took `) exclusions. Code and search results are excluded up front via `throw new`, `console.error`, `logger.error` and similar patterns.
|
| 262 |
-
|
| 263 |
-
Tiering matters because the earlier flat gate suppressed every signal on payloads of 600 characters or more: a routine multi-frame Python traceback exceeds that ceiling, and any output echoing a shell prompt or a timing figure was discarded outright. Strong signals now escalate regardless of payload size.
|
| 264 |
-
|
| 265 |
-
### 7. minijinja Compatibility Constraints
|
| 266 |
-
Python-only Jinja2 features crash or misbehave on `minijinja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 267 |
-
* `content | replace('<|think_on|>', '')` became `content.split('<|think_on|>') | join('')` (fixes a bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 268 |
-
* `| items` became `mapping.items()` iteration, which `minijinja` implements natively.
|
| 269 |
-
* `loop.previtem` became explicit array indexing.
|
| 270 |
-
* `map('string')` became `join('|')`.
|
| 271 |
-
* `| first` became `'$ ' in content`.
|
| 272 |
-
|
| 273 |
-
### 8. AST Flattening for C++ Throughput
|
| 274 |
-
Deeply nested Jinja loops and macros create severe parsing bottlenecks in C++ inference engines. We flattened the AST architecture, effectively curing an 80% inference throughput drop on `llama.cpp` by streamlining how `ns_state` tracking and historical rendering loops are evaluated.
|
| 275 |
-
|
| 276 |
-
### 9. Dynamic Payload Truncation
|
| 277 |
-
Massive API or database returns can instantly blow out a model's context window. We implemented `max_tool_arg_chars` and `max_tool_response_chars` limiters that safely slice oversized payloads. Argument truncation covers both mapping arguments and serialized JSON string arguments, so the OpenAI proxy wire format is limited on the same terms as native dictionaries. Both limiters are automatically disabled when `tool_call_format="json"` is active, as slicing serialized JSON structurally corrupts the data and crashes downstream parsers.
|
| 278 |
-
|
| 279 |
-
### 10. Reasoning Bypass Hallucination Mitigation
|
| 280 |
-
When thinking is disabled, Qwen models often hallucinate reasoning tags due to their training bias. We injected a safe boundary and adjusted the `<IMPORTANT>` system block to remove explicit mentions of `</think>` during tool instructions. This stops the model from hallucinating closing tags when calling tools in a no-reasoning state.
|
| 281 |
-
|
| 282 |
-
</details>
|
| 283 |
-
|
| 284 |
-
---
|
| 285 |
-
|
| 286 |
-
<details>
|
| 287 |
-
<summary>Update History & Changelog</summary>
|
| 288 |
-
|
| 289 |
-
> **2026-08-24 Update (v22.4): Multi-Tool Token Parity, Extended Reasoning Formats, and UI Default Knobs.**
|
| 290 |
-
> 1. **Parallel Tool Call Token Parity:** Consecutive `<tool_call>` blocks in assistant history are now separated by a single newline (`\n`) instead of double newlines (`\n\n`), restoring exact token alignment with official Qwen model generation and preventing prefix KV cache divergence during multi-tool turns (#87).
|
| 291 |
-
> 2. **vLLM & Responses API Reasoning Support:** Added native support for `message.reasoning` (used by vLLM OpenAI-compatible endpoints and Responses API schemas) alongside `reasoning_content` and `thinking` (#88).
|
| 292 |
-
> 3. **Top-Level `_default_reasoning_effort` Knob:** Added an explicit `_default_reasoning_effort` configuration variable at the top of the template and removed dead assignment code, allowing LM Studio and WebUI users who cannot pass kwargs to change the default reasoning level cleanly without template errors (#91).
|
| 293 |
-
> 4. **Documented Sticky Inline Tags & UI Configuration:** Documented inline tag persistence across turns, string-argument fallbacks in XML, and clarified vLLM launch commands (`--reasoning-parser qwen3` alongside `--tool-call-parser qwen3_xml`) (#89, #90).
|
| 294 |
-
> 5. **Expanded Test Suite (101 -> 102):** Added automated test coverage for `message.reasoning` extraction, `_default_reasoning_effort` knob behavior, and updated parallel tool call assertions.
|
| 295 |
-
>
|
| 296 |
-
> *(Credits: Huge thanks to Gabriel Devenyi (@gdevenyi) for their contributions to the v22.4 release!)*
|
| 297 |
-
>
|
| 298 |
-
> **2026-08-20 Update (v22.3): Error Detection Tiering, Reasoning De-duplication, and Test Suite Expansion.**
|
| 299 |
-
> 1. **Two-Tier Error Signals:** Structural error markers now escalate regardless of payload size. The previous flat 600-character ceiling plus whole-body `$ ` and `took ` exclusions silently suppressed warnings on ordinary multi-frame tracebacks and on any shell transcript that echoed its command. Successful exit-code reports (`Exit code: 0`) and JSON success envelopes (`"error": null` / `false` / `""`) are excluded from the strong tier so success payloads do not escalate.
|
| 300 |
-
> 2. **Reasoning De-duplication:** When `reasoning_content` or `thinking` is supplied, a leading think block in `content` is stripped so clients that populate both no longer emit two consecutive think blocks. Literal tags later in the answer (for example inside code fences) are preserved verbatim; full heuristic extraction still applies when no explicit reasoning field is present.
|
| 301 |
-
> 3. **Single-Line Think Blocks:** `<think>...</think>` written without surrounding newlines is now extracted instead of leaking raw tags into rendered history.
|
| 302 |
-
> 4. **Complete Argument Serialization:** Scalar and list tool arguments are serialized via `| tojson` rather than silently dropped, in both XML and JSON wire formats.
|
| 303 |
-
> 5. **Consistent Truncation:** `max_tool_arg_chars` now applies to serialized JSON string arguments, and `max_tool_response_chars` is bypassed under `tool_call_format="json"` as documented.
|
| 304 |
-
> 6. **KV Cache Documentation:** Documented that `preserve_thinking: false` rewrites rendered history at each new user turn and therefore voids the prefix cache guarantee, which applies to the default setting.
|
| 305 |
-
> 7. **Test Suite Expansion (44 -> 101):** Added coverage for control tag and alias completeness, malformed input and exception paths, vision payloads, agentic wire-format shapes, reasoning extraction variants, error detection precision, prefix KV cache stability, and jinja/oneline build parity.
|
| 306 |
-
> 8. **Property-Based Fuzz Harness:** Added `scripts/fuzz_template.py`, a deterministic conversation generator asserting nine structural invariants (render success, jinja/oneline parity, token balance, verbatim content preservation, XML parameter fidelity, JSON tool-call validity, warning precision, prefix KV stability, no-think prefill). Prefix stability is checked at generation boundaries: merged system blocks and consecutive tool-result batches are atomic, since the model only generates after a full batch is appended.
|
| 307 |
-
>
|
| 308 |
-
> *(Credits: Huge thanks to Juan Calderon-Perez (@g-a-b-y) for their massive contributions to the v22.3 release, including the property-based fuzz harness, the two-tier error signal architecture, and extensive test coverage!)*
|
| 309 |
-
>
|
| 310 |
-
> **2026-08-19 Update (v22.2): Extended Effort Aliases, String Argument Safety, and Error Disambiguation.**
|
| 311 |
-
> 1. **Universal Tool Argument Handling:** Hardened XML tool argument parsing to safely handle both mappings and serialized JSON strings in assistant history without syntax crashes or KV cache mutation.
|
| 312 |
-
> 2. **Reasoning Effort Aliases:** Added `"ultracode"` and `"extreme"` mappings to `'xhigh'`, with inline `<|think_ultracode|>` and `<|think_extreme|>` support (#78).
|
| 313 |
-
> 3. **Multi-System Message Merging:** Merged consecutive leading `system` and `developer` messages into a single unified system turn with `\n\n`.
|
| 314 |
-
> 4. **Grep / Search Error Disambiguation:** Eliminated false-positive tool error warnings on code search results containing patterns like `throw new Error` or `console.error` (#66).
|
| 315 |
-
> 5. **Safe XML Primitive Serialization:** Boolean and null XML parameter values now serialize to `true`, `false`, and `null` via `| tojson`.
|
| 316 |
-
> 6. **Zero-Crash Resilience:** Maintained full resilience without fatal `raise_exception` aborts across all runtime engines.
|
| 317 |
-
|
| 318 |
-
> **2026-08-16 Update (v22.1): The Qwen 3.8 Update (Bounded Reasoning Defaults, Inline Chat Tags, and Diagnostic Utility).**
|
| 319 |
-
> 1. **Full Qwen 3.8 Support:** Single drop-in template covering all Qwen 3.5, 3.6, and 3.8 model variants (`Qwen3.8-2.4T-A95B`, `Qwen3.8-27B`).
|
| 320 |
-
> 2. **Default `reasoning_effort` to `medium`:** Replaced the unsafe `xhigh` default with `medium` (zero injected tokens). Eliminates the runaway reasoning token-burn failure where models explore branches until hitting `max_tokens` with empty content (#72).
|
| 321 |
-
> 3. **Inline Chat Tags:** Added support for `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` tags directly inside chat messages (#70).
|
| 322 |
-
> 4. **API Mapping & Aliasing:** Added case-insensitive alias support for client and serving runtimes (`high`/`max` -> `xhigh`, `minimal` -> `low`, `none` -> thinking off).
|
| 323 |
-
> 5. **Sequential Control Tag Stripping:** All 7 control tags are cleanly stripped across system, user, and multi-part content without prompt leakage.
|
| 324 |
-
> 6. **Cured Official 3.8 Empty Think Bug:** Fixed a regression in the official Qwen 3.8 template where removing `<think>` extraction caused blank `<think></think>` blocks to be prepended to real thoughts in chat history.
|
| 325 |
-
> 7. **Restored Fast Mode:** Replaced official 3.8's hard exception on `enable_thinking=false`, restoring full user freedom to disable reasoning via kwargs or `<|think_off|>`.
|
| 326 |
-
> 8. **Universal Tool Arguments:** Hardened tool call parsing to handle both dictionary structures and JSON-serialized strings without crashing.
|
| 327 |
-
> 9. **Diagnostic Utility:** Added `scripts/check_applied.py` to inspect model folders and GGUFs for template consistency.
|
| 328 |
-
|
| 329 |
-
> **2026-08-13 Update (v22): Qwen 3.8 Support, Reasoning Effort Controls, and Engine Hardening.**
|
| 330 |
-
|
| 331 |
-
> **2026-07-02 Update (v21.3): Optional JSON Tool Format Kwarg.** Added an optional `tool_call_format="json"` override for `chat_template_kwargs`.
|
| 332 |
-
|
| 333 |
-
> **2026-07-02 Update (v21.2): Reasoning Bypass Hallucination Fix.** Adjusted `<IMPORTANT>` block instructions to remove explicit mentions of `</think>` during tool definitions.
|
| 334 |
-
|
| 335 |
-
> **2026-07-02 Update (v21.1): Reliability Overhaul & XML Revert.** Reverted to native XML format for vLLM `qwen3_coder` compatibility and restored `preserve_thinking` default to `true`.
|
| 336 |
-
|
| 337 |
-
</details>
|
|
|
|
| 6 |
- qwen
|
| 7 |
- qwen3.5
|
| 8 |
- qwen3.6
|
|
|
|
| 9 |
- lm-studio
|
| 10 |
- mlx
|
| 11 |
- llama.cpp
|
|
|
|
| 14 |
- thinking
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# Fixed jinja chat templates for Qwen 3.5 & 3.6 (v20)
|
| 18 |
|
| 19 |
+
<details open>
|
| 20 |
+
<summary><b>Update History & Changelog (v20)</b></summary>
|
| 21 |
|
| 22 |
+
> **2026-06-05 Update (v20): The Architect Patch.** A monumental structural overhaul targeting deep agentic loops and C++ inference engine compatibility. (1) **Minja AST Flattening:** Dramatically optimized Jinja nesting depths to resolve severe parsing bottlenecks that were dropping inference throughput by 80% on `llama.cpp`. (2) **Minja Replace Bug Fix (Hotfix):** Bypassed a severe C++ parsing bug in `llama.cpp` where using the `replace` filter at index 0 of a user prompt silently dropped the entire text payload. Inline thinking toggles now use `split` and `join` for robust stripping. (3) **Auto-disable Thinking:** Introduced `auto_disable_thinking_with_tools` kwarg (default `false`) that allows users to instantly shut off reasoning blocks during tool use. (4) **Deep Agent Fallbacks:** Resolved exceptions triggered by mid-conversation system prompts or loops lacking human `user` messages. (5) **Payload Truncation:** Implemented `max_tool_arg_chars` and `max_tool_response_chars` configurations to definitively stop context-window explosions from massive data returns. *(Huge thanks to `barubary` / `spiritbuun` for their contributions to these C++ architecture optimizations!)*
|
| 23 |
|
| 24 |
+
</details>
|
| 25 |
|
| 26 |
+
<details>
|
| 27 |
+
<summary><b>Update History & Changelog (v19)</b></summary>
|
| 28 |
|
| 29 |
+
> **2026-05-18 Update (v19): The Agentic Loop Cure.** (1) **Abolished "Empty Think" Poisoning:** Rewrote the AST history rendering to completely remove the injection of empty `<think>\n</think>` blocks. This cures a severe in-context learning bias where the model assumed tools could only be called if it didn't think first, which was causing 80%+ of premature `<|im_end|>` turn aborts. (2) **System Prompt Logic Trap Removed:** Softened the absolute tool mandate in the `<IMPORTANT>` block and restored Universal Synthesis instructions. The model is now explicitly permitted to transition from `</think>` to a conversational answer without panicking. (3) **True 100% KV Cache & Amnesia Fix:** `preserve_thinking` now defaults to `true`. Past thoughts are retained chronologically, permanently curing "amnesia stalls" during multi-step tool loops while mathematically guaranteeing 100% KV Cache prefix matching out-of-the-box.
|
| 30 |
|
| 31 |
+
</details>
|
| 32 |
|
| 33 |
+
<details>
|
| 34 |
+
<summary><b>Update History & Changelog (v11-v18)</b></summary>
|
| 35 |
+
|
| 36 |
+
> **2026-05-16 Update (v18): Stability & Precision Patch.** (1) **Bulletproof False-Positive Detection:** Shifted agentic error detection from broad substring matching to strict structural formats (e.g., `"error":`, `Exception:`, `Traceback`), completely curing false-positive retry loops when successful JSON returns simply contain the word "error" or "fail". (2) **Legacy Engine Compatibility:** Replaced `loop.previtem` with explicit array indexing, fixing AST crashes on older `llama.cpp` and `minijinja` builds that do not track loop state items. (3) **True Whitespace Normalization:** Fixed a bug where reasoning bypasses and hallucinated tag recovery stacked hidden multi-newlines (`\n\n\n`), strictly fulfilling the 100% KV Cache hit rate claim for all edge cases. (4) **Code Cleanup:** Removed dead conditional branches during XML tool parsing.
|
| 37 |
+
>
|
| 38 |
+
> **2026-05-15 Update (v17):** Major architecture overhaul resolving edge cases in agentic tooling and KV Cache. (1) **Unified Template:** Consolidated Qwen 3.5 and Qwen 3.6 into a single `chat_template.jinja` file that handles all variants seamlessly. (2) **Fixed "Mutually Exclusive" Stopping Bug:** Changed the history-pruning logic from wiping the entire turn to safely array-slicing out just the raw tool tags (`content.split('<tool_call>')[0]`). This preserves the conversational text in the history, which cures the bug where the model would artificially abort its turn (output `<|im_end|>`) when it wanted to talk and use a tool simultaneously. (3) **100% KV Cache Hit Rate Restoration:** Fully normalized internal whitespace logic (`\n\n` -> `\n`) around think blocks and tool calls to exactly match the model's native autoregressive generation spacing. This perfectly synchronizes the template's rendered history with the cached generated tokens, completely eliminating the severe cache invalidation and full-prompt re-processing issues present in v16.
|
| 39 |
+
>
|
| 40 |
+
> **2026-05-14 Update (v16):** Four-part fix addressing community-reported regressions. (1) **Native XML tool format:** reverted from JSON back to the native `<function=name>` / `<parameter=x>` format the model was trained on, restoring full compatibility with vLLM's `qwen3_coder` parser and all inference engines that implement the Qwen tool protocol. (2) **`--reasoning off` respected in error paths:** when thinking is disabled (`enable_thinking=false` / `--reasoning off`), the error escalation directives are now injected as plain text without opening any `<think>` block, preventing degenerate prompts in no-reasoning sessions. (3) **Smarter false-positive detection:** short shell command results (starting with `$ `) and search results with timing footers (`Took X.Xs`) are now correctly excluded from error detection, preventing tool-retry loops when commands succeed but their output happens to contain the word `error`. (4) **`consecutive_failures` counter no longer resets on assistant messages**, allowing Tier 2 escalation to actually fire across multi-turn tool retry chains.
|
| 41 |
+
>
|
| 42 |
+
> **2026-05-13 Update (v15):** Three-part fix for agentic tool-loop failures. (1) **Two-tier error escalation:** replaced the brittle backwards-lookahead error detection with a fully forward-tracking `last_tool_failed` + `consecutive_failures` counter. On the first error the generation prompt is pre-seeded with a correction directive inside `<think>`; on the 2nd+ consecutive error the think block is bypassed and an out-of-band directive forces an immediate corrected action. (2) **Length-gated detection:** error signals are only read from short tool responses (< 500 chars), preventing false positives when reading code files containing `error`, `exception`, etc. in legitimate content. (3) **Static system prompt:** tool instructions are now fully unconditional, permanently eliminating the KV cache invalidation vector introduced in v14.
|
| 43 |
+
>
|
| 44 |
+
> **2026-05-12 Update (v14):** Cured tool amnesia loops and post-tool overthinking friction! Implemented **Smart Loop Preservation** to dynamically scan subsequent tool returns for error markers and conditionally preserve historical reasoning context during active tool failures. Broadened the system instruction scope to define `<think>` as a dual-purpose planning **or synthesis** space, completely eliminating indecisiveness post-tool retrieval.
|
| 45 |
+
>
|
| 46 |
+
> **2026-05-11 Update (v13):** Radical simplification and compatibility overhaul! Reverted tool schemas and assistant output formatting to standard JSON to natively fix downstream MCP parser crashes and C++ implicit enum coercion bugs. Removed the `ns_scan` history loop to permanently fix KV cache invalidation mid-conversation. Replaced global string replacement for hallucinated tags with a C++ safe, localized array-slicing method to prevent data-corruption on user code blocks.
|
| 47 |
+
>
|
| 48 |
+
> **2026-05-10 Update (v12):** Fixed agent stalls, parameter data-loss, and hallucination bugs! Restored dynamic tool instructions and the `<IMPORTANT>` formatting reminder block to stop grammar parser crashes.
|
| 49 |
+
>
|
| 50 |
+
> **2026-05-10 Update (v11):** Fixed agent looping and overthinking! Re-implemented `preserve_thinking` kwarg to properly strip reasoning blocks from history by default, and restored the reasoning bypass (`<think>\n\n</think>\n\n`).
|
| 51 |
+
|
| 52 |
+
</details>
|
| 53 |
+
|
| 54 |
+
This is a drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, and fatal agentic stalling in the official Qwen chat templates.
|
| 55 |
+
|
| 56 |
+
It is tested to work across LM Studio, llama.cpp, vLLM, MLX, oMLX, and any engine that supports HuggingFace Jinja templates.
|
| 57 |
|
| 58 |
---
|
| 59 |
|
| 60 |
+
## Why you need this
|
| 61 |
+
The official Qwen templates contain restrictions and Python-specific Jinja logic that break usage on many inference engines and agent frameworks.
|
| 62 |
+
|
| 63 |
+
Here are the critical issues this template fixes:
|
| 64 |
+
|
| 65 |
+
| Category | Problem | Impact | Fix |
|
| 66 |
+
|---|---|---|---|
|
| 67 |
+
| **Agentic Loop** | **Premature Stalls (Stopping Bug)** | Model aborts its turn (`<\|im_end\|>`) when trying to combine conversation and a tool call. | Resolved the System Prompt logic trap and cured "Empty Think" poisoning (v19). |
|
| 68 |
+
| **Agentic Loop** | **Retry Stall & Reasoning Spiral** | Model correctly diagnoses a tool error but repeatedly emits the identical failing `<tool_call>`. | Two-tier escalation: seeds `<think>` with correction directive; injects urgent out-of-band directive. |
|
| 69 |
+
| **Agentic Loop** | **Post-Tool Overthinking** | Forced `<think>` block prefilling causes model to panic and debate internal rules after fetching data. | Broadened instructions to define `<think>` as a dual-purpose space for planning *or synthesis*. |
|
| 70 |
+
| **Agentic Loop** | **False-Positive Error Detection** | Short successful API/JSON returns containing the word `error` trigger false retry loops. | Strict structural guards look for exact system failures (`"error":`, `Traceback`, etc.) instead of broad words (v18). |
|
| 71 |
+
| **Performance** | **KV Cache Invalidation** | History pruning dynamically mutates past turns, causing full prompt re-processing every turn. | `preserve_thinking` defaults to `true`, maintaining strict chronological rendering for a 100% KV cache hit rate (v19). |
|
| 72 |
+
| **Performance** | **Empty Think Poisoning** | Stripped past turns leave behind empty `<think></think>` tags, tricking the model into a severe in-context learning bias. | Template completely abolishes the injection of empty think blocks (v19). |
|
| 73 |
+
| **Compatibility** | **Legacy Engine Crashes** | Older C++ parsing engines crash when evaluating `loop.previtem`. | Uses strict chronological array indexing universally supported by all Jinja iterations (v18). |
|
| 74 |
+
| **Compatibility** | **Wrong Tool Call Format** | Qwen-native parsers (like vLLM's `qwen3_coder`) expect XML `<function=name>`. JSON format breaks them. | Restored native XML format while keeping C++ safety. |
|
| 75 |
+
| **Compatibility** | **Jinja C++ Crashes** | Python-specific filters (`map`, `first` on strings) crash on `minijinja`. | All filters replaced with universally compatible equivalents. |
|
| 76 |
+
| **Stability** | **Mid-Conversation System Crash** | Frameworks injecting mid-conversation steering instructions trigger a hard crash. | Native, chronological rendering for system messages anywhere in the history. |
|
| 77 |
+
| **Stability** | **No-User-Query Crash** | `raise_exception` crashes agentic loops or system-only contexts. | Graceful fallback implemented. |
|
| 78 |
+
| **Stability** | **Unclosed Thinking Before Tool** | Model calls a tool without closing its reasoning, bleeding XML tags into tool parsers. | Auto-injects closing tags before tool boundaries securely. |
|
| 79 |
+
| **Edge Cases** | **`developer` Role Rejected** | Modern APIs send the developer role; the official template rejects it. | Added full support for `"developer"`. |
|
| 80 |
+
| **Edge Cases** | **`--reasoning off` Ignored** | When thinking is disabled, tool error escalation still opened a `<think>` block, corrupting the prompt. | Error escalation branches now fully respect `enable_thinking=false`. |
|
| 81 |
+
| **Edge Cases** | **Reasoning Bypass Hallucinations** | When thinking is disabled, Qwen models inherently hallucinate reasoning tags anyway. | Injects a safe boundary to successfully force reasoning bypass without stacking newlines (v18). |
|
| 82 |
|
| 83 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
|
| 85 |
+
## Quick install
|
| 86 |
+
|
| 87 |
+
Choose your environment and update the template:
|
| 88 |
|
| 89 |
### LM Studio
|
| 90 |
+
1. Open your Qwen model in the right-side panel.
|
| 91 |
2. Scroll down to **Prompt Template**.
|
| 92 |
3. Replace the template with the contents of `chat_template.jinja`.
|
| 93 |
4. Click **Save**.
|
| 94 |
|
| 95 |
+
### llama.cpp / koboldcpp
|
|
|
|
| 96 |
```bash
|
| 97 |
+
--jinja --chat-template-file chat_template.jinja
|
| 98 |
```
|
|
|
|
| 99 |
|
| 100 |
+
### vLLM
|
| 101 |
+
Replace the `"chat_template"` string in your `tokenizer_config.json` with the raw file contents. Use the `qwen3_coder` tool parser:
|
| 102 |
+
```bash
|
| 103 |
+
--tool-call-parser qwen3_coder
|
| 104 |
+
```
|
| 105 |
|
| 106 |
+
### oMLX
|
| 107 |
+
Overwrite `chat_template.jinja` in your local model directory. Load with `--jinja`. Remove any `chat_template_kwargs` overrides because the template handles everything internally.
|
|
|
|
|
|
|
| 108 |
|
| 109 |
---
|
| 110 |
|
| 111 |
+
## Which file do I use?
|
| 112 |
|
| 113 |
+
Both Qwen 3.5 and Qwen 3.6 variants (including 35B, 32B, 27B, and 14B parameters) have been consolidated. You only need the single `chat_template.jinja` file at the root of the repository.
|
| 114 |
|
| 115 |
+
One-line versions (`chat_template_oneline.txt`) are pre-minified for engines that require a single-line template string.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
|
| 117 |
---
|
| 118 |
|
| 119 |
+
## The thinking toggle
|
| 120 |
+
You can control the model reasoning behavior. Insert `<|think_on|>` or `<|think_off|>` anywhere in your system or user prompt.
|
| 121 |
|
| 122 |
+
The template natively intercepts the tag, removes it from the final context so the model never sees it, and flips the reasoning mode instantly.
|
|
|
|
|
|
|
|
|
|
| 123 |
|
| 124 |
+
**Fast answer, no reasoning:**
|
| 125 |
+
```text
|
| 126 |
+
System: You are a coding assistant. <|think_off|>
|
| 127 |
+
User: What's 2+2?
|
|
|
|
| 128 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
|
| 130 |
+
**Deep reasoning:**
|
| 131 |
+
```text
|
| 132 |
+
System: You are a coding assistant. <|think_on|>
|
| 133 |
+
User: Implement a red-black tree in Rust.
|
| 134 |
+
```
|
| 135 |
+
*(The tag syntax uses Qwen's control-token delimiters to guarantee it will never collide with legitimate text or file paths, unlike earlier community templates that used `/think`)*
|
|
|
|
|
|
|
| 136 |
|
| 137 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
| 138 |
|
| 139 |
+
## Token Saving: Stripping past thoughts
|
| 140 |
|
| 141 |
+
By default in v19, this template **preserves** all past `<think>` blocks in the chat history. This is intentional: it prevents the model from suffering "amnesia stalls" during complex, multi-step agentic loops, and it mathematically guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
|
|
|
| 142 |
|
| 143 |
+
However, if you are running constrained hardware and need to save context tokens, you can explicitly disable this feature in your engine's template kwargs to automatically strip past thoughts:
|
| 144 |
|
|
|
|
|
|
|
| 145 |
```json
|
| 146 |
{
|
| 147 |
+
"preserve_thinking": false
|
| 148 |
}
|
| 149 |
```
|
| 150 |
+
*(Note: Setting this to false will naturally reduce your KV Cache hit rate during multi-turn chats, as the prompt string will dynamically mutate).*
|
| 151 |
|
| 152 |
+
---
|
| 153 |
|
| 154 |
+
<details>
|
| 155 |
+
<summary>Technical Details of the Critical Fixes</summary>
|
| 156 |
|
| 157 |
+
### 1. The "Empty Think" Poisoning & Logic Trap Cure (v19)
|
| 158 |
+
Previous versions attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic in-context learning pattern: the model associated empty thoughts with tools, and full thoughts with forbidden conversational text, causing an 80%+ premature `<|im_end|>` stalling rate. v19 abolishes empty think injection and rewrites the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after a thought block.
|
| 159 |
+
|
| 160 |
+
### 2. KV Cache Safety & Autoregressive Normalization (v18/v19)
|
| 161 |
+
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because v19 now preserves historical thoughts chronologically by default, the rendered history perfectly synchronizes with the cached generated tokens. Combined with strict single `\n` normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn loops.
|
| 162 |
+
|
| 163 |
+
### 3. Native XML Tool Call Format (v16)
|
| 164 |
+
The model was trained with the XML-based tool call format used by Qwen3-Coder:
|
| 165 |
+
```xml
|
| 166 |
+
<tool_call>
|
| 167 |
+
<function=tool_name>
|
| 168 |
+
<parameter=param_name>
|
| 169 |
+
value
|
| 170 |
+
</parameter>
|
| 171 |
+
</function>
|
| 172 |
+
</tool_call>
|
| 173 |
+
```
|
| 174 |
+
v16 restored this format natively, making it compatible with all parsers while bypassing the `|items` crash by using C++ safe key iteration (`for args_name in tool_call.arguments`).
|
| 175 |
|
| 176 |
+
### 4. Two-Tier Agentic Error Escalation (v15)
|
| 177 |
+
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter:
|
| 178 |
+
- **Tier 1 (1st error):** Generation prompt prefix changes to seed reasoning at a different token position, breaking the cached attractor state.
|
| 179 |
+
- **Tier 2 (2nd+ consecutive errors):** Think block bypassed entirely. An urgent out-of-band directive forces an immediate corrected action wrapped safely within the user `tool_response` block.
|
| 180 |
|
| 181 |
+
### 5. Smart False-Positive Detection (v18)
|
| 182 |
+
Instead of broad substring matching that triggers false retry-loops on successful database returns containing words like "error", v18 utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell-echo exclusions (`$ `).
|
|
|
|
| 183 |
|
| 184 |
+
### 6. minijinja Compatibility Constraints (v18/v20)
|
| 185 |
+
Python-only Jinja2 features crash or misbehave on `minijinja`/`minja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 186 |
+
- `content | replace('<|think_on|>', '')` -> `content.split('<|think_on|>') | join('')` (Fixes a severe bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 187 |
+
- `\| items` -> `for key in mapping`
|
| 188 |
+
- `loop.previtem` -> `messages[loop.index0 - 1]`
|
| 189 |
+
- `map('string')` -> `join('|')`
|
| 190 |
+
- `\| first` -> `'$ ' in content`
|
|
|
|
| 191 |
|
| 192 |
</details>
|
| 193 |
|
| 194 |
<details>
|
| 195 |
+
<summary>Comparison Matrix: Official vs Fixed vs Community</summary>
|
| 196 |
+
|
| 197 |
+
| Feature | Official Qwen Templates | LuffyTheFox | mod-ellary | Pneuny | **This Fixed Template (v19)** |
|
| 198 |
+
|---------|----------|-------------|------------|--------|----------------|
|
| 199 |
+
| Tool call format | XML (native) | JSON | JSON | JSON | **XML (native, qwen3_coder compatible)** |
|
| 200 |
+
| Tool arguments | Fails (`\|items`) | Fixed | Missing | Fixed | **Fixed (C++ safe XML)** |
|
| 201 |
+
| Premature Stalls (Stopping Bug) | Stalls | Stalls | Stalls | Stalls | **Fixed via Logic Trap / Poisoning removal (v19)** |
|
| 202 |
+
| Agentic Retry Stall & Reasoning Spiral | Stalls | Stalls | Stalls | Stalls | **Two-tier escalation system** |
|
| 203 |
+
| False-positive tool errors | N/A | N/A | N/A | N/A | **Guarded (Strict structural matching)** |
|
| 204 |
+
| Post-Tool Overthinking | Spams/Stalls | Broken | Broken | Broken | **Universal Synthesis** |
|
| 205 |
+
| `--reasoning off` on tool errors | N/A | N/A | N/A | N/A | **Fully respected** |
|
| 206 |
+
| `developer` role | Missing | Missing | Missing | Missing | **Added** |
|
| 207 |
+
| Thinking toggle | None | None | `/think` (system only) | None | **`<\|think_off\|>` anywhere** |
|
| 208 |
+
| Empty think in history | Spams empty blocks | Broken | Tags omitted | Broken | **Abolished completely (v19)** |
|
| 209 |
+
| KV prefix caching | Breaks on dynamic history | Breaks | Breaks | Breaks | **100% stable out-of-the-box (v19)** |
|
| 210 |
+
| Mid-conversation system | Crashes | Crashes | Crashes | Crashes | **Fixed** |
|
| 211 |
+
| No-user-query crash | Crashes | Crashes | Crashes | Crashes | **Graceful fallback** |
|
| 212 |
+
| Legacy AST support | Fails (`previtem`) | Fails | Fails | Fails | **Fixed (`index0`)** |
|
| 213 |
+
| `</thinking>` hallucination | Fails | N/A | N/A | N/A | **Detected and safely trimmed** |
|
| 214 |
|
| 215 |
</details>
|
| 216 |
|
| 217 |
---
|
| 218 |
|
| 219 |
+
## Running the test suite
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 220 |
|
|
|
|
| 221 |
```bash
|
| 222 |
+
python3 scripts/test_v20.py
|
| 223 |
```
|
|
|
|
| 224 |
|
| 225 |
+
Tests cover: `auto_disable_thinking_with_tools`, payload truncation logic, parallel tool spacing, mid-conversation system rendering, deep agent loop fallback, XML tool format, `<|think_off|>` / `<|think_on|>` inline overrides, and all legacy v19 regression tests.
|
| 226 |
|
| 227 |
---
|
| 228 |
|
| 229 |
+
## Authorship
|
| 230 |
+
|
| 231 |
+
| Role | Author |
|
| 232 |
+
|------|--------|
|
| 233 |
| Original models | Alibaba Cloud (Qwen team) |
|
| 234 |
+
| Template fixes | [froggeric](https://huggingface.co/froggeric) |
|
|
|
|
|
|
|
| 235 |
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 236 |
|
| 237 |
## License
|
|
|
|
|
|
|
|
|
|
| 238 |
|
| 239 |
+
Apache-2.0, inherited from Qwen.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/README-v20.md
DELETED
|
@@ -1,239 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
tags:
|
| 4 |
-
- jinja
|
| 5 |
-
- chat-template
|
| 6 |
-
- qwen
|
| 7 |
-
- qwen3.5
|
| 8 |
-
- qwen3.6
|
| 9 |
-
- lm-studio
|
| 10 |
-
- mlx
|
| 11 |
-
- llama.cpp
|
| 12 |
-
- vllm
|
| 13 |
-
- tool-calling
|
| 14 |
-
- thinking
|
| 15 |
-
---
|
| 16 |
-
|
| 17 |
-
# Fixed jinja chat templates for Qwen 3.5 & 3.6 (v20)
|
| 18 |
-
|
| 19 |
-
<details open>
|
| 20 |
-
<summary><b>Update History & Changelog (v20)</b></summary>
|
| 21 |
-
|
| 22 |
-
> **2026-06-05 Update (v20): The Architect Patch.** A monumental structural overhaul targeting deep agentic loops and C++ inference engine compatibility. (1) **Minja AST Flattening:** Dramatically optimized Jinja nesting depths to resolve severe parsing bottlenecks that were dropping inference throughput by 80% on `llama.cpp`. (2) **Minja Replace Bug Fix (Hotfix):** Bypassed a severe C++ parsing bug in `llama.cpp` where using the `replace` filter at index 0 of a user prompt silently dropped the entire text payload. Inline thinking toggles now use `split` and `join` for robust stripping. (3) **Auto-disable Thinking:** Introduced `auto_disable_thinking_with_tools` kwarg (default `false`) that allows users to instantly shut off reasoning blocks during tool use. (4) **Deep Agent Fallbacks:** Resolved exceptions triggered by mid-conversation system prompts or loops lacking human `user` messages. (5) **Payload Truncation:** Implemented `max_tool_arg_chars` and `max_tool_response_chars` configurations to definitively stop context-window explosions from massive data returns. *(Huge thanks to `barubary` / `spiritbuun` for their contributions to these C++ architecture optimizations!)*
|
| 23 |
-
|
| 24 |
-
</details>
|
| 25 |
-
|
| 26 |
-
<details>
|
| 27 |
-
<summary><b>Update History & Changelog (v19)</b></summary>
|
| 28 |
-
|
| 29 |
-
> **2026-05-18 Update (v19): The Agentic Loop Cure.** (1) **Abolished "Empty Think" Poisoning:** Rewrote the AST history rendering to completely remove the injection of empty `<think>\n</think>` blocks. This cures a severe in-context learning bias where the model assumed tools could only be called if it didn't think first, which was causing 80%+ of premature `<|im_end|>` turn aborts. (2) **System Prompt Logic Trap Removed:** Softened the absolute tool mandate in the `<IMPORTANT>` block and restored Universal Synthesis instructions. The model is now explicitly permitted to transition from `</think>` to a conversational answer without panicking. (3) **True 100% KV Cache & Amnesia Fix:** `preserve_thinking` now defaults to `true`. Past thoughts are retained chronologically, permanently curing "amnesia stalls" during multi-step tool loops while mathematically guaranteeing 100% KV Cache prefix matching out-of-the-box.
|
| 30 |
-
|
| 31 |
-
</details>
|
| 32 |
-
|
| 33 |
-
<details>
|
| 34 |
-
<summary><b>Update History & Changelog (v11-v18)</b></summary>
|
| 35 |
-
|
| 36 |
-
> **2026-05-16 Update (v18): Stability & Precision Patch.** (1) **Bulletproof False-Positive Detection:** Shifted agentic error detection from broad substring matching to strict structural formats (e.g., `"error":`, `Exception:`, `Traceback`), completely curing false-positive retry loops when successful JSON returns simply contain the word "error" or "fail". (2) **Legacy Engine Compatibility:** Replaced `loop.previtem` with explicit array indexing, fixing AST crashes on older `llama.cpp` and `minijinja` builds that do not track loop state items. (3) **True Whitespace Normalization:** Fixed a bug where reasoning bypasses and hallucinated tag recovery stacked hidden multi-newlines (`\n\n\n`), strictly fulfilling the 100% KV Cache hit rate claim for all edge cases. (4) **Code Cleanup:** Removed dead conditional branches during XML tool parsing.
|
| 37 |
-
>
|
| 38 |
-
> **2026-05-15 Update (v17):** Major architecture overhaul resolving edge cases in agentic tooling and KV Cache. (1) **Unified Template:** Consolidated Qwen 3.5 and Qwen 3.6 into a single `chat_template.jinja` file that handles all variants seamlessly. (2) **Fixed "Mutually Exclusive" Stopping Bug:** Changed the history-pruning logic from wiping the entire turn to safely array-slicing out just the raw tool tags (`content.split('<tool_call>')[0]`). This preserves the conversational text in the history, which cures the bug where the model would artificially abort its turn (output `<|im_end|>`) when it wanted to talk and use a tool simultaneously. (3) **100% KV Cache Hit Rate Restoration:** Fully normalized internal whitespace logic (`\n\n` -> `\n`) around think blocks and tool calls to exactly match the model's native autoregressive generation spacing. This perfectly synchronizes the template's rendered history with the cached generated tokens, completely eliminating the severe cache invalidation and full-prompt re-processing issues present in v16.
|
| 39 |
-
>
|
| 40 |
-
> **2026-05-14 Update (v16):** Four-part fix addressing community-reported regressions. (1) **Native XML tool format:** reverted from JSON back to the native `<function=name>` / `<parameter=x>` format the model was trained on, restoring full compatibility with vLLM's `qwen3_coder` parser and all inference engines that implement the Qwen tool protocol. (2) **`--reasoning off` respected in error paths:** when thinking is disabled (`enable_thinking=false` / `--reasoning off`), the error escalation directives are now injected as plain text without opening any `<think>` block, preventing degenerate prompts in no-reasoning sessions. (3) **Smarter false-positive detection:** short shell command results (starting with `$ `) and search results with timing footers (`Took X.Xs`) are now correctly excluded from error detection, preventing tool-retry loops when commands succeed but their output happens to contain the word `error`. (4) **`consecutive_failures` counter no longer resets on assistant messages**, allowing Tier 2 escalation to actually fire across multi-turn tool retry chains.
|
| 41 |
-
>
|
| 42 |
-
> **2026-05-13 Update (v15):** Three-part fix for agentic tool-loop failures. (1) **Two-tier error escalation:** replaced the brittle backwards-lookahead error detection with a fully forward-tracking `last_tool_failed` + `consecutive_failures` counter. On the first error the generation prompt is pre-seeded with a correction directive inside `<think>`; on the 2nd+ consecutive error the think block is bypassed and an out-of-band directive forces an immediate corrected action. (2) **Length-gated detection:** error signals are only read from short tool responses (< 500 chars), preventing false positives when reading code files containing `error`, `exception`, etc. in legitimate content. (3) **Static system prompt:** tool instructions are now fully unconditional, permanently eliminating the KV cache invalidation vector introduced in v14.
|
| 43 |
-
>
|
| 44 |
-
> **2026-05-12 Update (v14):** Cured tool amnesia loops and post-tool overthinking friction! Implemented **Smart Loop Preservation** to dynamically scan subsequent tool returns for error markers and conditionally preserve historical reasoning context during active tool failures. Broadened the system instruction scope to define `<think>` as a dual-purpose planning **or synthesis** space, completely eliminating indecisiveness post-tool retrieval.
|
| 45 |
-
>
|
| 46 |
-
> **2026-05-11 Update (v13):** Radical simplification and compatibility overhaul! Reverted tool schemas and assistant output formatting to standard JSON to natively fix downstream MCP parser crashes and C++ implicit enum coercion bugs. Removed the `ns_scan` history loop to permanently fix KV cache invalidation mid-conversation. Replaced global string replacement for hallucinated tags with a C++ safe, localized array-slicing method to prevent data-corruption on user code blocks.
|
| 47 |
-
>
|
| 48 |
-
> **2026-05-10 Update (v12):** Fixed agent stalls, parameter data-loss, and hallucination bugs! Restored dynamic tool instructions and the `<IMPORTANT>` formatting reminder block to stop grammar parser crashes.
|
| 49 |
-
>
|
| 50 |
-
> **2026-05-10 Update (v11):** Fixed agent looping and overthinking! Re-implemented `preserve_thinking` kwarg to properly strip reasoning blocks from history by default, and restored the reasoning bypass (`<think>\n\n</think>\n\n`).
|
| 51 |
-
|
| 52 |
-
</details>
|
| 53 |
-
|
| 54 |
-
This is a drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, and fatal agentic stalling in the official Qwen chat templates.
|
| 55 |
-
|
| 56 |
-
It is tested to work across LM Studio, llama.cpp, vLLM, MLX, oMLX, and any engine that supports HuggingFace Jinja templates.
|
| 57 |
-
|
| 58 |
-
---
|
| 59 |
-
|
| 60 |
-
## Why you need this
|
| 61 |
-
The official Qwen templates contain restrictions and Python-specific Jinja logic that break usage on many inference engines and agent frameworks.
|
| 62 |
-
|
| 63 |
-
Here are the critical issues this template fixes:
|
| 64 |
-
|
| 65 |
-
| Category | Problem | Impact | Fix |
|
| 66 |
-
|---|---|---|---|
|
| 67 |
-
| **Agentic Loop** | **Premature Stalls (Stopping Bug)** | Model aborts its turn (`<\|im_end\|>`) when trying to combine conversation and a tool call. | Resolved the System Prompt logic trap and cured "Empty Think" poisoning (v19). |
|
| 68 |
-
| **Agentic Loop** | **Retry Stall & Reasoning Spiral** | Model correctly diagnoses a tool error but repeatedly emits the identical failing `<tool_call>`. | Two-tier escalation: seeds `<think>` with correction directive; injects urgent out-of-band directive. |
|
| 69 |
-
| **Agentic Loop** | **Post-Tool Overthinking** | Forced `<think>` block prefilling causes model to panic and debate internal rules after fetching data. | Broadened instructions to define `<think>` as a dual-purpose space for planning *or synthesis*. |
|
| 70 |
-
| **Agentic Loop** | **False-Positive Error Detection** | Short successful API/JSON returns containing the word `error` trigger false retry loops. | Strict structural guards look for exact system failures (`"error":`, `Traceback`, etc.) instead of broad words (v18). |
|
| 71 |
-
| **Performance** | **KV Cache Invalidation** | History pruning dynamically mutates past turns, causing full prompt re-processing every turn. | `preserve_thinking` defaults to `true`, maintaining strict chronological rendering for a 100% KV cache hit rate (v19). |
|
| 72 |
-
| **Performance** | **Empty Think Poisoning** | Stripped past turns leave behind empty `<think></think>` tags, tricking the model into a severe in-context learning bias. | Template completely abolishes the injection of empty think blocks (v19). |
|
| 73 |
-
| **Compatibility** | **Legacy Engine Crashes** | Older C++ parsing engines crash when evaluating `loop.previtem`. | Uses strict chronological array indexing universally supported by all Jinja iterations (v18). |
|
| 74 |
-
| **Compatibility** | **Wrong Tool Call Format** | Qwen-native parsers (like vLLM's `qwen3_coder`) expect XML `<function=name>`. JSON format breaks them. | Restored native XML format while keeping C++ safety. |
|
| 75 |
-
| **Compatibility** | **Jinja C++ Crashes** | Python-specific filters (`map`, `first` on strings) crash on `minijinja`. | All filters replaced with universally compatible equivalents. |
|
| 76 |
-
| **Stability** | **Mid-Conversation System Crash** | Frameworks injecting mid-conversation steering instructions trigger a hard crash. | Native, chronological rendering for system messages anywhere in the history. |
|
| 77 |
-
| **Stability** | **No-User-Query Crash** | `raise_exception` crashes agentic loops or system-only contexts. | Graceful fallback implemented. |
|
| 78 |
-
| **Stability** | **Unclosed Thinking Before Tool** | Model calls a tool without closing its reasoning, bleeding XML tags into tool parsers. | Auto-injects closing tags before tool boundaries securely. |
|
| 79 |
-
| **Edge Cases** | **`developer` Role Rejected** | Modern APIs send the developer role; the official template rejects it. | Added full support for `"developer"`. |
|
| 80 |
-
| **Edge Cases** | **`--reasoning off` Ignored** | When thinking is disabled, tool error escalation still opened a `<think>` block, corrupting the prompt. | Error escalation branches now fully respect `enable_thinking=false`. |
|
| 81 |
-
| **Edge Cases** | **Reasoning Bypass Hallucinations** | When thinking is disabled, Qwen models inherently hallucinate reasoning tags anyway. | Injects a safe boundary to successfully force reasoning bypass without stacking newlines (v18). |
|
| 82 |
-
|
| 83 |
-
---
|
| 84 |
-
|
| 85 |
-
## Quick install
|
| 86 |
-
|
| 87 |
-
Choose your environment and update the template:
|
| 88 |
-
|
| 89 |
-
### LM Studio
|
| 90 |
-
1. Open your Qwen model in the right-side panel.
|
| 91 |
-
2. Scroll down to **Prompt Template**.
|
| 92 |
-
3. Replace the template with the contents of `chat_template.jinja`.
|
| 93 |
-
4. Click **Save**.
|
| 94 |
-
|
| 95 |
-
### llama.cpp / koboldcpp
|
| 96 |
-
```bash
|
| 97 |
-
--jinja --chat-template-file chat_template.jinja
|
| 98 |
-
```
|
| 99 |
-
|
| 100 |
-
### vLLM
|
| 101 |
-
Replace the `"chat_template"` string in your `tokenizer_config.json` with the raw file contents. Use the `qwen3_coder` tool parser:
|
| 102 |
-
```bash
|
| 103 |
-
--tool-call-parser qwen3_coder
|
| 104 |
-
```
|
| 105 |
-
|
| 106 |
-
### oMLX
|
| 107 |
-
Overwrite `chat_template.jinja` in your local model directory. Load with `--jinja`. Remove any `chat_template_kwargs` overrides because the template handles everything internally.
|
| 108 |
-
|
| 109 |
-
---
|
| 110 |
-
|
| 111 |
-
## Which file do I use?
|
| 112 |
-
|
| 113 |
-
Both Qwen 3.5 and Qwen 3.6 variants (including 35B, 32B, 27B, and 14B parameters) have been consolidated. You only need the single `chat_template.jinja` file at the root of the repository.
|
| 114 |
-
|
| 115 |
-
One-line versions (`chat_template_oneline.txt`) are pre-minified for engines that require a single-line template string.
|
| 116 |
-
|
| 117 |
-
---
|
| 118 |
-
|
| 119 |
-
## The thinking toggle
|
| 120 |
-
You can control the model reasoning behavior. Insert `<|think_on|>` or `<|think_off|>` anywhere in your system or user prompt.
|
| 121 |
-
|
| 122 |
-
The template natively intercepts the tag, removes it from the final context so the model never sees it, and flips the reasoning mode instantly.
|
| 123 |
-
|
| 124 |
-
**Fast answer, no reasoning:**
|
| 125 |
-
```text
|
| 126 |
-
System: You are a coding assistant. <|think_off|>
|
| 127 |
-
User: What's 2+2?
|
| 128 |
-
```
|
| 129 |
-
|
| 130 |
-
**Deep reasoning:**
|
| 131 |
-
```text
|
| 132 |
-
System: You are a coding assistant. <|think_on|>
|
| 133 |
-
User: Implement a red-black tree in Rust.
|
| 134 |
-
```
|
| 135 |
-
*(The tag syntax uses Qwen's control-token delimiters to guarantee it will never collide with legitimate text or file paths, unlike earlier community templates that used `/think`)*
|
| 136 |
-
|
| 137 |
-
---
|
| 138 |
-
|
| 139 |
-
## Token Saving: Stripping past thoughts
|
| 140 |
-
|
| 141 |
-
By default in v19, this template **preserves** all past `<think>` blocks in the chat history. This is intentional: it prevents the model from suffering "amnesia stalls" during complex, multi-step agentic loops, and it mathematically guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
| 142 |
-
|
| 143 |
-
However, if you are running constrained hardware and need to save context tokens, you can explicitly disable this feature in your engine's template kwargs to automatically strip past thoughts:
|
| 144 |
-
|
| 145 |
-
```json
|
| 146 |
-
{
|
| 147 |
-
"preserve_thinking": false
|
| 148 |
-
}
|
| 149 |
-
```
|
| 150 |
-
*(Note: Setting this to false will naturally reduce your KV Cache hit rate during multi-turn chats, as the prompt string will dynamically mutate).*
|
| 151 |
-
|
| 152 |
-
---
|
| 153 |
-
|
| 154 |
-
<details>
|
| 155 |
-
<summary>Technical Details of the Critical Fixes</summary>
|
| 156 |
-
|
| 157 |
-
### 1. The "Empty Think" Poisoning & Logic Trap Cure (v19)
|
| 158 |
-
Previous versions attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic in-context learning pattern: the model associated empty thoughts with tools, and full thoughts with forbidden conversational text, causing an 80%+ premature `<|im_end|>` stalling rate. v19 abolishes empty think injection and rewrites the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after a thought block.
|
| 159 |
-
|
| 160 |
-
### 2. KV Cache Safety & Autoregressive Normalization (v18/v19)
|
| 161 |
-
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because v19 now preserves historical thoughts chronologically by default, the rendered history perfectly synchronizes with the cached generated tokens. Combined with strict single `\n` normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn loops.
|
| 162 |
-
|
| 163 |
-
### 3. Native XML Tool Call Format (v16)
|
| 164 |
-
The model was trained with the XML-based tool call format used by Qwen3-Coder:
|
| 165 |
-
```xml
|
| 166 |
-
<tool_call>
|
| 167 |
-
<function=tool_name>
|
| 168 |
-
<parameter=param_name>
|
| 169 |
-
value
|
| 170 |
-
</parameter>
|
| 171 |
-
</function>
|
| 172 |
-
</tool_call>
|
| 173 |
-
```
|
| 174 |
-
v16 restored this format natively, making it compatible with all parsers while bypassing the `|items` crash by using C++ safe key iteration (`for args_name in tool_call.arguments`).
|
| 175 |
-
|
| 176 |
-
### 4. Two-Tier Agentic Error Escalation (v15)
|
| 177 |
-
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter:
|
| 178 |
-
- **Tier 1 (1st error):** Generation prompt prefix changes to seed reasoning at a different token position, breaking the cached attractor state.
|
| 179 |
-
- **Tier 2 (2nd+ consecutive errors):** Think block bypassed entirely. An urgent out-of-band directive forces an immediate corrected action wrapped safely within the user `tool_response` block.
|
| 180 |
-
|
| 181 |
-
### 5. Smart False-Positive Detection (v18)
|
| 182 |
-
Instead of broad substring matching that triggers false retry-loops on successful database returns containing words like "error", v18 utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell-echo exclusions (`$ `).
|
| 183 |
-
|
| 184 |
-
### 6. minijinja Compatibility Constraints (v18/v20)
|
| 185 |
-
Python-only Jinja2 features crash or misbehave on `minijinja`/`minja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 186 |
-
- `content | replace('<|think_on|>', '')` -> `content.split('<|think_on|>') | join('')` (Fixes a severe bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 187 |
-
- `\| items` -> `for key in mapping`
|
| 188 |
-
- `loop.previtem` -> `messages[loop.index0 - 1]`
|
| 189 |
-
- `map('string')` -> `join('|')`
|
| 190 |
-
- `\| first` -> `'$ ' in content`
|
| 191 |
-
|
| 192 |
-
</details>
|
| 193 |
-
|
| 194 |
-
<details>
|
| 195 |
-
<summary>Comparison Matrix: Official vs Fixed vs Community</summary>
|
| 196 |
-
|
| 197 |
-
| Feature | Official Qwen Templates | LuffyTheFox | mod-ellary | Pneuny | **This Fixed Template (v19)** |
|
| 198 |
-
|---------|----------|-------------|------------|--------|----------------|
|
| 199 |
-
| Tool call format | XML (native) | JSON | JSON | JSON | **XML (native, qwen3_coder compatible)** |
|
| 200 |
-
| Tool arguments | Fails (`\|items`) | Fixed | Missing | Fixed | **Fixed (C++ safe XML)** |
|
| 201 |
-
| Premature Stalls (Stopping Bug) | Stalls | Stalls | Stalls | Stalls | **Fixed via Logic Trap / Poisoning removal (v19)** |
|
| 202 |
-
| Agentic Retry Stall & Reasoning Spiral | Stalls | Stalls | Stalls | Stalls | **Two-tier escalation system** |
|
| 203 |
-
| False-positive tool errors | N/A | N/A | N/A | N/A | **Guarded (Strict structural matching)** |
|
| 204 |
-
| Post-Tool Overthinking | Spams/Stalls | Broken | Broken | Broken | **Universal Synthesis** |
|
| 205 |
-
| `--reasoning off` on tool errors | N/A | N/A | N/A | N/A | **Fully respected** |
|
| 206 |
-
| `developer` role | Missing | Missing | Missing | Missing | **Added** |
|
| 207 |
-
| Thinking toggle | None | None | `/think` (system only) | None | **`<\|think_off\|>` anywhere** |
|
| 208 |
-
| Empty think in history | Spams empty blocks | Broken | Tags omitted | Broken | **Abolished completely (v19)** |
|
| 209 |
-
| KV prefix caching | Breaks on dynamic history | Breaks | Breaks | Breaks | **100% stable out-of-the-box (v19)** |
|
| 210 |
-
| Mid-conversation system | Crashes | Crashes | Crashes | Crashes | **Fixed** |
|
| 211 |
-
| No-user-query crash | Crashes | Crashes | Crashes | Crashes | **Graceful fallback** |
|
| 212 |
-
| Legacy AST support | Fails (`previtem`) | Fails | Fails | Fails | **Fixed (`index0`)** |
|
| 213 |
-
| `</thinking>` hallucination | Fails | N/A | N/A | N/A | **Detected and safely trimmed** |
|
| 214 |
-
|
| 215 |
-
</details>
|
| 216 |
-
|
| 217 |
-
---
|
| 218 |
-
|
| 219 |
-
## Running the test suite
|
| 220 |
-
|
| 221 |
-
```bash
|
| 222 |
-
python3 scripts/test_v20.py
|
| 223 |
-
```
|
| 224 |
-
|
| 225 |
-
Tests cover: `auto_disable_thinking_with_tools`, payload truncation logic, parallel tool spacing, mid-conversation system rendering, deep agent loop fallback, XML tool format, `<|think_off|>` / `<|think_on|>` inline overrides, and all legacy v19 regression tests.
|
| 226 |
-
|
| 227 |
-
---
|
| 228 |
-
|
| 229 |
-
## Authorship
|
| 230 |
-
|
| 231 |
-
| Role | Author |
|
| 232 |
-
|------|--------|
|
| 233 |
-
| Original models | Alibaba Cloud (Qwen team) |
|
| 234 |
-
| Template fixes | [froggeric](https://huggingface.co/froggeric) |
|
| 235 |
-
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 236 |
-
|
| 237 |
-
## License
|
| 238 |
-
|
| 239 |
-
Apache-2.0, inherited from Qwen.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/README-v21.md
DELETED
|
@@ -1,234 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
tags:
|
| 4 |
-
- jinja
|
| 5 |
-
- chat-template
|
| 6 |
-
- qwen
|
| 7 |
-
- qwen3.5
|
| 8 |
-
- qwen3.6
|
| 9 |
-
- lm-studio
|
| 10 |
-
- mlx
|
| 11 |
-
- llama.cpp
|
| 12 |
-
- vllm
|
| 13 |
-
- tool-calling
|
| 14 |
-
- thinking
|
| 15 |
-
---
|
| 16 |
-
|
| 17 |
-
# Fixed jinja chat templates for Qwen 3.5 & 3.6 (v21)
|
| 18 |
-
|
| 19 |
-
This is a drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, and fatal agentic stalling in the official Qwen chat templates.
|
| 20 |
-
|
| 21 |
-
It works across LM Studio, llama.cpp, vLLM, MLX, oMLX, and any engine that supports HuggingFace Jinja templates. You only need the single `chat_template.jinja` file at the root of the repository for all Qwen 3.5 and 3.6 variants.
|
| 22 |
-
|
| 23 |
-
---
|
| 24 |
-
|
| 25 |
-
<details>
|
| 26 |
-
<summary><b>Quick Install</b></summary>
|
| 27 |
-
|
| 28 |
-
Choose your environment and update the template:
|
| 29 |
-
|
| 30 |
-
### LM Studio
|
| 31 |
-
1. Open your Qwen model in the right side panel.
|
| 32 |
-
2. Scroll down to **Prompt Template**.
|
| 33 |
-
3. Replace the template with the contents of `chat_template.jinja`.
|
| 34 |
-
4. Click **Save**.
|
| 35 |
-
|
| 36 |
-
### llama.cpp / koboldcpp
|
| 37 |
-
Use the file directly in your launch command:
|
| 38 |
-
```bash
|
| 39 |
-
--jinja --chat-template-file chat_template.jinja
|
| 40 |
-
```
|
| 41 |
-
|
| 42 |
-
### vLLM
|
| 43 |
-
Replace the `"chat_template"` string in your `tokenizer_config.json` with the raw file contents. Use the `qwen3_coder` tool parser:
|
| 44 |
-
```bash
|
| 45 |
-
--tool-call-parser qwen3_coder
|
| 46 |
-
```
|
| 47 |
-
|
| 48 |
-
### oMLX
|
| 49 |
-
Overwrite `chat_template.jinja` in your local model directory. Load with `--jinja`. Remove any `chat_template_kwargs` overrides because the template handles everything internally.
|
| 50 |
-
|
| 51 |
-
</details>
|
| 52 |
-
|
| 53 |
-
---
|
| 54 |
-
|
| 55 |
-
## Why you need this
|
| 56 |
-
|
| 57 |
-
The official Qwen templates contain restrictions and Python-specific Jinja logic that break usage on many inference engines and agent frameworks.
|
| 58 |
-
|
| 59 |
-
<details open>
|
| 60 |
-
<summary><b>Critical Issues Fixed</b></summary>
|
| 61 |
-
|
| 62 |
-
| Area | Issue | The Fix |
|
| 63 |
-
|---|---|---|
|
| 64 |
-
| **Agentic Loop** | Model aborts turn when combining chat and a tool call. | Cured "Empty Think" poisoning. |
|
| 65 |
-
| **Agentic Loop** | Model gets stuck emitting the identical failing tool call. | Added two-tier error escalation to force correction. |
|
| 66 |
-
| **Agentic Loop** | Model panics and debates internal rules after fetching data. | Broadened `<think>` instructions to allow synthesis. |
|
| 67 |
-
| **Agentic Loop** | API returns containing the word "error" trigger false loops. | Replaced broad matching with strict structural guards. |
|
| 68 |
-
| **Performance** | Mutated past turns constantly destroy the prefix cache. | Enforced chronological history for a 100% KV Cache hit rate. |
|
| 69 |
-
| **Performance** | Deep Jinja nesting drops `llama.cpp` speed by 80%. | Flattened the AST architecture to maximize throughput. |
|
| 70 |
-
| **Compatibility** | Python-specific filters crash C++ inference engines. | Rewrote all filters to be 100% `minijinja` safe. |
|
| 71 |
-
| **Compatibility** | Qwen-native parsers (like vLLM) crash on JSON formatting. | Maintained canonical Qwen XML format as the default. |
|
| 72 |
-
| **Compatibility** | Older API setups and wrappers crash on native XML. | Added a `tool_call_format="json"` opt-in override. |
|
| 73 |
-
| **Compatibility** | Anthropic `message.thinking` payloads are rejected. | Added native Anthropic reasoning support. |
|
| 74 |
-
| **Stability** | Massive tool data returns blow out the context window. | Added dynamic payload truncation limits. |
|
| 75 |
-
| **Stability** | Mid-conversation system prompts crash the template. | Added native support for arbitrary system messages. |
|
| 76 |
-
| **Stability** | Model calls a tool without closing its reasoning tags. | Auto-injects closing tags before tool boundaries. |
|
| 77 |
-
| **Edge Cases** | Text occasionally duplicates during streaming generation. | Restored canonical spacing to the generation prompt. |
|
| 78 |
-
| **Edge Cases** | Model hallucinates reasoning tags when thinking is disabled. | Injected strict boundaries to force reasoning bypass. |
|
| 79 |
-
|
| 80 |
-
</details>
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
---
|
| 85 |
-
|
| 86 |
-
## Customization & Usage
|
| 87 |
-
|
| 88 |
-
<details>
|
| 89 |
-
<summary><b>The Thinking Toggle</b></summary>
|
| 90 |
-
|
| 91 |
-
You can control the model reasoning behavior. Insert `<|think_on|>` or `<|think_off|>` anywhere in your system or user prompt. The template intercepts the tag, removes it from the final context so the model never sees it, and flips the reasoning mode instantly.
|
| 92 |
-
|
| 93 |
-
**Fast answer, no reasoning:**
|
| 94 |
-
```text
|
| 95 |
-
System: You are a coding assistant. <|think_off|>
|
| 96 |
-
User: What's 2+2?
|
| 97 |
-
```
|
| 98 |
-
|
| 99 |
-
**Deep reasoning:**
|
| 100 |
-
```text
|
| 101 |
-
System: You are a coding assistant. <|think_on|>
|
| 102 |
-
User: Implement a red-black tree in Rust.
|
| 103 |
-
```
|
| 104 |
-
*(The tag syntax uses Qwen's control token delimiters to guarantee it will never collide with legitimate text or file paths, unlike earlier community templates that used `/think`)*
|
| 105 |
-
|
| 106 |
-
</details>
|
| 107 |
-
|
| 108 |
-
<details>
|
| 109 |
-
<summary><b>Token Saving (Stripping past thoughts)</b></summary>
|
| 110 |
-
|
| 111 |
-
By default, this template **preserves** all past `<think>` blocks in the chat history. This prevents the model from suffering "amnesia stalls" during complex agentic loops, and it mathematically guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
| 112 |
-
|
| 113 |
-
If you are running constrained hardware and need to save context tokens, you can explicitly disable this feature in your engine's template kwargs to automatically strip past thoughts:
|
| 114 |
-
```json
|
| 115 |
-
{
|
| 116 |
-
"preserve_thinking": false
|
| 117 |
-
}
|
| 118 |
-
```
|
| 119 |
-
*(Note: Setting this to false will reduce your KV Cache hit rate during multi-turn chats because the prompt string will dynamically mutate).*
|
| 120 |
-
|
| 121 |
-
</details>
|
| 122 |
-
|
| 123 |
-
<details>
|
| 124 |
-
<summary><b>Tool Call Format Override (JSON vs XML)</b></summary>
|
| 125 |
-
|
| 126 |
-
Qwen models are natively trained to output tool calls in XML (`<function=name>`). By default, this template uses this native XML format to maximize the model's intelligence and reliability.
|
| 127 |
-
|
| 128 |
-
**For most users (including those using `vLLM` or `llama.cpp` / `llama-server`), you do NOT need to change anything.** Modern inference engines natively understand Qwen's XML and will automatically translate it into standard OpenAI API responses for your downstream clients.
|
| 129 |
-
|
| 130 |
-
**When to use the JSON override:**
|
| 131 |
-
If you are using a custom wrapper, an older engine version, or a specific framework (like `ik_llama`) that *strictly* expects the model itself to output Hermes JSON (`{"name": "...", "arguments": {...}}`) and crashes on XML, you can force the template to use JSON by passing this kwarg:
|
| 132 |
-
```json
|
| 133 |
-
{
|
| 134 |
-
"tool_call_format": "json"
|
| 135 |
-
}
|
| 136 |
-
```
|
| 137 |
-
> [!WARNING]
|
| 138 |
-
> When you opt into the JSON format, the template explicitly disables the `max_tool_arg_chars` truncation feature. Truncating a JSON string structurally corrupts its syntax, which would poison the model's history and crash downstream parsers.
|
| 139 |
-
|
| 140 |
-
</details>
|
| 141 |
-
|
| 142 |
-
---
|
| 143 |
-
|
| 144 |
-
<details>
|
| 145 |
-
<summary><b>Running the test suite</b></summary>
|
| 146 |
-
|
| 147 |
-
```bash
|
| 148 |
-
python3 scripts/test_v21.py
|
| 149 |
-
```
|
| 150 |
-
Tests cover `auto_disable_thinking_with_tools`, payload truncation logic, parallel tool spacing, mid-conversation system rendering, deep agent loop fallback, XML tool format, `<|think_off|>` / `<|think_on|>` inline overrides, and all legacy regression tests.
|
| 151 |
-
|
| 152 |
-
</details>
|
| 153 |
-
|
| 154 |
-
---
|
| 155 |
-
|
| 156 |
-
## Authorship
|
| 157 |
-
| Role | Author |
|
| 158 |
-
|------|--------|
|
| 159 |
-
| Original models | Alibaba Cloud (Qwen team) |
|
| 160 |
-
| Template fixes | [froggeric](https://huggingface.co/froggeric) |
|
| 161 |
-
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 162 |
-
|
| 163 |
-
## License
|
| 164 |
-
Apache-2.0, inherited from Qwen.
|
| 165 |
-
|
| 166 |
-
---
|
| 167 |
-
|
| 168 |
-
<details>
|
| 169 |
-
<summary>Technical Details of the Critical Fixes</summary>
|
| 170 |
-
|
| 171 |
-
### 1. The "Empty Think" Poisoning & Logic Trap Cure
|
| 172 |
-
Previous versions attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic learning pattern: the model associated empty thoughts with tools, and full thoughts with forbidden conversational text, causing an 80%+ premature `<|im_end|>` stalling rate. We abolished empty think injection and rewrote the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after a thought block.
|
| 173 |
-
|
| 174 |
-
### 2. KV Cache Safety & Autoregressive Normalization
|
| 175 |
-
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because this template now preserves historical thoughts chronologically by default, the rendered history perfectly synchronizes with the cached generated tokens. Combined with strict single `\n` normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn loops.
|
| 176 |
-
|
| 177 |
-
### 3. Native XML Tool Call Format
|
| 178 |
-
The model was trained with the XML tool call format used by Qwen3-Coder. We restored this format natively, making it compatible with all parsers while bypassing the `|items` crash by using C++ safe key iteration (`for args_name in tool_call.arguments`).
|
| 179 |
-
|
| 180 |
-
### 4. Two-Tier Agentic Error Escalation
|
| 181 |
-
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward tracked `consecutive_failures` counter. On the first error, the generation prompt prefix changes to seed reasoning at a different token position, breaking the cached attractor state. On the second consecutive error, the think block is bypassed entirely and an urgent out-of-band directive forces an immediate corrected action wrapped safely within the user `tool_response` block.
|
| 182 |
-
|
| 183 |
-
### 5. Smart False-Positive Detection
|
| 184 |
-
Instead of broad substring matching that triggers false retry loops on successful database returns containing words like "error", this template utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell echo exclusions (`$ `).
|
| 185 |
-
|
| 186 |
-
### 6. minijinja Compatibility Constraints
|
| 187 |
-
Python-only Jinja2 features crash or misbehave on `minijinja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 188 |
-
* `content | replace('<|think_on|>', '')` became `content.split('<|think_on|>') | join('')` (Fixes a bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 189 |
-
* `\| items` became `for key in mapping`
|
| 190 |
-
* `loop.previtem` became `messages[loop.index0 - 1]`
|
| 191 |
-
* `map('string')` became `join('|')`
|
| 192 |
-
* `\| first` became `'$ ' in content`
|
| 193 |
-
|
| 194 |
-
### 7. AST Flattening for C++ Throughput
|
| 195 |
-
Deeply nested Jinja loops and macros create severe parsing bottlenecks in C++ inference engines. We flattened the AST architecture, effectively curing an 80% inference throughput drop on `llama.cpp` by streamlining how `ns_state` tracking and historical rendering loops are evaluated.
|
| 196 |
-
|
| 197 |
-
### 8. Dynamic Payload Truncation
|
| 198 |
-
Massive API or database returns can instantly blow out a model's context window. We implemented `max_tool_arg_chars` and `max_tool_response_chars` limiters that safely slice oversized payloads. Crucially, this truncation is automatically disabled when the `tool_call_format="json"` override is active, as slicing a serialized JSON string structurally corrupts the data and crashes downstream parsers.
|
| 199 |
-
|
| 200 |
-
### 9. Reasoning Bypass Hallucination Mitigation
|
| 201 |
-
When thinking is disabled, Qwen models often hallucinate reasoning tags due to their training bias. We injected a safe boundary and adjusted the `<IMPORTANT>` system block to remove explicit mentions of `</think>` during tool instructions. This successfully stops the model from hallucinating closing tags when attempting to call tools in a no-reasoning state.
|
| 202 |
-
|
| 203 |
-
</details>
|
| 204 |
-
|
| 205 |
-
<details>
|
| 206 |
-
<summary>Update History & Changelog</summary>
|
| 207 |
-
|
| 208 |
-
> **2026-07-02 Update (v21.3): Optional JSON Tool Format Kwarg.** Added an optional `tool_call_format="json"` override for the `chat_template_kwargs`. This provides an escape hatch for users on specific setups (like custom wrappers or older engines that only support Hermes JSON parsing) without breaking the canonical XML default or degrading the model's primary training distribution. `max_tool_arg_chars` truncation is safely bypassed in JSON mode to prevent syntax corruption.
|
| 209 |
-
|
| 210 |
-
> **2026-07-02 Update (v21.2): Reasoning Bypass Hallucination Fix.** Adjusted the `<IMPORTANT>` block instructions to remove explicit mentions of `</think>` when defining tool call behavior. This stops the model from hallucinating `</think>` tags as a prefix when reasoning is explicitly disabled.
|
| 211 |
-
|
| 212 |
-
> **2026-07-02 Update (v21.1): Reliability Overhaul & XML Revert.** Addressed critical bugs and compatibility issues, particularly around prefix cache efficiency. (1) **Tool Call XML Format Revert:** Reverted PR 45 to restore the native XML format, as JSON fundamentally broke `vLLM`'s native `qwen3_coder` parser. (2) **Prefix Cache Fixes:** Restored the `preserve_thinking` default to `true` and removed extraneous newlines that broke caching. (3) **Prompt Injection Guard:** `<|think_off|>` tags in untrusted tool responses are now correctly ignored. (4) **Quoted Tag Bug Fix:** The template no longer corrupts history when the assistant quotes `</think>`. (5) **Anthropic `message.thinking`:** Added support for Anthropic reasoning content. (6) **False-Positive Tool Errors:** Reduced error detection scope to the first 80 characters of a response to prevent false positives. (7) **Canonical Generation Prompt:** Restored the canonical `\n\n` spacing to the non-thinking generation prompt to fix answer duplication issues under streaming. *(Huge thanks to `Moore2877`, `choongng`, and the community for their excellent contributions!)*
|
| 213 |
-
|
| 214 |
-
> **2026-06-05 Update (v20): The Architect Patch.** Major structural update for agentic loops and C++ inference engines. (1) **Minja AST Flattening:** Optimized Jinja nesting to fix parsing bottlenecks that dropped inference throughput by 80% on `llama.cpp`. (2) **Minja Replace Bug Fix (Hotfix):** Bypassed a C++ parsing bug in `llama.cpp` where using the `replace` filter at index 0 of a user prompt silently dropped the entire text payload. Inline thinking toggles now use `split` and `join` for robust stripping. (3) **Auto-disable Thinking:** Introduced `auto_disable_thinking_with_tools` kwarg (default `false`) that allows users to instantly shut off reasoning blocks during tool use. (4) **Deep Agent Fallbacks:** Resolved exceptions triggered by mid-conversation system prompts or loops lacking human `user` messages. (5) **Payload Truncation:** Implemented `max_tool_arg_chars` and `max_tool_response_chars` configurations to definitively stop context window explosions from massive data returns. *(Huge thanks to `barubary` / `spiritbuun` for their contributions to these C++ architecture optimizations!)*
|
| 215 |
-
|
| 216 |
-
> **2026-05-18 Update (v19): The Agentic Loop Cure.** (1) **Abolished "Empty Think" Poisoning:** Rewrote the AST history rendering to completely remove the injection of empty `<think>\n</think>` blocks. This cures a severe in-context learning bias where the model assumed tools could only be called if it didn't think first, which was causing 80%+ of premature `<|im_end|>` turn aborts. (2) **System Prompt Logic Trap Removed:** Softened the absolute tool mandate in the `<IMPORTANT>` block and restored Universal Synthesis instructions. The model is now explicitly permitted to transition from `</think>` to a conversational answer without panicking. (3) **True 100% KV Cache & Amnesia Fix:** `preserve_thinking` now defaults to `true`. Past thoughts are retained chronologically, permanently curing "amnesia stalls" during multi-step tool loops while mathematically guaranteeing 100% KV Cache prefix matching out of the box.
|
| 217 |
-
|
| 218 |
-
> **2026-05-16 Update (v18): Stability & Precision Patch.** (1) **Bulletproof False-Positive Detection:** Shifted agentic error detection from broad substring matching to strict structural formats (e.g. `"error":`, `Exception:`, `Traceback`), completely curing false-positive retry loops when successful JSON returns simply contain the word "error" or "fail". (2) **Legacy Engine Compatibility:** Replaced `loop.previtem` with explicit array indexing, fixing AST crashes on older `llama.cpp` and `minijinja` builds that do not track loop state items. (3) **True Whitespace Normalization:** Fixed a bug where reasoning bypasses and hallucinated tag recovery stacked hidden multi-newlines (`\n\n\n`), strictly fulfilling the 100% KV Cache hit rate claim for all edge cases. (4) **Code Cleanup:** Removed dead conditional branches during XML tool parsing.
|
| 219 |
-
|
| 220 |
-
> **2026-05-15 Update (v17):** (1) **Unified Template:** Consolidated Qwen 3.5 and Qwen 3.6 into a single `chat_template.jinja` file that handles all variants. (2) **Fixed "Mutually Exclusive" Stopping Bug:** Changed the history pruning logic from wiping the entire turn to safely array slicing out just the raw tool tags (`content.split('<tool_call>')[0]`). This preserves the conversational text in the history, which cures the bug where the model would artificially abort its turn (output `<|im_end|>`) when it wanted to talk and use a tool simultaneously. (3) **100% KV Cache Hit Rate Restoration:** Fully normalized internal whitespace logic (`\n\n` -> `\n`) around think blocks and tool calls to exactly match the model's native autoregressive generation spacing. This perfectly synchronizes the template's rendered history with the cached generated tokens, completely eliminating the severe cache invalidation and full prompt re-processing issues present in v16.
|
| 221 |
-
|
| 222 |
-
> **2026-05-14 Update (v16):** (1) **Native XML tool format:** reverted from JSON back to the native `<function=name>` / `<parameter=x>` format the model was trained on, restoring full compatibility with vLLM's `qwen3_coder` parser and all inference engines that implement the Qwen tool protocol. (2) **`--reasoning off` respected in error paths:** when thinking is disabled (`enable_thinking=false` / `--reasoning off`), the error escalation directives are now injected as plain text without opening any `<think>` block, preventing degenerate prompts in no-reasoning sessions. (3) **Smarter false-positive detection:** short shell command results (starting with `$ `) and search results with timing footers (`Took X.Xs`) are now correctly excluded from error detection, preventing tool retry loops when commands succeed but their output happens to contain the word `error`. (4) **`consecutive_failures` counter no longer resets on assistant messages**, allowing Tier 2 escalation to actually fire across multi-turn tool retry chains.
|
| 223 |
-
|
| 224 |
-
> **2026-05-13 Update (v15):** (1) **Two-tier error escalation:** replaced the brittle backwards lookahead error detection with a fully forward tracking `last_tool_failed` + `consecutive_failures` counter. On the first error the generation prompt is pre-seeded with a correction directive inside `<think>`; on the 2nd+ consecutive error the think block is bypassed and an out-of-band directive forces an immediate corrected action. (2) **Length-gated detection:** error signals are only read from short tool responses (< 500 chars), preventing false positives when reading code files containing `error`, `exception`, etc in legitimate content. (3) **Static system prompt:** tool instructions are now fully unconditional, permanently eliminating the KV cache invalidation vector introduced in v14.
|
| 225 |
-
|
| 226 |
-
> **2026-05-12 Update (v14):** Cured tool amnesia loops and post-tool overthinking friction! Implemented Smart Loop Preservation to dynamically scan subsequent tool returns for error markers and conditionally preserve historical reasoning context during active tool failures. Broadened the system instruction scope to define `<think>` as a dual-purpose planning or synthesis space, completely eliminating indecisiveness post-tool retrieval.
|
| 227 |
-
|
| 228 |
-
> **2026-05-11 Update (v13):** Reverted tool schemas and assistant output formatting to standard JSON to natively fix downstream MCP parser crashes and C++ implicit enum coercion bugs. Removed the `ns_scan` history loop to permanently fix KV cache invalidation mid-conversation. Replaced global string replacement for hallucinated tags with a C++ safe, localized array slicing method to prevent data corruption on user code blocks.
|
| 229 |
-
|
| 230 |
-
> **2026-05-10 Update (v12):** Fixed agent stalls, parameter data loss, and hallucination bugs! Restored dynamic tool instructions and the `<IMPORTANT>` formatting reminder block to stop grammar parser crashes.
|
| 231 |
-
|
| 232 |
-
> **2026-05-10 Update (v11):** Fixed agent looping and overthinking! Re-implemented `preserve_thinking` kwarg to properly strip reasoning blocks from history by default, and restored the reasoning bypass (`<think>\n\n</think>\n\n`).
|
| 233 |
-
|
| 234 |
-
</details>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/README-v22.1.md
DELETED
|
@@ -1,285 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
tags:
|
| 4 |
-
- jinja
|
| 5 |
-
- chat-template
|
| 6 |
-
- qwen
|
| 7 |
-
- qwen3.5
|
| 8 |
-
- qwen3.6
|
| 9 |
-
- qwen3.8
|
| 10 |
-
- lm-studio
|
| 11 |
-
- mlx
|
| 12 |
-
- llama.cpp
|
| 13 |
-
- vllm
|
| 14 |
-
- tool-calling
|
| 15 |
-
- thinking
|
| 16 |
-
---
|
| 17 |
-
|
| 18 |
-
# Fixed jinja chat templates for Qwen 3.5, 3.6 & 3.8 (v22.1)
|
| 19 |
-
|
| 20 |
-
This is a universal drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, empty think poisoning, and fatal agentic stalling across official Qwen chat templates.
|
| 21 |
-
|
| 22 |
-
It works across LM Studio, llama.cpp, vLLM, MLX, oMLX, KoboldCPP, and any engine that supports Hugging Face Jinja templates. You only need the single `chat_template.jinja` file at the root of this repository for all Qwen 3.5, 3.6, and 3.8 model sizes.
|
| 23 |
-
|
| 24 |
-
---
|
| 25 |
-
|
| 26 |
-
## What is new: The Qwen 3.8 Update
|
| 27 |
-
|
| 28 |
-
Qwen has released their 3.8 generation starting with `Qwen3.8-2.4T-A95B` and `Qwen3.8-27B`. While the official 3.8 template adopted our default reasoning preservation, it also introduced severe regressions and rigid lockdowns that break local setups. This release brings full Qwen 3.8 support while fixing these official bugs:
|
| 29 |
-
|
| 30 |
-
1. **Prompt-Steered Reasoning Effort:** Qwen 3.8 introduces 3 core reasoning levels (`xhigh`, `medium`, `low`). The template injects official steering instructions when requested, and auto-suppresses them when thinking is turned off. Client API aliases (`high` / `max` for `xhigh`, `minimal` for `low`, and `none` for thinking off) are supported automatically for OpenAI and vLLM client compatibility.
|
| 31 |
-
2. **Safe Default Reasoning Baseline (`medium`):** In official Qwen 3.8, `reasoning_effort` defaulted to `xhigh`. On complex tasks with finite token limits, this pushed the model to explore alternate branches until hitting `max_tokens`, returning zero content. We set the default to `medium` (zero injected tokens), preserving 100% prefix cache parity with v21.3 while eliminating empty-content failures.
|
| 32 |
-
3. **Inline Chat Tags for Direct Effort Steering:** You can steer thinking depth directly inside chat prompts (for LM Studio, OpenCode, or WebUI) using inline tags:
|
| 33 |
-
* `<|think_low|>`: Enables thinking with concise reasoning instructions.
|
| 34 |
-
* `<|think_medium|>`: Enables thinking with standard baseline instructions.
|
| 35 |
-
* `<|think_xhigh|>`: Enables thinking with deep reasoning instructions.
|
| 36 |
-
* `<|think_off|>`: Turns off reasoning completely for fast responses.
|
| 37 |
-
*(All tags are automatically stripped from the rendered prompt, so the model never sees raw control tags)*.
|
| 38 |
-
4. **Restored Fast Mode (No Reasoning):** Official 3.8 throws a fatal runtime exception if you pass `enable_thinking=false`. We removed this lockdown, giving you complete freedom to disable reasoning via kwargs or inline `<|think_off|>` tags.
|
| 39 |
-
5. **Cured Official 3.8 Empty Think Bug:** Official 3.8 removed the in-content thinking parser. In multi-turn chats where reasoning is stored inside message content, official 3.8 injects a blank `<think></think>` block before the real thoughts. This template extracts reasoning cleanly without duplicating tags.
|
| 40 |
-
6. **Universal Tool Arguments:** Official 3.8 crashes with `TypeError: Can only get item pairs from a mapping` when clients send standard OpenAI string arguments. This template handles both Python dictionaries and JSON strings seamlessly.
|
| 41 |
-
7. **Native llama.cpp `--reasoning-preserve` Support:** Added native support for the new `llama.cpp` CLI flag via the `preserve_reasoning` alias.
|
| 42 |
-
8. **Diagnostic Tooling:** Added `scripts/check_applied.py` to check whether `chat_template.jinja` or `tokenizer_config.json` is actively loaded by your runtime.
|
| 43 |
-
|
| 44 |
-
---
|
| 45 |
-
|
| 46 |
-
<details open>
|
| 47 |
-
<summary><b>Quick Install & Engine Setup</b></summary>
|
| 48 |
-
|
| 49 |
-
### llama.cpp / llama-server / koboldcpp
|
| 50 |
-
Run `llama-server` with the template file and DeepSeek reasoning format:
|
| 51 |
-
```bash
|
| 52 |
-
llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek
|
| 53 |
-
```
|
| 54 |
-
*Why `--reasoning-format deepseek` matters:* When connecting coding agents like OpenCode, Claude Code, or Pi.dev to `llama-server`, this flag extracts `<think>` blocks into the dedicated `reasoning_content` API response field. This prevents raw thinking tokens from leaking into the text stream and stopping tool calls midway.
|
| 55 |
-
|
| 56 |
-
*Native CLI flag:* On recent `llama.cpp` builds, you can pass `--reasoning-preserve` directly to ensure 100% Prefix KV Cache retention.
|
| 57 |
-
|
| 58 |
-
### LM Studio
|
| 59 |
-
1. Open your Qwen model in the right side panel.
|
| 60 |
-
2. Scroll down to **Prompt Template**.
|
| 61 |
-
3. Replace the template with the contents of `chat_template.jinja`.
|
| 62 |
-
4. Click **Save**.
|
| 63 |
-
|
| 64 |
-
### vLLM
|
| 65 |
-
Replace the `"chat_template"` string in your `tokenizer_config.json` with `chat_template_oneline.txt` (or raw `chat_template.jinja`).
|
| 66 |
-
```bash
|
| 67 |
-
vllm serve Qwen/Qwen3.8-2.4T-A95B --tool-call-parser qwen3_xml
|
| 68 |
-
```
|
| 69 |
-
*Parser selection:* Use `--tool-call-parser qwen3_xml` on current vLLM releases. If you are on an older vLLM build, use `--tool-call-parser qwen3_coder`. If you explicitly set `tool_call_format="json"`, use `--tool-call-parser hermes`.
|
| 70 |
-
|
| 71 |
-
### oMLX / MLX
|
| 72 |
-
Overwrite `chat_template.jinja` in your local model directory and launch with `--jinja`.
|
| 73 |
-
|
| 74 |
-
</details>
|
| 75 |
-
|
| 76 |
-
---
|
| 77 |
-
|
| 78 |
-
## Why you need this
|
| 79 |
-
|
| 80 |
-
The official Qwen templates contain engine restrictions, Python-specific Jinja logic, and regressions that break local inference and agent workflows.
|
| 81 |
-
|
| 82 |
-
<details open>
|
| 83 |
-
<summary><b>Critical Issues Fixed</b></summary>
|
| 84 |
-
|
| 85 |
-
| Area | Issue in Official Templates | The Fix |
|
| 86 |
-
|---|---|---|
|
| 87 |
-
| **Qwen 3.8 Support** | Official 3.8 crashes if `enable_thinking=false`. | **Restored Fast Mode**. Supports fast non-reasoning mode via kwargs or `<\|think_off\|>`. |
|
| 88 |
-
| **Qwen 3.8 Token Safety** | Official `xhigh` default burns token budgets with zero content returned. | **Safe `medium` Default**. Zero prompt injection unless explicitly requested. |
|
| 89 |
-
| **Qwen 3.8 Regression** | Official 3.8 injects duplicate blank `<think></think>` in chat history. | **Cured Empty Think Poisoning**. Robust multi-format reasoning extraction. |
|
| 90 |
-
| **Reasoning Control** | Inability to change reasoning effort in chat interfaces. | **Inline Chat Tags**. Full support for `<\|think_low\|>`, `<\|think_medium\|>`, and `<\|think_xhigh\|>`. |
|
| 91 |
-
| **Compatibility** | `llama.cpp --reasoning-preserve` CLI flag compatibility. | **Native Alias Support**. Supports both `preserve_reasoning` and `preserve_thinking`. |
|
| 92 |
-
| **Compatibility** | JSON-string tool arguments (OpenAI / Ollama) crash official templates. | **Universal Tool Parsing**. Safely handles mappings, JSON strings, and scalar args. |
|
| 93 |
-
| **Agentic Loop** | Model aborts turn when combining conversational text and a tool call. | Cured "Empty Think" poisoning and softened imperative system directives. |
|
| 94 |
-
| **Agentic Loop** | Model gets stuck emitting the identical failing tool call. | Added two-tier error escalation to force correction while retaining reasoning. |
|
| 95 |
-
| **Agentic Loop** | Model panics and debates internal rules after fetching data. | Broadened `<think>` instructions to authorize conversational synthesis. |
|
| 96 |
-
| **Agentic Loop** | API returns containing the word "error" trigger false retry loops. | Replaced broad matching with strict structural guards. |
|
| 97 |
-
| **Performance** | Mutated past turns destroy the prefix cache. | Enforced chronological history for a 100% KV Cache hit rate. |
|
| 98 |
-
| **Performance** | Deep Jinja nesting drops `llama.cpp` speed by 80%. | Flattened the AST architecture to maximize throughput. |
|
| 99 |
-
| **Compatibility** | Python-specific filters crash C++ inference engines. | Rewrote all filters to be 100% `minijinja` safe. |
|
| 100 |
-
| **Compatibility** | Qwen-native parsers (like vLLM) crash on JSON formatting. | Maintained canonical Qwen XML format as the default. |
|
| 101 |
-
| **Compatibility** | Older API setups and wrappers crash on native XML. | Added a `tool_call_format="json"` opt-in override. |
|
| 102 |
-
| **Compatibility** | Anthropic `message.thinking` payloads are rejected. | Added native Anthropic reasoning support. |
|
| 103 |
-
| **Stability** | Massive tool data returns blow out the context window. | Added dynamic payload truncation limits. |
|
| 104 |
-
| **Stability** | Mid-conversation system prompts crash the template. | Added native support for arbitrary system and developer messages. |
|
| 105 |
-
| **Edge Cases** | Text duplicates during streaming generation. | Restored canonical spacing to the generation prompt. |
|
| 106 |
-
| **Edge Cases** | Model hallucinates reasoning tags when thinking is disabled. | Injected strict boundaries to force clean reasoning bypass. |
|
| 107 |
-
|
| 108 |
-
</details>
|
| 109 |
-
|
| 110 |
-
---
|
| 111 |
-
|
| 112 |
-
## Customization & Kwarg Reference
|
| 113 |
-
|
| 114 |
-
<details open>
|
| 115 |
-
<summary><b>1. Reasoning Effort Steering (Qwen 3.8)</b></summary>
|
| 116 |
-
|
| 117 |
-
Qwen 3.8 has 3 native prompt-steered reasoning levels. You can control this via template kwargs or directly inline in your chat messages:
|
| 118 |
-
|
| 119 |
-
**Via Template Kwargs (`chat_template_kwargs`):**
|
| 120 |
-
```json
|
| 121 |
-
{
|
| 122 |
-
"reasoning_effort": "xhigh"
|
| 123 |
-
}
|
| 124 |
-
```
|
| 125 |
-
* **`"medium"` (Default / Safe Baseline):** No extra instruction text injected. Preserves 100% Prefix KV Cache parity with v21 and lets the model reason naturally without token-budget traps.
|
| 126 |
-
* **`"xhigh"` (Deep Reasoning):** Injects Qwen's official deep reasoning instruction:
|
| 127 |
-
> *"Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."*
|
| 128 |
-
* **`"low"` (Concise Reasoning):** Injects concise thinking instructions for fast, summary-oriented reasoning:
|
| 129 |
-
> *"Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."*
|
| 130 |
-
|
| 131 |
-
*API Compatibility Aliases:*
|
| 132 |
-
To prevent errors when calling the model through standard API proxies:
|
| 133 |
-
* `"high"` and `"max"` automatically map to `"xhigh"` (standard OpenAI API clients send `"high"`).
|
| 134 |
-
* `"minimal"` automatically maps to `"low"`.
|
| 135 |
-
* `"none"` disables thinking entirely.
|
| 136 |
-
|
| 137 |
-
**Via Inline Chat Tags (Per-Prompt Steering):**
|
| 138 |
-
* `Solve this proof <|think_xhigh|>` (Deep reasoning)
|
| 139 |
-
* `Explain recursion briefly <|think_low|>` (Concise reasoning)
|
| 140 |
-
* `What is the capital of France? <|think_off|>` (No reasoning / fast mode)
|
| 141 |
-
|
| 142 |
-
*(Note: When thinking is disabled, reasoning effort instructions are automatically suppressed).*
|
| 143 |
-
|
| 144 |
-
</details>
|
| 145 |
-
|
| 146 |
-
<details>
|
| 147 |
-
<summary><b>2. KV Cache Preservation (`preserve_reasoning` & `preserve_thinking`)</b></summary>
|
| 148 |
-
|
| 149 |
-
By default, this template **preserves** all past `<think>` blocks in the chat history. This prevents the model from suffering "amnesia stalls" during complex agentic loops and guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
| 150 |
-
|
| 151 |
-
* On recent `llama.cpp` builds, pass `--reasoning-preserve` directly.
|
| 152 |
-
* Or pass via template kwargs:
|
| 153 |
-
```json
|
| 154 |
-
{
|
| 155 |
-
"preserve_thinking": true
|
| 156 |
-
}
|
| 157 |
-
```
|
| 158 |
-
|
| 159 |
-
If you are on severely memory-constrained hardware and need to save context tokens, set `"preserve_thinking": false` (or `"preserve_reasoning": false`) to strip past thoughts.
|
| 160 |
-
|
| 161 |
-
</details>
|
| 162 |
-
|
| 163 |
-
<details>
|
| 164 |
-
<summary><b>3. Tool Call Format Override (JSON vs XML)</b></summary>
|
| 165 |
-
|
| 166 |
-
Qwen models are natively trained to output tool calls in XML (`<function=name>`). By default, this template uses native XML to maximize reliability.
|
| 167 |
-
|
| 168 |
-
**When to use the JSON override:**
|
| 169 |
-
If you are using a framework or harness (such as specific Hermes Agent configurations) that strictly requires Hermes JSON (`{"name": "...", "arguments": {...}}`), pass:
|
| 170 |
-
```json
|
| 171 |
-
{
|
| 172 |
-
"tool_call_format": "json"
|
| 173 |
-
}
|
| 174 |
-
```
|
| 175 |
-
*(When opting into JSON format, argument truncation is safely bypassed to avoid corrupting JSON syntax)*.
|
| 176 |
-
|
| 177 |
-
</details>
|
| 178 |
-
|
| 179 |
-
<details>
|
| 180 |
-
<summary><b>4. Dynamic Payload Truncation</b></summary>
|
| 181 |
-
|
| 182 |
-
To prevent oversized tool returns from blowing out context limits:
|
| 183 |
-
* `max_tool_arg_chars` (default `0` / disabled): Slices oversized tool call arguments.
|
| 184 |
-
* `max_tool_response_chars` (default `0` / disabled): Slices oversized tool output data.
|
| 185 |
-
|
| 186 |
-
</details>
|
| 187 |
-
|
| 188 |
-
---
|
| 189 |
-
|
| 190 |
-
<details>
|
| 191 |
-
<summary><b>Diagnostic Script & Test Suite</b></summary>
|
| 192 |
-
|
| 193 |
-
### Check Active Template on Your Model
|
| 194 |
-
Run the included diagnostic utility on your model folder or GGUF:
|
| 195 |
-
```bash
|
| 196 |
-
python3 scripts/check_applied.py /path/to/your/model
|
| 197 |
-
```
|
| 198 |
-
|
| 199 |
-
### Running the Test Suite
|
| 200 |
-
```bash
|
| 201 |
-
python3 scripts/test_v22.py
|
| 202 |
-
```
|
| 203 |
-
Tests cover 34 automated verification cells including `reasoning_effort` levels, monotonic API mappings, inline chat tags, multi-part payloads, tool call serialization, dynamic truncation, error escalation, and multi-turn history parsing.
|
| 204 |
-
|
| 205 |
-
</details>
|
| 206 |
-
|
| 207 |
-
---
|
| 208 |
-
|
| 209 |
-
## Authorship
|
| 210 |
-
| Role | Author |
|
| 211 |
-
|------|--------|
|
| 212 |
-
| Original models | Alibaba Cloud (Qwen team) |
|
| 213 |
-
| Template fixes | [froggeric](https://huggingface.co/froggeric) |
|
| 214 |
-
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 215 |
-
|
| 216 |
-
## License
|
| 217 |
-
Apache-2.0, inherited from Qwen.
|
| 218 |
-
|
| 219 |
-
---
|
| 220 |
-
|
| 221 |
-
<details>
|
| 222 |
-
<summary>Technical Details of the Critical Fixes</summary>
|
| 223 |
-
|
| 224 |
-
### 1. The "Empty Think" Poisoning and Logic Trap Cure
|
| 225 |
-
Previous templates attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic pattern where the model associated empty thoughts with tools, causing an 80%+ premature turn abort rate. We abolished empty think injection and rewrote the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after thinking. In this release, we also cured official Qwen 3.8's history bug where missing in-content parsers created duplicate blank think tags.
|
| 226 |
-
|
| 227 |
-
### 2. Upfront Pre-Scan for Control Tags & Inline Effort Steering
|
| 228 |
-
In Jinja templates, system prompts are assembled before iterating over message history. We introduce a comprehensive upfront pre-scan covering raw strings, string lists, and multi-part content lists (`[{'type': 'text', 'text': '...'}]`). This resolves `<|think_off|>`, `<|think_on|>`, `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` states before the system message is built, ensuring reasoning instructions are never injected into non-reasoning turns and all tags are cleanly stripped during rendering.
|
| 229 |
-
|
| 230 |
-
### 3. KV Cache Safety and Autoregressive Normalization
|
| 231 |
-
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because this template preserves historical thoughts chronologically by default and defaults `reasoning_effort` to `medium` (zero system tokens), rendered history perfectly synchronizes with cached generated tokens. Combined with strict single newline normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn sessions.
|
| 232 |
-
|
| 233 |
-
### 4. Native XML Tool Format and Universal Serialization
|
| 234 |
-
The model was trained with the XML tool format used by Qwen3-Coder. We restored this format natively while bypassing the `|items` crash by handling both mapping dictionaries and JSON strings. This eliminates crashes when standard OpenAI proxies pass stringified arguments.
|
| 235 |
-
|
| 236 |
-
### 5. Two-Tier Agentic Error Escalation
|
| 237 |
-
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter. On the first error, a diagnostic warning is injected. On the second consecutive error, an urgent system warning forces a fundamentally different approach while retaining the reasoning block so the model can plan its correction.
|
| 238 |
-
|
| 239 |
-
### 6. Smart False-Positive Detection
|
| 240 |
-
Instead of broad substring matching that triggers false retry loops on successful database returns containing words like "error", this template utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell echo exclusions (`$ `).
|
| 241 |
-
|
| 242 |
-
### 7. minijinja Compatibility Constraints
|
| 243 |
-
Python-only Jinja2 features crash or misbehave on `minijinja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 244 |
-
* `content | replace('<|think_on|>', '')` became `content.split('<|think_on|>') | join('')` (fixes a bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 245 |
-
* `| items` became `for key in mapping`.
|
| 246 |
-
* `loop.previtem` became explicit array indexing.
|
| 247 |
-
* `map('string')` became `join('|')`.
|
| 248 |
-
* `| first` became `'$ ' in content`.
|
| 249 |
-
|
| 250 |
-
### 8. AST Flattening for C++ Throughput
|
| 251 |
-
Deeply nested Jinja loops and macros create severe parsing bottlenecks in C++ inference engines. We flattened the AST architecture, effectively curing an 80% inference throughput drop on `llama.cpp` by streamlining how `ns_state` tracking and historical rendering loops are evaluated.
|
| 252 |
-
|
| 253 |
-
### 9. Dynamic Payload Truncation
|
| 254 |
-
Massive API or database returns can instantly blow out a model's context window. We implemented `max_tool_arg_chars` and `max_tool_response_chars` limiters that safely slice oversized payloads. Crucially, this truncation is automatically disabled when `tool_call_format="json"` is active, as slicing a serialized JSON string structurally corrupts the data and crashes downstream parsers.
|
| 255 |
-
|
| 256 |
-
### 10. Reasoning Bypass Hallucination Mitigation
|
| 257 |
-
When thinking is disabled, Qwen models often hallucinate reasoning tags due to their training bias. We injected a safe boundary and adjusted the `<IMPORTANT>` system block to remove explicit mentions of `</think>` during tool instructions. This stops the model from hallucinating closing tags when calling tools in a no-reasoning state.
|
| 258 |
-
|
| 259 |
-
</details>
|
| 260 |
-
|
| 261 |
-
---
|
| 262 |
-
|
| 263 |
-
<details>
|
| 264 |
-
<summary>Update History & Changelog</summary>
|
| 265 |
-
|
| 266 |
-
> **2026-08-16 Update (v22.1): The Qwen 3.8 Update (Bounded Reasoning Defaults, Inline Chat Tags, and Diagnostic Utility).**
|
| 267 |
-
> 1. **Full Qwen 3.8 Support:** Single drop-in template covering all Qwen 3.5, 3.6, and 3.8 model variants (`Qwen3.8-2.4T-A95B`, `Qwen3.8-27B`).
|
| 268 |
-
> 2. **Default `reasoning_effort` to `medium`:** Replaced the unsafe `xhigh` default with `medium` (zero injected tokens). Eliminates the runaway reasoning token-burn failure where models explore branches until hitting `max_tokens` with empty content (#72).
|
| 269 |
-
> 3. **Inline Chat Tags:** Added support for `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` tags directly inside chat messages (#70).
|
| 270 |
-
> 4. **API Mapping & Aliasing:** Added case-insensitive alias support for client and serving runtimes (`high`/`max` -> `xhigh`, `minimal` -> `low`, `none` -> thinking off).
|
| 271 |
-
> 5. **Sequential Control Tag Stripping:** Ensured all 7 control tags are cleanly stripped across system, user, and multi-part content without prompt leakage.
|
| 272 |
-
> 6. **Cured Official 3.8 Empty Think Bug:** Fixed a regression in the official Qwen 3.8 template where removing `<think>` extraction caused blank `<think></think>` blocks to be prepended to real thoughts in chat history.
|
| 273 |
-
> 7. **Restored Fast Mode:** Replaced official 3.8's hard exception on `enable_thinking=false`, restoring full user freedom to disable reasoning via kwargs or `<|think_off|>`.
|
| 274 |
-
> 8. **Universal Tool Arguments:** Hardened tool call parsing to handle both dictionary structures and JSON-serialized strings without crashing.
|
| 275 |
-
> 9. **Diagnostic Utility:** Added `scripts/check_applied.py` to inspect model folders and GGUFs for template consistency.
|
| 276 |
-
|
| 277 |
-
> **2026-08-13 Update (v22): Qwen 3.8 Support, Reasoning Effort Controls, and Engine Hardening.**
|
| 278 |
-
|
| 279 |
-
> **2026-07-02 Update (v21.3): Optional JSON Tool Format Kwarg.** Added an optional `tool_call_format="json"` override for `chat_template_kwargs`.
|
| 280 |
-
|
| 281 |
-
> **2026-07-02 Update (v21.2): Reasoning Bypass Hallucination Fix.** Adjusted `<IMPORTANT>` block instructions to remove explicit mentions of `</think>` during tool definitions.
|
| 282 |
-
|
| 283 |
-
> **2026-07-02 Update (v21.1): Reliability Overhaul & XML Revert.** Reverted to native XML format for vLLM `qwen3_coder` compatibility and restored `preserve_thinking` default to `true`.
|
| 284 |
-
|
| 285 |
-
</details>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/README-v22.2.md
DELETED
|
@@ -1,296 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
tags:
|
| 4 |
-
- jinja
|
| 5 |
-
- chat-template
|
| 6 |
-
- qwen
|
| 7 |
-
- qwen3.5
|
| 8 |
-
- qwen3.6
|
| 9 |
-
- qwen3.8
|
| 10 |
-
- lm-studio
|
| 11 |
-
- mlx
|
| 12 |
-
- llama.cpp
|
| 13 |
-
- vllm
|
| 14 |
-
- tool-calling
|
| 15 |
-
- thinking
|
| 16 |
-
---
|
| 17 |
-
|
| 18 |
-
# Fixed jinja chat templates for Qwen 3.5, 3.6 & 3.8 (v22.2)
|
| 19 |
-
|
| 20 |
-
This is a universal drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, empty think poisoning, and fatal agentic stalling across official Qwen chat templates.
|
| 21 |
-
|
| 22 |
-
It works across LM Studio, llama.cpp, vLLM, MLX, oMLX, KoboldCPP, and any engine that supports Hugging Face Jinja templates. You only need the single `chat_template.jinja` file at the root of this repository for all Qwen 3.5, 3.6, and 3.8 model sizes.
|
| 23 |
-
|
| 24 |
-
---
|
| 25 |
-
|
| 26 |
-
## What is New with Qwen 3.8 (v22 / v22.1 / v22.2)
|
| 27 |
-
|
| 28 |
-
Qwen 3.8 introduces prompt-steered reasoning depth and new model architectures (`Qwen3.8-2.4T-A95B`, `Qwen3.8-27B`). While the official template added reasoning effort steering, it also introduced several lockdowns, regressions, and syntax crashes.
|
| 29 |
-
|
| 30 |
-
The v22 generation delivers full Qwen 3.8 support with all official bugs resolved:
|
| 31 |
-
|
| 32 |
-
| Feature / Fix | Official Qwen 3.8 Template | This Template (v22.2) |
|
| 33 |
-
|---|---|---|
|
| 34 |
-
| **Default Reasoning Baseline** | Hardcodes `xhigh` by default, often exhausting the token budget on reasoning before generating answers. | Defaults to `medium` (zero injected tokens), preserving KV cache parity with v21 and preventing empty-content timeouts. |
|
| 35 |
-
| **Non-Reasoning Fast Mode** | Throws a fatal exception if `enable_thinking=false`. | Full non-reasoning mode restored via kwargs (`enable_thinking=false`, `reasoning_effort="none"`) or inline `<\|think_off\|>`. |
|
| 36 |
-
| **History Reasoning Extraction** | Drops in-content thinking extraction, prepending blank `<think></think>` blocks to real thoughts in chat history. | Extracts reasoning across OpenAI (`reasoning_content`), Anthropic (`thinking`), and in-content tags (`<think>`) without tag duplication. |
|
| 37 |
-
| **Tool Argument Wire Format** | Crashes on serialized JSON strings from standard OpenAI API clients. | Canonical XML default with universal argument handling: safely renders stringified JSON arguments in history without syntax crashes or KV cache invalidation. |
|
| 38 |
-
| **Client Reasoning Aliases** | Rejects non-standard effort names. | Maps OpenAI, Claude Code, Cursor, and Cline aliases automatically: `high`, `max`, `ultracode`, `extreme` $\to$ `xhigh`; `minimal` $\to$ `low`; `none`, `off` $\to$ disabled. |
|
| 39 |
-
| **Per-Turn Inline Chat Tags** | Not supported. | Inline steering via chat text: `<\|think_low\|>`, `<\|think_medium\|>`, `<\|think_xhigh\|>`, `<\|think_ultracode\|>`, `<\|think_off\|>`. Tags are stripped before inference. |
|
| 40 |
-
| **Leading System Prompts** | Treats multiple leading system messages as separate turns. | Merges consecutive leading `system` and `developer` messages into a single system turn joined by double newlines. |
|
| 41 |
-
| **Tool Error Recovery** | Not supported or triggers false warnings on search results containing words like "error". | Two-tier error recovery with smart code/grep disambiguation to avoid false warnings on search results like `throw new Error(...)`. |
|
| 42 |
-
| **llama.cpp Flag Support** | Requires manual template kwargs. | Native alias support for `--reasoning-preserve` CLI flag via `preserve_reasoning`. |
|
| 43 |
-
| **Diagnostic Tool** | None provided. | Includes `scripts/check_applied.py` to inspect model directories and GGUFs for active template versions. |
|
| 44 |
-
|
| 45 |
-
---
|
| 46 |
-
|
| 47 |
-
<details open>
|
| 48 |
-
<summary><b>Quick Install & Engine Setup</b></summary>
|
| 49 |
-
|
| 50 |
-
### llama.cpp / llama-server / koboldcpp
|
| 51 |
-
Run `llama-server` with the template file and DeepSeek reasoning format:
|
| 52 |
-
```bash
|
| 53 |
-
llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek
|
| 54 |
-
```
|
| 55 |
-
*Why `--reasoning-format deepseek` matters:* When connecting coding agents like OpenCode, Claude Code, or Pi.dev to `llama-server`, this flag extracts `<think>` blocks into the dedicated `reasoning_content` API response field. This prevents raw thinking tokens from leaking into the text stream and stopping tool calls midway.
|
| 56 |
-
|
| 57 |
-
*Native CLI flag:* On recent `llama.cpp` builds, you can pass `--reasoning-preserve` directly to ensure 100% Prefix KV Cache retention.
|
| 58 |
-
|
| 59 |
-
### LM Studio
|
| 60 |
-
1. Open your Qwen model in the right side panel.
|
| 61 |
-
2. Scroll down to **Prompt Template**.
|
| 62 |
-
3. Replace the template with the contents of `chat_template.jinja`.
|
| 63 |
-
4. Click **Save**.
|
| 64 |
-
|
| 65 |
-
### vLLM
|
| 66 |
-
Replace the `"chat_template"` string in your `tokenizer_config.json` with `chat_template_oneline.txt` (or raw `chat_template.jinja`).
|
| 67 |
-
```bash
|
| 68 |
-
vllm serve Qwen/Qwen3.8-2.4T-A95B --tool-call-parser qwen3_xml
|
| 69 |
-
```
|
| 70 |
-
*Parser selection:* Use `--tool-call-parser qwen3_xml` on current vLLM releases. If you are on an older vLLM build, use `--tool-call-parser qwen3_coder`. If you explicitly set `tool_call_format="json"`, use `--tool-call-parser hermes`.
|
| 71 |
-
|
| 72 |
-
### oMLX / MLX
|
| 73 |
-
Overwrite `chat_template.jinja` in your local model directory and launch with `--jinja`.
|
| 74 |
-
|
| 75 |
-
</details>
|
| 76 |
-
|
| 77 |
-
---
|
| 78 |
-
|
| 79 |
-
## Why you need this
|
| 80 |
-
|
| 81 |
-
The official Qwen templates contain engine restrictions, Python-specific Jinja logic, and regressions that break local inference and agent workflows.
|
| 82 |
-
|
| 83 |
-
<details open>
|
| 84 |
-
<summary><b>Critical Issues Fixed</b></summary>
|
| 85 |
-
|
| 86 |
-
| Area | Issue in Official Templates | The Fix |
|
| 87 |
-
|---|---|---|
|
| 88 |
-
| **Qwen 3.8 Support** | Official 3.8 crashes if `enable_thinking=false`. | **Restored Fast Mode**. Supports fast non-reasoning mode via kwargs or `<\|think_off\|>`. |
|
| 89 |
-
| **Qwen 3.8 Token Safety** | Official `xhigh` default burns token budgets with zero content returned. | **Safe `medium` Default**. Zero prompt injection unless explicitly requested. |
|
| 90 |
-
| **Qwen 3.8 Regression** | Official 3.8 injects duplicate blank `<think></think>` in chat history. | **Cured Empty Think Poisoning**. Robust multi-format reasoning extraction. |
|
| 91 |
-
| **Reasoning Control** | Inability to change reasoning effort in chat interfaces. | **Inline Chat Tags**. Full support for `<\|think_low\|>`, `<\|think_medium\|>`, and `<\|think_xhigh\|>`. |
|
| 92 |
-
| **Compatibility** | `llama.cpp --reasoning-preserve` CLI flag compatibility. | **Native Alias Support**. Supports both `preserve_reasoning` and `preserve_thinking`. |
|
| 93 |
-
| **Compatibility** | JSON-string tool arguments (OpenAI / Ollama) crash official templates. | **Universal Tool Parsing**. Safely handles mappings, JSON strings, and scalar args. |
|
| 94 |
-
| **Agentic Loop** | Model aborts turn when combining conversational text and a tool call. | Cured "Empty Think" poisoning and softened imperative system directives. |
|
| 95 |
-
| **Agentic Loop** | Model gets stuck emitting the identical failing tool call. | Added two-tier error escalation to force correction while retaining reasoning. |
|
| 96 |
-
| **Agentic Loop** | Model panics and debates internal rules after fetching data. | Broadened `<think>` instructions to authorize conversational synthesis. |
|
| 97 |
-
| **Agentic Loop** | API returns containing the word "error" trigger false retry loops. | Replaced broad matching with strict structural guards. |
|
| 98 |
-
| **Performance** | Mutated past turns destroy the prefix cache. | Enforced chronological history for a 100% KV Cache hit rate. |
|
| 99 |
-
| **Performance** | Deep Jinja nesting drops `llama.cpp` speed by 80%. | Flattened the AST architecture to maximize throughput. |
|
| 100 |
-
| **Compatibility** | Python-specific filters crash C++ inference engines. | Rewrote all filters to be 100% `minijinja` safe. |
|
| 101 |
-
| **Compatibility** | Qwen-native parsers (like vLLM) crash on JSON formatting. | Maintained canonical Qwen XML format as the default. |
|
| 102 |
-
| **Compatibility** | Older API setups and wrappers crash on native XML. | Added a `tool_call_format="json"` opt-in override. |
|
| 103 |
-
| **Compatibility** | Anthropic `message.thinking` payloads are rejected. | Added native Anthropic reasoning support. |
|
| 104 |
-
| **Stability** | Massive tool data returns blow out the context window. | Added dynamic payload truncation limits. |
|
| 105 |
-
| **Stability** | Mid-conversation system prompts crash the template. | Added native support for arbitrary system and developer messages. |
|
| 106 |
-
| **Edge Cases** | Text duplicates during streaming generation. | Restored canonical spacing to the generation prompt. |
|
| 107 |
-
| **Edge Cases** | Model hallucinates reasoning tags when thinking is disabled. | Injected strict boundaries to force clean reasoning bypass. |
|
| 108 |
-
|
| 109 |
-
</details>
|
| 110 |
-
|
| 111 |
-
---
|
| 112 |
-
|
| 113 |
-
## Customization & Kwarg Reference
|
| 114 |
-
|
| 115 |
-
<details open>
|
| 116 |
-
<summary><b>1. Reasoning Effort Steering (Qwen 3.8)</b></summary>
|
| 117 |
-
|
| 118 |
-
Qwen 3.8 has 3 native prompt-steered reasoning levels. You can control this via template kwargs or directly inline in your chat messages:
|
| 119 |
-
|
| 120 |
-
**Via Template Kwargs (`chat_template_kwargs`):**
|
| 121 |
-
```json
|
| 122 |
-
{
|
| 123 |
-
"reasoning_effort": "xhigh"
|
| 124 |
-
}
|
| 125 |
-
```
|
| 126 |
-
* **`"medium"` (Default / Safe Baseline):** No extra instruction text injected. Preserves 100% Prefix KV Cache parity with v21 and lets the model reason naturally without token-budget traps.
|
| 127 |
-
* **`"xhigh"` (Deep Reasoning):** Injects Qwen's official deep reasoning instruction:
|
| 128 |
-
> *"Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."*
|
| 129 |
-
* **`"low"` (Concise Reasoning):** Injects concise thinking instructions for fast, summary-oriented reasoning:
|
| 130 |
-
> *"Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."*
|
| 131 |
-
|
| 132 |
-
*API Compatibility Aliases:*
|
| 133 |
-
To prevent errors when calling the model through standard API proxies and coding agent harnesses:
|
| 134 |
-
* `"high"`, `"max"`, `"ultracode"`, and `"extreme"` automatically map to `"xhigh"` (supported by OpenAI, Claude Code, Cline, Cursor).
|
| 135 |
-
* `"minimal"` and `"low"` automatically map to `"low"`.
|
| 136 |
-
* `"none"` and `"off"` disable thinking entirely.
|
| 137 |
-
|
| 138 |
-
**Via Inline Chat Tags (Per-Prompt Steering):**
|
| 139 |
-
* `Solve this proof <|think_xhigh|>` or `<|think_ultracode|>` (Deep reasoning)
|
| 140 |
-
* `Explain recursion briefly <|think_low|>` (Concise reasoning)
|
| 141 |
-
* `What is the capital of France? <|think_off|>` (No reasoning / fast mode)
|
| 142 |
-
|
| 143 |
-
*(Note: When thinking is disabled, reasoning effort instructions are automatically suppressed).*
|
| 144 |
-
|
| 145 |
-
</details>
|
| 146 |
-
|
| 147 |
-
<details>
|
| 148 |
-
<summary><b>2. KV Cache Preservation (`preserve_reasoning` & `preserve_thinking`)</b></summary>
|
| 149 |
-
|
| 150 |
-
By default, this template **preserves** all past `<think>` blocks in the chat history. This prevents the model from suffering "amnesia stalls" during complex agentic loops and guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
| 151 |
-
|
| 152 |
-
* On recent `llama.cpp` builds, pass `--reasoning-preserve` directly.
|
| 153 |
-
* Or pass via template kwargs:
|
| 154 |
-
```json
|
| 155 |
-
{
|
| 156 |
-
"preserve_thinking": true
|
| 157 |
-
}
|
| 158 |
-
```
|
| 159 |
-
|
| 160 |
-
If you are on severely memory-constrained hardware and need to save context tokens, set `"preserve_thinking": false` (or `"preserve_reasoning": false`) to strip past thoughts.
|
| 161 |
-
|
| 162 |
-
</details>
|
| 163 |
-
|
| 164 |
-
<details>
|
| 165 |
-
<summary><b>3. Tool Call Format (XML vs JSON)</b></summary>
|
| 166 |
-
|
| 167 |
-
Qwen models are natively trained on XML tool calls (`<function=name>`). By default, this template uses **`xml`** format:
|
| 168 |
-
* **`"xml"` (Default):** Generates canonical XML instructions for tool calls and safely handles both dictionary arguments and serialized JSON strings in assistant history without corrupting XML tags or mutating system prompts between turns.
|
| 169 |
-
* **`"json"` (Optional Override):** Forces Hermes JSON format (`{"name": "...", "arguments": {...}}`) for both system prompt instructions and history rendering.
|
| 170 |
-
|
| 171 |
-
**When to use the JSON override:**
|
| 172 |
-
If you are using a framework or harness (such as specific Hermes Agent configurations) that strictly requires Hermes JSON, pass:
|
| 173 |
-
```json
|
| 174 |
-
{
|
| 175 |
-
"tool_call_format": "json"
|
| 176 |
-
}
|
| 177 |
-
```
|
| 178 |
-
*(When opting into JSON format, argument truncation is safely bypassed to avoid corrupting JSON syntax)*.
|
| 179 |
-
|
| 180 |
-
</details>
|
| 181 |
-
|
| 182 |
-
<details>
|
| 183 |
-
<summary><b>4. Dynamic Payload Truncation</b></summary>
|
| 184 |
-
|
| 185 |
-
To prevent oversized tool returns from blowing out context limits:
|
| 186 |
-
* `max_tool_arg_chars` (default `0` / disabled): Slices oversized tool call arguments.
|
| 187 |
-
* `max_tool_response_chars` (default `0` / disabled): Slices oversized tool output data.
|
| 188 |
-
|
| 189 |
-
</details>
|
| 190 |
-
|
| 191 |
-
---
|
| 192 |
-
|
| 193 |
-
<details>
|
| 194 |
-
<summary><b>Diagnostic Script & Test Suite</b></summary>
|
| 195 |
-
|
| 196 |
-
### Check Active Template on Your Model
|
| 197 |
-
Run the included diagnostic utility on your model folder or GGUF:
|
| 198 |
-
```bash
|
| 199 |
-
python3 scripts/check_applied.py /path/to/your/model
|
| 200 |
-
```
|
| 201 |
-
|
| 202 |
-
### Running the Test Suite
|
| 203 |
-
```bash
|
| 204 |
-
python3 scripts/test_v22.py
|
| 205 |
-
```
|
| 206 |
-
Tests cover 44 automated verification cells including `reasoning_effort` levels, monotonic API mappings, inline chat tags, multi-part payloads, tool call serialization, dynamic truncation, error escalation, multi-turn history parsing, and KV cache stability.
|
| 207 |
-
|
| 208 |
-
</details>
|
| 209 |
-
|
| 210 |
-
---
|
| 211 |
-
|
| 212 |
-
## Authorship
|
| 213 |
-
| Role | Author |
|
| 214 |
-
|------|--------|
|
| 215 |
-
| Original models | Alibaba Cloud (Qwen team) |
|
| 216 |
-
| Template fixes | [froggeric](https://huggingface.co/froggeric) |
|
| 217 |
-
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 218 |
-
|
| 219 |
-
## License
|
| 220 |
-
Apache-2.0, inherited from Qwen.
|
| 221 |
-
|
| 222 |
-
---
|
| 223 |
-
|
| 224 |
-
<details>
|
| 225 |
-
<summary>Technical Details of the Critical Fixes</summary>
|
| 226 |
-
|
| 227 |
-
### 1. The "Empty Think" Poisoning and Logic Trap Cure
|
| 228 |
-
Previous templates attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic pattern where the model associated empty thoughts with tools, causing an 80%+ premature turn abort rate. We abolished empty think injection and rewrote the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after thinking. In this release, we also cured official Qwen 3.8's history bug where missing in-content parsers created duplicate blank think tags.
|
| 229 |
-
|
| 230 |
-
### 2. Upfront Pre-Scan for Control Tags & Inline Effort Steering
|
| 231 |
-
In Jinja templates, system prompts are assembled before iterating over message history. We introduce a comprehensive upfront pre-scan covering raw strings, string lists, and multi-part content lists (`[{'type': 'text', 'text': '...'}]`). This resolves `<|think_off|>`, `<|think_on|>`, `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` states before the system message is built, ensuring reasoning instructions are never injected into non-reasoning turns and all tags are cleanly stripped during rendering.
|
| 232 |
-
|
| 233 |
-
### 3. KV Cache Safety and Autoregressive Normalization
|
| 234 |
-
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because this template preserves historical thoughts chronologically by default and defaults `reasoning_effort` to `medium` (zero system tokens), rendered history perfectly synchronizes with cached generated tokens. Combined with strict single newline normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn sessions.
|
| 235 |
-
|
| 236 |
-
### 4. Native XML Tool Format and Universal Serialization
|
| 237 |
-
The model was trained with the XML tool format used by Qwen3-Coder. We restored this format natively while bypassing the `|items` crash by handling both mapping dictionaries and JSON strings. This eliminates crashes when standard OpenAI proxies pass stringified arguments.
|
| 238 |
-
|
| 239 |
-
### 5. Two-Tier Agentic Error Escalation
|
| 240 |
-
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter. On the first error, a diagnostic warning is injected. On the second consecutive error, an urgent system warning forces a fundamentally different approach while retaining the reasoning block so the model can plan its correction.
|
| 241 |
-
|
| 242 |
-
### 6. Smart False-Positive Detection
|
| 243 |
-
Instead of broad substring matching that triggers false retry loops on successful database returns containing words like "error", this template utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell echo exclusions (`$ `).
|
| 244 |
-
|
| 245 |
-
### 7. minijinja Compatibility Constraints
|
| 246 |
-
Python-only Jinja2 features crash or misbehave on `minijinja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 247 |
-
* `content | replace('<|think_on|>', '')` became `content.split('<|think_on|>') | join('')` (fixes a bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 248 |
-
* `| items` became `for key in mapping`.
|
| 249 |
-
* `loop.previtem` became explicit array indexing.
|
| 250 |
-
* `map('string')` became `join('|')`.
|
| 251 |
-
* `| first` became `'$ ' in content`.
|
| 252 |
-
|
| 253 |
-
### 8. AST Flattening for C++ Throughput
|
| 254 |
-
Deeply nested Jinja loops and macros create severe parsing bottlenecks in C++ inference engines. We flattened the AST architecture, effectively curing an 80% inference throughput drop on `llama.cpp` by streamlining how `ns_state` tracking and historical rendering loops are evaluated.
|
| 255 |
-
|
| 256 |
-
### 9. Dynamic Payload Truncation
|
| 257 |
-
Massive API or database returns can instantly blow out a model's context window. We implemented `max_tool_arg_chars` and `max_tool_response_chars` limiters that safely slice oversized payloads. Crucially, this truncation is automatically disabled when `tool_call_format="json"` is active, as slicing a serialized JSON string structurally corrupts the data and crashes downstream parsers.
|
| 258 |
-
|
| 259 |
-
### 10. Reasoning Bypass Hallucination Mitigation
|
| 260 |
-
When thinking is disabled, Qwen models often hallucinate reasoning tags due to their training bias. We injected a safe boundary and adjusted the `<IMPORTANT>` system block to remove explicit mentions of `</think>` during tool instructions. This stops the model from hallucinating closing tags when calling tools in a no-reasoning state.
|
| 261 |
-
|
| 262 |
-
</details>
|
| 263 |
-
|
| 264 |
-
---
|
| 265 |
-
|
| 266 |
-
<details>
|
| 267 |
-
<summary>Update History & Changelog</summary>
|
| 268 |
-
|
| 269 |
-
> **2026-08-19 Update (v22.2): Extended Effort Aliases, String Argument Safety, and Error Disambiguation.**
|
| 270 |
-
> 1. **Universal Tool Argument Handling:** Hardened XML tool argument parsing to safely handle both mappings and serialized JSON strings in assistant history without syntax crashes or KV cache mutation.
|
| 271 |
-
> 2. **Reasoning Effort Aliases:** Added `"ultracode"` and `"extreme"` mappings to `'xhigh'`, with inline `<|think_ultracode|>` and `<|think_extreme|>` support (#78).
|
| 272 |
-
> 3. **Multi-System Message Merging:** Merged consecutive leading `system` and `developer` messages into a single unified system turn with `\n\n`.
|
| 273 |
-
> 4. **Grep / Search Error Disambiguation:** Eliminated false-positive tool error warnings on code search results containing patterns like `throw new Error` or `console.error` (#66).
|
| 274 |
-
> 5. **Safe XML Primitive Serialization:** Boolean and null XML parameter values now serialize to `true`, `false`, and `null` via `| tojson`.
|
| 275 |
-
> 6. **Zero-Crash Resilience:** Maintained full resilience without fatal `raise_exception` aborts across all runtime engines.
|
| 276 |
-
|
| 277 |
-
> **2026-08-16 Update (v22.1): The Qwen 3.8 Update (Bounded Reasoning Defaults, Inline Chat Tags, and Diagnostic Utility).**
|
| 278 |
-
> 1. **Full Qwen 3.8 Support:** Single drop-in template covering all Qwen 3.5, 3.6, and 3.8 model variants (`Qwen3.8-2.4T-A95B`, `Qwen3.8-27B`).
|
| 279 |
-
> 2. **Default `reasoning_effort` to `medium`:** Replaced the unsafe `xhigh` default with `medium` (zero injected tokens). Eliminates the runaway reasoning token-burn failure where models explore branches until hitting `max_tokens` with empty content (#72).
|
| 280 |
-
> 3. **Inline Chat Tags:** Added support for `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` tags directly inside chat messages (#70).
|
| 281 |
-
> 4. **API Mapping & Aliasing:** Added case-insensitive alias support for client and serving runtimes (`high`/`max` -> `xhigh`, `minimal` -> `low`, `none` -> thinking off).
|
| 282 |
-
> 5. **Sequential Control Tag Stripping:** Ensured all 7 control tags are cleanly stripped across system, user, and multi-part content without prompt leakage.
|
| 283 |
-
> 6. **Cured Official 3.8 Empty Think Bug:** Fixed a regression in the official Qwen 3.8 template where removing `<think>` extraction caused blank `<think></think>` blocks to be prepended to real thoughts in chat history.
|
| 284 |
-
> 7. **Restored Fast Mode:** Replaced official 3.8's hard exception on `enable_thinking=false`, restoring full user freedom to disable reasoning via kwargs or `<|think_off|>`.
|
| 285 |
-
> 8. **Universal Tool Arguments:** Hardened tool call parsing to handle both dictionary structures and JSON-serialized strings without crashing.
|
| 286 |
-
> 9. **Diagnostic Utility:** Added `scripts/check_applied.py` to inspect model folders and GGUFs for template consistency.
|
| 287 |
-
|
| 288 |
-
> **2026-08-13 Update (v22): Qwen 3.8 Support, Reasoning Effort Controls, and Engine Hardening.**
|
| 289 |
-
|
| 290 |
-
> **2026-07-02 Update (v21.3): Optional JSON Tool Format Kwarg.** Added an optional `tool_call_format="json"` override for `chat_template_kwargs`.
|
| 291 |
-
|
| 292 |
-
> **2026-07-02 Update (v21.2): Reasoning Bypass Hallucination Fix.** Adjusted `<IMPORTANT>` block instructions to remove explicit mentions of `</think>` during tool definitions.
|
| 293 |
-
|
| 294 |
-
> **2026-07-02 Update (v21.1): Reliability Overhaul & XML Revert.** Reverted to native XML format for vLLM `qwen3_coder` compatibility and restored `preserve_thinking` default to `true`.
|
| 295 |
-
|
| 296 |
-
</details>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/README-v22.3.md
DELETED
|
@@ -1,317 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
tags:
|
| 4 |
-
- jinja
|
| 5 |
-
- chat-template
|
| 6 |
-
- qwen
|
| 7 |
-
- qwen3.5
|
| 8 |
-
- qwen3.6
|
| 9 |
-
- qwen3.8
|
| 10 |
-
- lm-studio
|
| 11 |
-
- mlx
|
| 12 |
-
- llama.cpp
|
| 13 |
-
- vllm
|
| 14 |
-
- tool-calling
|
| 15 |
-
- thinking
|
| 16 |
-
---
|
| 17 |
-
|
| 18 |
-
# Fixed jinja chat templates for Qwen 3.5, 3.6 & 3.8 (v22.3)
|
| 19 |
-
|
| 20 |
-
This is a universal drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, empty think poisoning, and fatal agentic stalling across official Qwen chat templates.
|
| 21 |
-
|
| 22 |
-
It works across LM Studio, llama.cpp, vLLM, MLX, oMLX, KoboldCPP, and any engine that supports Hugging Face Jinja templates. You only need the single `chat_template.jinja` file at the root of this repository for all Qwen 3.5, 3.6, and 3.8 model sizes.
|
| 23 |
-
|
| 24 |
-
---
|
| 25 |
-
|
| 26 |
-
## What is New with Qwen 3.8 (v22 / v22.1 / v22.2 / v22.3)
|
| 27 |
-
|
| 28 |
-
Qwen 3.8 introduces prompt-steered reasoning depth and new model architectures (`Qwen3.8-2.4T-A95B`, `Qwen3.8-27B`). While the official template added reasoning effort steering, it also introduced several lockdowns, regressions, and syntax crashes.
|
| 29 |
-
|
| 30 |
-
The v22 generation delivers full Qwen 3.8 support with all official bugs resolved:
|
| 31 |
-
|
| 32 |
-
| Feature / Fix | Official Qwen 3.8 Template | This Template (v22.3) |
|
| 33 |
-
|---|---|---|
|
| 34 |
-
| **Default Reasoning Baseline** | Hardcodes `xhigh` by default, often exhausting the token budget on reasoning before generating answers. | Defaults to `medium` (zero injected tokens), preserving KV cache parity with v21 and preventing empty-content timeouts. |
|
| 35 |
-
| **Non-Reasoning Fast Mode** | Throws a fatal exception if `enable_thinking=false`. | Full non-reasoning mode restored via kwargs (`enable_thinking=false`, `reasoning_effort="none"`) or inline `<\|think_off\|>`. |
|
| 36 |
-
| **History Reasoning Extraction** | Drops in-content thinking extraction, prepending blank `<think></think>` blocks to real thoughts in chat history. | Extracts reasoning across OpenAI (`reasoning_content`), Anthropic (`thinking`), and in-content tags (`<think>`) without tag duplication. |
|
| 37 |
-
| **Tool Argument Wire Format** | Crashes on serialized JSON strings from standard OpenAI API clients. | Canonical XML default with universal argument handling: safely renders stringified JSON arguments in history without syntax crashes or KV cache invalidation. |
|
| 38 |
-
| **Client Reasoning Aliases** | Rejects non-standard effort names. | Maps OpenAI, Claude Code, Cursor, and Cline aliases automatically: `high`, `max`, `ultracode`, `extreme` $\to$ `xhigh`; `minimal` $\to$ `low`; `none`, `off` $\to$ disabled. |
|
| 39 |
-
| **Per-Turn Inline Chat Tags** | Not supported. | Inline steering via chat text: `<\|think_low\|>`, `<\|think_medium\|>`, `<\|think_xhigh\|>`, `<\|think_ultracode\|>`, `<\|think_off\|>`. Tags are stripped before inference. |
|
| 40 |
-
| **Leading System Prompts** | Treats multiple leading system messages as separate turns. | Merges consecutive leading `system` and `developer` messages into a single system turn joined by double newlines. |
|
| 41 |
-
| **Tool Error Recovery** | Not supported or triggers false warnings on search results containing words like "error". | Two-tier error recovery with smart code/grep disambiguation to avoid false warnings on search results like `throw new Error(...)`. Structural signals escalate on payloads of any size. |
|
| 42 |
-
| **llama.cpp Flag Support** | Requires manual template kwargs. | Native alias support for `--reasoning-preserve` CLI flag via `preserve_reasoning`. |
|
| 43 |
-
| **Diagnostic Tool** | None provided. | Includes `scripts/check_applied.py` to inspect model directories and GGUFs for active template versions. |
|
| 44 |
-
|
| 45 |
-
---
|
| 46 |
-
|
| 47 |
-
<details open>
|
| 48 |
-
<summary><b>Quick Install & Engine Setup</b></summary>
|
| 49 |
-
|
| 50 |
-
### llama.cpp / llama-server / koboldcpp
|
| 51 |
-
Run `llama-server` with the template file and DeepSeek reasoning format:
|
| 52 |
-
```bash
|
| 53 |
-
llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek
|
| 54 |
-
```
|
| 55 |
-
*Why `--reasoning-format deepseek` matters:* When connecting coding agents like OpenCode, Claude Code, or Pi.dev to `llama-server`, this flag extracts `<think>` blocks into the dedicated `reasoning_content` API response field. This prevents raw thinking tokens from leaking into the text stream and stopping tool calls midway.
|
| 56 |
-
|
| 57 |
-
*Native CLI flag:* On recent `llama.cpp` builds, you can pass `--reasoning-preserve` directly for 100% Prefix KV Cache retention.
|
| 58 |
-
|
| 59 |
-
### LM Studio
|
| 60 |
-
1. Open your Qwen model in the right side panel.
|
| 61 |
-
2. Scroll down to **Prompt Template**.
|
| 62 |
-
3. Replace the template with the contents of `chat_template.jinja`.
|
| 63 |
-
4. Click **Save**.
|
| 64 |
-
|
| 65 |
-
### vLLM
|
| 66 |
-
Replace the `"chat_template"` string in your `tokenizer_config.json` with `chat_template_oneline.txt` (or raw `chat_template.jinja`).
|
| 67 |
-
```bash
|
| 68 |
-
vllm serve Qwen/Qwen3.8-2.4T-A95B --tool-call-parser qwen3_xml
|
| 69 |
-
```
|
| 70 |
-
*Parser selection:* Use `--tool-call-parser qwen3_xml` on current vLLM releases. If you are on an older vLLM build, use `--tool-call-parser qwen3_coder`. If you explicitly set `tool_call_format="json"`, use `--tool-call-parser hermes`.
|
| 71 |
-
|
| 72 |
-
### oMLX / MLX
|
| 73 |
-
Overwrite `chat_template.jinja` in your local model directory and launch with `--jinja`.
|
| 74 |
-
|
| 75 |
-
</details>
|
| 76 |
-
|
| 77 |
-
---
|
| 78 |
-
|
| 79 |
-
## Why you need this
|
| 80 |
-
|
| 81 |
-
The official Qwen templates contain engine restrictions, Python-specific Jinja logic, and regressions that break local inference and agent workflows.
|
| 82 |
-
|
| 83 |
-
<details open>
|
| 84 |
-
<summary><b>Critical Issues Fixed</b></summary>
|
| 85 |
-
|
| 86 |
-
| Area | Issue in Official Templates | The Fix |
|
| 87 |
-
|---|---|---|
|
| 88 |
-
| **Qwen 3.8 Support** | Official 3.8 crashes if `enable_thinking=false`. | **Restored Fast Mode**. Supports fast non-reasoning mode via kwargs or `<\|think_off\|>`. |
|
| 89 |
-
| **Qwen 3.8 Token Safety** | Official `xhigh` default burns token budgets with zero content returned. | **Safe `medium` Default**. Zero prompt injection unless explicitly requested. |
|
| 90 |
-
| **Qwen 3.8 Regression** | Official 3.8 injects duplicate blank `<think></think>` in chat history. | **Cured Empty Think Poisoning**. Multi-format reasoning extraction. |
|
| 91 |
-
| **Reasoning Control** | Inability to change reasoning effort in chat interfaces. | **Inline Chat Tags**. Full support for `<\|think_low\|>`, `<\|think_medium\|>`, and `<\|think_xhigh\|>`. |
|
| 92 |
-
| **Compatibility** | `llama.cpp --reasoning-preserve` CLI flag compatibility. | **Native Alias Support**. Supports both `preserve_reasoning` and `preserve_thinking`. |
|
| 93 |
-
| **Compatibility** | JSON-string tool arguments (OpenAI / Ollama) crash official templates. | **Universal Tool Parsing**. Safely handles mappings, JSON strings, and scalar args. |
|
| 94 |
-
| **Agentic Loop** | Model aborts turn when combining conversational text and a tool call. | Cured "Empty Think" poisoning and softened imperative system directives. |
|
| 95 |
-
| **Agentic Loop** | Model gets stuck emitting the identical failing tool call. | Added two-tier error escalation to force correction while retaining reasoning. |
|
| 96 |
-
| **Agentic Loop** | Model panics and debates internal rules after fetching data. | Broadened `<think>` instructions to authorize conversational synthesis. |
|
| 97 |
-
| **Agentic Loop** | API returns containing the word "error" trigger false retry loops. | Replaced broad matching with strict structural guards. |
|
| 98 |
-
| **Performance** | Mutated past turns destroy the prefix cache. | Enforced chronological history for a 100% KV Cache hit rate. |
|
| 99 |
-
| **Performance** | Deep Jinja nesting drops `llama.cpp` speed by 80%. | Flattened the AST architecture to maximize throughput. |
|
| 100 |
-
| **Compatibility** | Python-specific filters crash C++ inference engines. | Rewrote all filters to be 100% `minijinja` safe. |
|
| 101 |
-
| **Compatibility** | Qwen-native parsers (like vLLM) crash on JSON formatting. | Maintained canonical Qwen XML format as the default. |
|
| 102 |
-
| **Compatibility** | Older API setups and wrappers crash on native XML. | Added a `tool_call_format="json"` opt-in override. |
|
| 103 |
-
| **Compatibility** | Anthropic `message.thinking` payloads are rejected. | Added native Anthropic reasoning support. |
|
| 104 |
-
| **Stability** | Massive tool data returns blow out the context window. | Added dynamic payload truncation limits. |
|
| 105 |
-
| **Stability** | Mid-conversation system prompts crash the template. | Added native support for arbitrary system and developer messages. |
|
| 106 |
-
| **Edge Cases** | Text duplicates during streaming generation. | Restored canonical spacing to the generation prompt. |
|
| 107 |
-
| **Edge Cases** | Model hallucinates reasoning tags when thinking is disabled. | Injected strict boundaries to force clean reasoning bypass. |
|
| 108 |
-
|
| 109 |
-
</details>
|
| 110 |
-
|
| 111 |
-
---
|
| 112 |
-
|
| 113 |
-
## Customization & Kwarg Reference
|
| 114 |
-
|
| 115 |
-
<details open>
|
| 116 |
-
<summary><b>1. Reasoning Effort Steering (Qwen 3.8)</b></summary>
|
| 117 |
-
|
| 118 |
-
Qwen 3.8 has 3 native prompt-steered reasoning levels. You can control this via template kwargs or directly inline in your chat messages:
|
| 119 |
-
|
| 120 |
-
**Via Template Kwargs (`chat_template_kwargs`):**
|
| 121 |
-
```json
|
| 122 |
-
{
|
| 123 |
-
"reasoning_effort": "xhigh"
|
| 124 |
-
}
|
| 125 |
-
```
|
| 126 |
-
* **`"medium"` (Default / Safe Baseline):** No extra instruction text injected. Preserves 100% Prefix KV Cache parity with v21 and lets the model reason naturally without token-budget traps.
|
| 127 |
-
* **`"xhigh"` (Deep Reasoning):** Injects Qwen's official deep reasoning instruction:
|
| 128 |
-
> *"Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."*
|
| 129 |
-
* **`"low"` (Concise Reasoning):** Injects concise thinking instructions for fast, summary-oriented reasoning:
|
| 130 |
-
> *"Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration."*
|
| 131 |
-
|
| 132 |
-
*API Compatibility Aliases:*
|
| 133 |
-
To prevent errors when calling the model through standard API proxies and coding agent harnesses:
|
| 134 |
-
* `"high"`, `"max"`, `"ultracode"`, and `"extreme"` automatically map to `"xhigh"` (supported by OpenAI, Claude Code, Cline, Cursor).
|
| 135 |
-
* `"minimal"` and `"low"` automatically map to `"low"`.
|
| 136 |
-
* `"none"` and `"off"` disable thinking entirely.
|
| 137 |
-
|
| 138 |
-
**Via Inline Chat Tags (Per-Prompt Steering):**
|
| 139 |
-
* `Solve this proof <|think_xhigh|>` or `<|think_ultracode|>` (Deep reasoning)
|
| 140 |
-
* `Explain recursion briefly <|think_low|>` (Concise reasoning)
|
| 141 |
-
* `What is the capital of France? <|think_off|>` (No reasoning / fast mode)
|
| 142 |
-
|
| 143 |
-
*(Note: When thinking is disabled, reasoning effort instructions are automatically suppressed).*
|
| 144 |
-
|
| 145 |
-
</details>
|
| 146 |
-
|
| 147 |
-
<details>
|
| 148 |
-
<summary><b>2. KV Cache Preservation (`preserve_reasoning` & `preserve_thinking`)</b></summary>
|
| 149 |
-
|
| 150 |
-
By default, this template **preserves** all past `<think>` blocks in the chat history. This prevents the model from suffering "amnesia stalls" during complex agentic loops and guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
| 151 |
-
|
| 152 |
-
* On recent `llama.cpp` builds, pass `--reasoning-preserve` directly.
|
| 153 |
-
* Or pass via template kwargs:
|
| 154 |
-
```json
|
| 155 |
-
{
|
| 156 |
-
"preserve_thinking": true
|
| 157 |
-
}
|
| 158 |
-
```
|
| 159 |
-
|
| 160 |
-
If you are on severely memory-constrained hardware and need to save context tokens, set `"preserve_thinking": false` (or `"preserve_reasoning": false`) to strip past thoughts.
|
| 161 |
-
|
| 162 |
-
> **KV cache trade-off:** `preserve_thinking: false` keeps reasoning only for the turns that follow the most recent user query, so historical thoughts are dropped as soon as a new user turn begins. That rewrites already-rendered history and invalidates the prefix cache from the first assistant turn of the previous segment. The 100% Prefix KV Cache hit rate applies to the default (`true`) setting; enable stripping only when context pressure outweighs prompt reprocessing cost.
|
| 163 |
-
|
| 164 |
-
</details>
|
| 165 |
-
|
| 166 |
-
<details>
|
| 167 |
-
<summary><b>3. Tool Call Format (XML vs JSON)</b></summary>
|
| 168 |
-
|
| 169 |
-
Qwen models are natively trained on XML tool calls (`<function=name>`). By default, this template uses **`xml`** format:
|
| 170 |
-
* **`"xml"` (Default):** Generates canonical XML instructions for tool calls and safely handles both dictionary arguments and serialized JSON strings in assistant history without corrupting XML tags or mutating system prompts between turns.
|
| 171 |
-
* **`"json"` (Optional Override):** Forces Hermes JSON format (`{"name": "...", "arguments": {...}}`) for both system prompt instructions and history rendering.
|
| 172 |
-
|
| 173 |
-
**When to use the JSON override:**
|
| 174 |
-
If you are using a framework or harness (such as specific Hermes Agent configurations) that strictly requires Hermes JSON, pass:
|
| 175 |
-
```json
|
| 176 |
-
{
|
| 177 |
-
"tool_call_format": "json"
|
| 178 |
-
}
|
| 179 |
-
```
|
| 180 |
-
*(When opting into JSON format, both argument and tool response truncation are bypassed to avoid corrupting JSON syntax)*.
|
| 181 |
-
|
| 182 |
-
</details>
|
| 183 |
-
|
| 184 |
-
<details>
|
| 185 |
-
<summary><b>4. Dynamic Payload Truncation</b></summary>
|
| 186 |
-
|
| 187 |
-
To prevent oversized tool returns from blowing out context limits:
|
| 188 |
-
* `max_tool_arg_chars` (default `0` / disabled): Slices oversized tool call arguments. Applies to mapping arguments and to serialized JSON string arguments alike.
|
| 189 |
-
* `max_tool_response_chars` (default `0` / disabled): Slices oversized tool output data.
|
| 190 |
-
|
| 191 |
-
Both limiters are disabled automatically when `tool_call_format="json"` is active, since slicing serialized JSON corrupts the payload for downstream parsers.
|
| 192 |
-
|
| 193 |
-
</details>
|
| 194 |
-
|
| 195 |
-
---
|
| 196 |
-
|
| 197 |
-
<details>
|
| 198 |
-
<summary><b>Diagnostic Script & Test Suite</b></summary>
|
| 199 |
-
|
| 200 |
-
### Check Active Template on Your Model
|
| 201 |
-
Run the included diagnostic utility on your model folder or GGUF:
|
| 202 |
-
```bash
|
| 203 |
-
python3 scripts/check_applied.py /path/to/your/model
|
| 204 |
-
```
|
| 205 |
-
|
| 206 |
-
### Running the Test Suite
|
| 207 |
-
```bash
|
| 208 |
-
python3 scripts/test_v22.py
|
| 209 |
-
```
|
| 210 |
-
Tests cover 101 automated verification cells including `reasoning_effort` levels, monotonic API mappings, inline chat tags, multi-part and vision payloads, tool call serialization across wire formats, dynamic truncation, error escalation precision, malformed input handling, multi-turn history parsing, prefix KV cache stability, and parity between `chat_template.jinja` and `chat_template_oneline.txt`. Set `QWEN_TEMPLATE_FILE=chat_template_oneline.txt` to run the entire suite against the minified build. Test 101 is a deterministic property fuzzer (`scripts/fuzz_template.py`) asserting nine structural invariants over generated conversations; run it standalone with `python3 scripts/fuzz_template.py --cases 2000`.
|
| 211 |
-
|
| 212 |
-
</details>
|
| 213 |
-
|
| 214 |
-
---
|
| 215 |
-
|
| 216 |
-
## Authorship
|
| 217 |
-
| Role | Author |
|
| 218 |
-
|------|--------|
|
| 219 |
-
| Original models | Alibaba Cloud (Qwen team) |
|
| 220 |
-
| Template fixes | [Frédéric Guigand](https://huggingface.co/froggeric) (`@froggeric`) |
|
| 221 |
-
| Property fuzzer, error tiering & test suite | [Juan Calderon-Perez](https://huggingface.co/g-a-b-y) (`@g-a-b-y`) |
|
| 222 |
-
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 223 |
-
|
| 224 |
-
## License
|
| 225 |
-
Apache-2.0, inherited from Qwen.
|
| 226 |
-
|
| 227 |
-
---
|
| 228 |
-
|
| 229 |
-
<details>
|
| 230 |
-
<summary>Technical Details of the Critical Fixes</summary>
|
| 231 |
-
|
| 232 |
-
### 1. The "Empty Think" Poisoning and Logic Trap Cure
|
| 233 |
-
Previous templates attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic pattern where the model associated empty thoughts with tools, causing an 80%+ premature turn abort rate. We abolished empty think injection and rewrote the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after thinking. In this release, we also cured official Qwen 3.8's history bug where missing in-content parsers created duplicate blank think tags.
|
| 234 |
-
|
| 235 |
-
### 2. Upfront Pre-Scan for Control Tags & Inline Effort Steering
|
| 236 |
-
In Jinja templates, system prompts are assembled before iterating over message history. We introduce an upfront pre-scan covering raw strings, string lists, and multi-part content lists (`[{'type': 'text', 'text': '...'}]`). This resolves `<|think_off|>`, `<|think_on|>`, `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` states before the system message is built, preventing reasoning instructions from being injected into non-reasoning turns and cleanly stripping all tags during rendering.
|
| 237 |
-
|
| 238 |
-
### 3. KV Cache Safety and Autoregressive Normalization
|
| 239 |
-
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because this template preserves historical thoughts chronologically by default and defaults `reasoning_effort` to `medium` (zero system tokens), rendered history perfectly synchronizes with cached generated tokens. Combined with strict single newline normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn sessions.
|
| 240 |
-
|
| 241 |
-
### 4. Native XML Tool Format and Universal Serialization
|
| 242 |
-
The model was trained with the XML tool format used by Qwen3-Coder. We restored this format natively while bypassing the `|items` crash by handling both mapping dictionaries and JSON strings. This eliminates crashes when standard OpenAI proxies pass stringified arguments.
|
| 243 |
-
|
| 244 |
-
### 5. Two-Tier Agentic Error Escalation
|
| 245 |
-
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter. On the first error, a diagnostic warning is injected. On the second consecutive error, an urgent system warning forces a fundamentally different approach while retaining the reasoning block so the model can plan its correction.
|
| 246 |
-
|
| 247 |
-
### 6. Smart False-Positive Detection
|
| 248 |
-
Instead of broad substring matching that triggers false retry loops on successful database returns containing words like "error", this template utilizes strict structural guards evaluated over the first 120 characters of the payload.
|
| 249 |
-
|
| 250 |
-
Signals are split into two tiers. **Strong signals** are unambiguous structural markers (`"error":`, `Traceback (most recent call last):`, `command not found`, `Exception:`, `fatal:`, nonzero exit codes, `invalid syntax`) and fire unconditionally. **Weak signals** are bare prefixes (`error:`, `err!`) and remain gated by a length ceiling plus shell echo (`$ `) and timing (`took `) exclusions. Code and search results are excluded up front via `throw new`, `console.error`, `logger.error` and similar patterns.
|
| 251 |
-
|
| 252 |
-
Tiering matters because the earlier flat gate suppressed every signal on payloads of 600 characters or more: a routine multi-frame Python traceback exceeds that ceiling, and any output echoing a shell prompt or a timing figure was discarded outright. Strong signals now escalate regardless of payload size.
|
| 253 |
-
|
| 254 |
-
### 7. minijinja Compatibility Constraints
|
| 255 |
-
Python-only Jinja2 features crash or misbehave on `minijinja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 256 |
-
* `content | replace('<|think_on|>', '')` became `content.split('<|think_on|>') | join('')` (fixes a bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 257 |
-
* `| items` became `mapping.items()` iteration, which `minijinja` implements natively.
|
| 258 |
-
* `loop.previtem` became explicit array indexing.
|
| 259 |
-
* `map('string')` became `join('|')`.
|
| 260 |
-
* `| first` became `'$ ' in content`.
|
| 261 |
-
|
| 262 |
-
### 8. AST Flattening for C++ Throughput
|
| 263 |
-
Deeply nested Jinja loops and macros create severe parsing bottlenecks in C++ inference engines. We flattened the AST architecture, effectively curing an 80% inference throughput drop on `llama.cpp` by streamlining how `ns_state` tracking and historical rendering loops are evaluated.
|
| 264 |
-
|
| 265 |
-
### 9. Dynamic Payload Truncation
|
| 266 |
-
Massive API or database returns can instantly blow out a model's context window. We implemented `max_tool_arg_chars` and `max_tool_response_chars` limiters that safely slice oversized payloads. Argument truncation covers both mapping arguments and serialized JSON string arguments, so the OpenAI proxy wire format is limited on the same terms as native dictionaries. Both limiters are automatically disabled when `tool_call_format="json"` is active, as slicing serialized JSON structurally corrupts the data and crashes downstream parsers.
|
| 267 |
-
|
| 268 |
-
### 10. Reasoning Bypass Hallucination Mitigation
|
| 269 |
-
When thinking is disabled, Qwen models often hallucinate reasoning tags due to their training bias. We injected a safe boundary and adjusted the `<IMPORTANT>` system block to remove explicit mentions of `</think>` during tool instructions. This stops the model from hallucinating closing tags when calling tools in a no-reasoning state.
|
| 270 |
-
|
| 271 |
-
</details>
|
| 272 |
-
|
| 273 |
-
---
|
| 274 |
-
|
| 275 |
-
<details>
|
| 276 |
-
<summary>Update History & Changelog</summary>
|
| 277 |
-
|
| 278 |
-
> **2026-08-20 Update (v22.3): Error Detection Tiering, Reasoning De-duplication, and Test Suite Expansion.**
|
| 279 |
-
> 1. **Two-Tier Error Signals:** Structural error markers now escalate regardless of payload size. The previous flat 600-character ceiling plus whole-body `$ ` and `took ` exclusions silently suppressed warnings on ordinary multi-frame tracebacks and on any shell transcript that echoed its command. Successful exit-code reports (`Exit code: 0`) and JSON success envelopes (`"error": null` / `false` / `""`) are excluded from the strong tier so success payloads do not escalate.
|
| 280 |
-
> 2. **Reasoning De-duplication:** When `reasoning_content` or `thinking` is supplied, a leading think block in `content` is stripped so clients that populate both no longer emit two consecutive think blocks. Literal tags later in the answer (for example inside code fences) are preserved verbatim; full heuristic extraction still applies when no explicit reasoning field is present.
|
| 281 |
-
> 3. **Single-Line Think Blocks:** `<think>...</think>` written without surrounding newlines is now extracted instead of leaking raw tags into rendered history.
|
| 282 |
-
> 4. **Complete Argument Serialization:** Scalar and list tool arguments are serialized via `| tojson` rather than silently dropped, in both XML and JSON wire formats.
|
| 283 |
-
> 5. **Consistent Truncation:** `max_tool_arg_chars` now applies to serialized JSON string arguments, and `max_tool_response_chars` is bypassed under `tool_call_format="json"` as documented.
|
| 284 |
-
> 6. **KV Cache Documentation:** Documented that `preserve_thinking: false` rewrites rendered history at each new user turn and therefore voids the prefix cache guarantee, which applies to the default setting.
|
| 285 |
-
> 7. **Test Suite Expansion (44 -> 101):** Added coverage for control tag and alias completeness, malformed input and exception paths, vision payloads, agentic wire-format shapes, reasoning extraction variants, error detection precision, prefix KV cache stability, and jinja/oneline build parity.
|
| 286 |
-
> 8. **Property-Based Fuzz Harness:** Added `scripts/fuzz_template.py`, a deterministic conversation generator asserting nine structural invariants (render success, jinja/oneline parity, token balance, verbatim content preservation, XML parameter fidelity, JSON tool-call validity, warning precision, prefix KV stability, no-think prefill). Prefix stability is checked at generation boundaries: merged system blocks and consecutive tool-result batches are atomic, since the model only generates after a full batch is appended.
|
| 287 |
-
>
|
| 288 |
-
> *(Credits: Huge thanks to Juan Calderon-Perez (@g-a-b-y) for their massive contributions to the v22.3 release, including the property-based fuzz harness, the two-tier error signal architecture, and extensive test coverage!)*
|
| 289 |
-
>
|
| 290 |
-
> **2026-08-19 Update (v22.2): Extended Effort Aliases, String Argument Safety, and Error Disambiguation.**
|
| 291 |
-
> 1. **Universal Tool Argument Handling:** Hardened XML tool argument parsing to safely handle both mappings and serialized JSON strings in assistant history without syntax crashes or KV cache mutation.
|
| 292 |
-
> 2. **Reasoning Effort Aliases:** Added `"ultracode"` and `"extreme"` mappings to `'xhigh'`, with inline `<|think_ultracode|>` and `<|think_extreme|>` support (#78).
|
| 293 |
-
> 3. **Multi-System Message Merging:** Merged consecutive leading `system` and `developer` messages into a single unified system turn with `\n\n`.
|
| 294 |
-
> 4. **Grep / Search Error Disambiguation:** Eliminated false-positive tool error warnings on code search results containing patterns like `throw new Error` or `console.error` (#66).
|
| 295 |
-
> 5. **Safe XML Primitive Serialization:** Boolean and null XML parameter values now serialize to `true`, `false`, and `null` via `| tojson`.
|
| 296 |
-
> 6. **Zero-Crash Resilience:** Maintained full resilience without fatal `raise_exception` aborts across all runtime engines.
|
| 297 |
-
|
| 298 |
-
> **2026-08-16 Update (v22.1): The Qwen 3.8 Update (Bounded Reasoning Defaults, Inline Chat Tags, and Diagnostic Utility).**
|
| 299 |
-
> 1. **Full Qwen 3.8 Support:** Single drop-in template covering all Qwen 3.5, 3.6, and 3.8 model variants (`Qwen3.8-2.4T-A95B`, `Qwen3.8-27B`).
|
| 300 |
-
> 2. **Default `reasoning_effort` to `medium`:** Replaced the unsafe `xhigh` default with `medium` (zero injected tokens). Eliminates the runaway reasoning token-burn failure where models explore branches until hitting `max_tokens` with empty content (#72).
|
| 301 |
-
> 3. **Inline Chat Tags:** Added support for `<|think_low|>`, `<|think_medium|>`, and `<|think_xhigh|>` tags directly inside chat messages (#70).
|
| 302 |
-
> 4. **API Mapping & Aliasing:** Added case-insensitive alias support for client and serving runtimes (`high`/`max` -> `xhigh`, `minimal` -> `low`, `none` -> thinking off).
|
| 303 |
-
> 5. **Sequential Control Tag Stripping:** All 7 control tags are cleanly stripped across system, user, and multi-part content without prompt leakage.
|
| 304 |
-
> 6. **Cured Official 3.8 Empty Think Bug:** Fixed a regression in the official Qwen 3.8 template where removing `<think>` extraction caused blank `<think></think>` blocks to be prepended to real thoughts in chat history.
|
| 305 |
-
> 7. **Restored Fast Mode:** Replaced official 3.8's hard exception on `enable_thinking=false`, restoring full user freedom to disable reasoning via kwargs or `<|think_off|>`.
|
| 306 |
-
> 8. **Universal Tool Arguments:** Hardened tool call parsing to handle both dictionary structures and JSON-serialized strings without crashing.
|
| 307 |
-
> 9. **Diagnostic Utility:** Added `scripts/check_applied.py` to inspect model folders and GGUFs for template consistency.
|
| 308 |
-
|
| 309 |
-
> **2026-08-13 Update (v22): Qwen 3.8 Support, Reasoning Effort Controls, and Engine Hardening.**
|
| 310 |
-
|
| 311 |
-
> **2026-07-02 Update (v21.3): Optional JSON Tool Format Kwarg.** Added an optional `tool_call_format="json"` override for `chat_template_kwargs`.
|
| 312 |
-
|
| 313 |
-
> **2026-07-02 Update (v21.2): Reasoning Bypass Hallucination Fix.** Adjusted `<IMPORTANT>` block instructions to remove explicit mentions of `</think>` during tool definitions.
|
| 314 |
-
|
| 315 |
-
> **2026-07-02 Update (v21.1): Reliability Overhaul & XML Revert.** Reverted to native XML format for vLLM `qwen3_coder` compatibility and restored `preserve_thinking` default to `true`.
|
| 316 |
-
|
| 317 |
-
</details>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/README-v22.md
DELETED
|
@@ -1,278 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
tags:
|
| 4 |
-
- jinja
|
| 5 |
-
- chat-template
|
| 6 |
-
- qwen
|
| 7 |
-
- qwen3.5
|
| 8 |
-
- qwen3.6
|
| 9 |
-
- qwen3.8
|
| 10 |
-
- lm-studio
|
| 11 |
-
- mlx
|
| 12 |
-
- llama.cpp
|
| 13 |
-
- vllm
|
| 14 |
-
- tool-calling
|
| 15 |
-
- thinking
|
| 16 |
-
---
|
| 17 |
-
|
| 18 |
-
# Fixed jinja chat templates for Qwen 3.5, 3.6 & 3.8 (v22)
|
| 19 |
-
|
| 20 |
-
This is a universal drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, empty think poisoning, and fatal agentic stalling across official Qwen chat templates.
|
| 21 |
-
|
| 22 |
-
It works across LM Studio, llama.cpp, vLLM, MLX, oMLX, KoboldCPP, and any engine that supports Hugging Face Jinja templates. You only need the single `chat_template.jinja` file at the root of this repository for all Qwen 3.5, 3.6, and 3.8 model sizes.
|
| 23 |
-
|
| 24 |
-
---
|
| 25 |
-
|
| 26 |
-
## What is new in v22: The Qwen 3.8 Update
|
| 27 |
-
|
| 28 |
-
Qwen has released their 3.8 generation starting with `Qwen3.8-2.4T-A95B`. While the official 3.8 template adopted our default reasoning preservation, it also introduced severe regressions and rigid lockdowns that break local setups. Version 22 brings full 3.8 support while fixing these official bugs:
|
| 29 |
-
|
| 30 |
-
1. **Reasoning Effort Steering:** Qwen 3.8 supports prompt-directed thinking budgets (`xhigh`, `high`, `low`, `medium`). The template injects official steering instructions into the system prompt by default (`xhigh`) and auto-suppresses them when thinking is turned off.
|
| 31 |
-
2. **Restored Fast Mode (No Reasoning):** Official 3.8 throws a fatal runtime exception if you pass `enable_thinking=false`. Version 22 removes this lockdown, giving you complete freedom to disable reasoning via kwargs or inline `<|think_off|>` tags.
|
| 32 |
-
3. **Cured Official 3.8 Empty Think Bug:** Official 3.8 removed the in-content thinking parser. In multi-turn chats where reasoning is stored inside message content, official 3.8 injects a blank `<think></think>` block before the real thoughts. Version 22 extracts reasoning cleanly without duplicating tags.
|
| 33 |
-
4. **Universal Tool Arguments:** Official 3.8 crashes with `TypeError: Can only get item pairs from a mapping` when clients send standard OpenAI string arguments. Version 22 handles both Python dictionaries and JSON strings seamlessly.
|
| 34 |
-
5. **Native llama.cpp Flag Support:** Added native support for the new `llama.cpp` `--reasoning-preserve` CLI flag via the `preserve_reasoning` alias.
|
| 35 |
-
|
| 36 |
-
---
|
| 37 |
-
|
| 38 |
-
<details open>
|
| 39 |
-
<summary><b>Quick Install & Engine Setup</b></summary>
|
| 40 |
-
|
| 41 |
-
### llama.cpp / llama-server / koboldcpp
|
| 42 |
-
Run `llama-server` with the template file and DeepSeek reasoning format:
|
| 43 |
-
```bash
|
| 44 |
-
llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek
|
| 45 |
-
```
|
| 46 |
-
*Why `--reasoning-format deepseek` matters:* When connecting coding agents like OpenCode, Claude Code, or Pi.dev to `llama-server`, this flag extracts `<think>` blocks into the dedicated `reasoning_content` API response field. This prevents raw thinking tokens from leaking into the text stream and stopping tool calls midway.
|
| 47 |
-
|
| 48 |
-
*Native CLI flag:* On recent `llama.cpp` builds, you can pass `--reasoning-preserve` directly to ensure 100% Prefix KV Cache retention.
|
| 49 |
-
|
| 50 |
-
### LM Studio
|
| 51 |
-
1. Open your Qwen model in the right side panel.
|
| 52 |
-
2. Scroll down to **Prompt Template**.
|
| 53 |
-
3. Replace the template with the contents of `chat_template.jinja`.
|
| 54 |
-
4. Click **Save**.
|
| 55 |
-
|
| 56 |
-
### vLLM
|
| 57 |
-
Replace the `"chat_template"` string in your `tokenizer_config.json` with `chat_template_oneline.txt` (or raw `chat_template.jinja`).
|
| 58 |
-
```bash
|
| 59 |
-
vllm serve Qwen/Qwen3.8-2.4T-A95B --tool-call-parser qwen3_xml
|
| 60 |
-
```
|
| 61 |
-
*Parser selection:* Use `--tool-call-parser qwen3_xml` on current vLLM releases. If you are on an older vLLM build, use `--tool-call-parser qwen3_coder`. If you explicitly set `tool_call_format="json"`, use `--tool-call-parser hermes`.
|
| 62 |
-
|
| 63 |
-
### oMLX / MLX
|
| 64 |
-
Overwrite `chat_template.jinja` in your local model directory and launch with `--jinja`.
|
| 65 |
-
|
| 66 |
-
</details>
|
| 67 |
-
|
| 68 |
-
---
|
| 69 |
-
|
| 70 |
-
## Why you need this
|
| 71 |
-
|
| 72 |
-
The official Qwen templates contain engine restrictions, Python-specific Jinja logic, and regressions that break local inference and agent workflows.
|
| 73 |
-
|
| 74 |
-
<details open>
|
| 75 |
-
<summary><b>Critical Issues Fixed</b></summary>
|
| 76 |
-
|
| 77 |
-
| Area | Issue in Official Templates | The Fix in v22 |
|
| 78 |
-
|---|---|---|
|
| 79 |
-
| **Qwen 3.8 Support** | Official 3.8 crashes if `enable_thinking=false`. | **Restored Fast Mode**. Supports fast non-reasoning mode via kwargs or `<\|think_off\|>`. |
|
| 80 |
-
| **Qwen 3.8 Regression** | Official 3.8 injects duplicate blank `<think></think>` in chat history. | **Cured Empty Think Poisoning**. Robust multi-format reasoning extraction. |
|
| 81 |
-
| **Reasoning Control** | Qwen 3.8 prompt-steered reasoning budget controls. | **Full `reasoning_effort` Support**. Supports `xhigh` (default), `high`, `medium`, and `low`. |
|
| 82 |
-
| **Compatibility** | `llama.cpp --reasoning-preserve` CLI flag compatibility. | **Native Alias Support**. Supports both `preserve_reasoning` and `preserve_thinking`. |
|
| 83 |
-
| **Compatibility** | JSON-string tool arguments (OpenAI / Ollama) crash official templates. | **Universal Tool Parsing**. Safely handles mappings, JSON strings, and scalar args. |
|
| 84 |
-
| **Agentic Loop** | Model aborts turn when combining conversational text and a tool call. | Cured "Empty Think" poisoning and softened imperative system directives. |
|
| 85 |
-
| **Agentic Loop** | Model gets stuck emitting the identical failing tool call. | Added two-tier error escalation to force correction while retaining reasoning. |
|
| 86 |
-
| **Agentic Loop** | Model panics and debates internal rules after fetching data. | Broadened `<think>` instructions to authorize conversational synthesis. |
|
| 87 |
-
| **Agentic Loop** | API returns containing the word "error" trigger false retry loops. | Replaced broad matching with strict structural guards. |
|
| 88 |
-
| **Performance** | Mutated past turns destroy the prefix cache. | Enforced chronological history for a 100% KV Cache hit rate. |
|
| 89 |
-
| **Performance** | Deep Jinja nesting drops `llama.cpp` speed by 80%. | Flattened the AST architecture to maximize throughput. |
|
| 90 |
-
| **Compatibility** | Python-specific filters crash C++ inference engines. | Rewrote all filters to be 100% `minijinja` safe. |
|
| 91 |
-
| **Compatibility** | Qwen-native parsers (like vLLM) crash on JSON formatting. | Maintained canonical Qwen XML format as the default. |
|
| 92 |
-
| **Compatibility** | Older API setups and wrappers crash on native XML. | Added a `tool_call_format="json"` opt-in override. |
|
| 93 |
-
| **Compatibility** | Anthropic `message.thinking` payloads are rejected. | Added native Anthropic reasoning support. |
|
| 94 |
-
| **Stability** | Massive tool data returns blow out the context window. | Added dynamic payload truncation limits. |
|
| 95 |
-
| **Stability** | Mid-conversation system prompts crash the template. | Added native support for arbitrary system and developer messages. |
|
| 96 |
-
| **Edge Cases** | Text duplicates during streaming generation. | Restored canonical spacing to the generation prompt. |
|
| 97 |
-
| **Edge Cases** | Model hallucinates reasoning tags when thinking is disabled. | Injected strict boundaries to force clean reasoning bypass. |
|
| 98 |
-
|
| 99 |
-
</details>
|
| 100 |
-
|
| 101 |
-
---
|
| 102 |
-
|
| 103 |
-
## Customization & Kwarg Reference
|
| 104 |
-
|
| 105 |
-
<details open>
|
| 106 |
-
<summary><b>1. Reasoning Effort Steering (Qwen 3.8)</b></summary>
|
| 107 |
-
|
| 108 |
-
Qwen 3.8 introduces prompt-steered reasoning effort levels. You can control this via template kwargs:
|
| 109 |
-
```json
|
| 110 |
-
{
|
| 111 |
-
"reasoning_effort": "xhigh"
|
| 112 |
-
}
|
| 113 |
-
```
|
| 114 |
-
* **`"xhigh"` (Default / Recommended):** Injects Qwen's official deep reasoning instruction:
|
| 115 |
-
> *"Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer."*
|
| 116 |
-
* **`"high"`:** Mapped automatically as an alias to `"xhigh"` for OpenAI proxy compatibility.
|
| 117 |
-
* **`"low"`:** Injects concise thinking instructions for fast, summary-oriented reasoning.
|
| 118 |
-
* **`"medium"`:** Neutral baseline (no extra instruction text injected). Ideal for users wanting exact byte-for-byte prefix cache parity with v21 sessions.
|
| 119 |
-
|
| 120 |
-
*(Note: When thinking is disabled via `<|think_off|>` or `enable_thinking=false`, reasoning effort instructions are automatically suppressed).*
|
| 121 |
-
|
| 122 |
-
</details>
|
| 123 |
-
|
| 124 |
-
<details>
|
| 125 |
-
<summary><b>2. The Thinking Toggle & Fast Mode</b></summary>
|
| 126 |
-
|
| 127 |
-
You can control model reasoning dynamically on a per-prompt basis. Insert `<|think_on|>` or `<|think_off|>` anywhere in your system or user prompt. The template intercepts the tag, strips it from the final context, and sets reasoning mode immediately.
|
| 128 |
-
|
| 129 |
-
**Fast answer, no reasoning:**
|
| 130 |
-
```text
|
| 131 |
-
System: You are a coding assistant. <|think_off|>
|
| 132 |
-
User: What is 2+2?
|
| 133 |
-
```
|
| 134 |
-
|
| 135 |
-
**Deep reasoning:**
|
| 136 |
-
```text
|
| 137 |
-
System: You are a coding assistant. <|think_on|>
|
| 138 |
-
User: Implement a red-black tree in Rust.
|
| 139 |
-
```
|
| 140 |
-
|
| 141 |
-
*(The tag syntax uses Qwen control token delimiters so it will never collide with file paths or code, unlike old community `/think` hacks)*.
|
| 142 |
-
|
| 143 |
-
</details>
|
| 144 |
-
|
| 145 |
-
<details>
|
| 146 |
-
<summary><b>3. KV Cache Preservation (`preserve_reasoning` & `preserve_thinking`)</b></summary>
|
| 147 |
-
|
| 148 |
-
By default, this template **preserves** all past `<think>` blocks in the chat history. This prevents the model from suffering "amnesia stalls" during complex agentic loops and guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
| 149 |
-
|
| 150 |
-
* On recent `llama.cpp` builds, pass `--reasoning-preserve` directly.
|
| 151 |
-
* Or pass via template kwargs:
|
| 152 |
-
```json
|
| 153 |
-
{
|
| 154 |
-
"preserve_thinking": true
|
| 155 |
-
}
|
| 156 |
-
```
|
| 157 |
-
|
| 158 |
-
If you are on severely memory-constrained hardware and need to save context tokens, set `"preserve_thinking": false` (or `"preserve_reasoning": false`) to strip past thoughts.
|
| 159 |
-
|
| 160 |
-
</details>
|
| 161 |
-
|
| 162 |
-
<details>
|
| 163 |
-
<summary><b>4. Tool Call Format Override (JSON vs XML)</b></summary>
|
| 164 |
-
|
| 165 |
-
Qwen models are natively trained to output tool calls in XML (`<function=name>`). By default, this template uses native XML to maximize reliability.
|
| 166 |
-
|
| 167 |
-
**When to use the JSON override:**
|
| 168 |
-
If you are using a framework or harness (such as specific Hermes Agent configurations) that strictly requires Hermes JSON (`{"name": "...", "arguments": {...}}`), pass:
|
| 169 |
-
```json
|
| 170 |
-
{
|
| 171 |
-
"tool_call_format": "json"
|
| 172 |
-
}
|
| 173 |
-
```
|
| 174 |
-
*(When opting into JSON format, argument truncation is safely bypassed to avoid corrupting JSON syntax)*.
|
| 175 |
-
|
| 176 |
-
</details>
|
| 177 |
-
|
| 178 |
-
<details>
|
| 179 |
-
<summary><b>5. Dynamic Payload Truncation</b></summary>
|
| 180 |
-
|
| 181 |
-
To prevent oversized tool returns from blowing out context limits:
|
| 182 |
-
* `max_tool_arg_chars` (default `0` / disabled): Slices oversized tool call arguments.
|
| 183 |
-
* `max_tool_response_chars` (default `0` / disabled): Slices oversized tool output data.
|
| 184 |
-
|
| 185 |
-
</details>
|
| 186 |
-
|
| 187 |
-
---
|
| 188 |
-
|
| 189 |
-
<details>
|
| 190 |
-
<summary><b>Running the test suite</b></summary>
|
| 191 |
-
|
| 192 |
-
```bash
|
| 193 |
-
python3 scripts/test_v22.py
|
| 194 |
-
```
|
| 195 |
-
Tests cover 28 automated verification cells including `reasoning_effort` levels, tag pre-scanning, tool call serialization, dynamic truncation, error escalation, and multi-turn history parsing.
|
| 196 |
-
|
| 197 |
-
</details>
|
| 198 |
-
|
| 199 |
-
---
|
| 200 |
-
|
| 201 |
-
## Authorship
|
| 202 |
-
| Role | Author |
|
| 203 |
-
|------|--------|
|
| 204 |
-
| Original models | Alibaba Cloud (Qwen team) |
|
| 205 |
-
| Template fixes | [froggeric](https://huggingface.co/froggeric) |
|
| 206 |
-
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 207 |
-
|
| 208 |
-
## License
|
| 209 |
-
Apache-2.0, inherited from Qwen.
|
| 210 |
-
|
| 211 |
-
---
|
| 212 |
-
|
| 213 |
-
<details>
|
| 214 |
-
<summary>Technical Details of the Critical Fixes</summary>
|
| 215 |
-
|
| 216 |
-
### 1. The "Empty Think" Poisoning and Logic Trap Cure
|
| 217 |
-
Previous templates attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic pattern where the model associated empty thoughts with tools, causing an 80%+ premature turn abort rate. We abolished empty think injection and rewrote the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after thinking. In v22, we also cured official Qwen 3.8's history bug where missing in-content parsers created duplicate blank think tags.
|
| 218 |
-
|
| 219 |
-
### 2. Upfront Pre-Scan for Control Tags
|
| 220 |
-
In Jinja templates, system prompts are assembled before iterating over message history. Version 22 introduces an upfront pre-scan covering both plain text and multi-part content lists (`[{'type': 'text', 'text': '...'}]`). This resolves `<|think_off|>` and `<|think_on|>` states before the system message is built, ensuring reasoning instructions are never injected into non-reasoning turns.
|
| 221 |
-
|
| 222 |
-
### 3. KV Cache Safety and Autoregressive Normalization
|
| 223 |
-
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because this template preserves historical thoughts chronologically by default, rendered history perfectly synchronizes with cached generated tokens. Combined with strict single newline normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn sessions.
|
| 224 |
-
|
| 225 |
-
### 4. Native XML Tool Format and Universal Serialization
|
| 226 |
-
The model was trained with the XML tool format used by Qwen3-Coder. We restored this format natively while bypassing the `|items` crash by handling both mapping dictionaries and JSON strings. This eliminates crashes when standard OpenAI proxies pass stringified arguments.
|
| 227 |
-
|
| 228 |
-
### 5. Two-Tier Agentic Error Escalation
|
| 229 |
-
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter. On the first error, a diagnostic warning is injected. On the second consecutive error, an urgent system warning forces a fundamentally different approach while retaining the reasoning block so the model can plan its correction.
|
| 230 |
-
|
| 231 |
-
### 6. Smart False-Positive Detection
|
| 232 |
-
Instead of broad substring matching that triggers false retry loops on successful database returns containing words like "error", this template utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell echo exclusions (`$ `).
|
| 233 |
-
|
| 234 |
-
### 7. minijinja Compatibility Constraints
|
| 235 |
-
Python-only Jinja2 features crash or misbehave on `minijinja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 236 |
-
* `content | replace('<|think_on|>', '')` became `content.split('<|think_on|>') | join('')` (fixes a bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 237 |
-
* `| items` became `for key in mapping`.
|
| 238 |
-
* `loop.previtem` became explicit array indexing.
|
| 239 |
-
* `map('string')` became `join('|')`.
|
| 240 |
-
* `| first` became `'$ ' in content`.
|
| 241 |
-
|
| 242 |
-
### 8. AST Flattening for C++ Throughput
|
| 243 |
-
Deeply nested Jinja loops and macros create severe parsing bottlenecks in C++ inference engines. We flattened the AST architecture, effectively curing an 80% inference throughput drop on `llama.cpp` by streamlining how `ns_state` tracking and historical rendering loops are evaluated.
|
| 244 |
-
|
| 245 |
-
### 9. Dynamic Payload Truncation
|
| 246 |
-
Massive API or database returns can instantly blow out a model's context window. We implemented `max_tool_arg_chars` and `max_tool_response_chars` limiters that safely slice oversized payloads. Crucially, this truncation is automatically disabled when `tool_call_format="json"` is active, as slicing a serialized JSON string structurally corrupts the data and crashes downstream parsers.
|
| 247 |
-
|
| 248 |
-
### 10. Reasoning Bypass Hallucination Mitigation
|
| 249 |
-
When thinking is disabled, Qwen models often hallucinate reasoning tags due to their training bias. We injected a safe boundary and adjusted the `<IMPORTANT>` system block to remove explicit mentions of `</think>` during tool instructions. This stops the model from hallucinating closing tags when calling tools in a no-reasoning state.
|
| 250 |
-
|
| 251 |
-
</details>
|
| 252 |
-
|
| 253 |
-
---
|
| 254 |
-
|
| 255 |
-
<details>
|
| 256 |
-
<summary>Update History & Changelog</summary>
|
| 257 |
-
|
| 258 |
-
> **2026-08-13 Update (v22): Qwen 3.8 Support, Reasoning Effort Controls, and Engine Hardening.**
|
| 259 |
-
> 1. **Qwen 3.8 Compatibility:** Added full support for the new Qwen 3.8 model family (`Qwen3.8-2.4T-A95B`).
|
| 260 |
-
> 2. **Reasoning Effort Steering:** Introduced `reasoning_effort` parameter (`xhigh` [default], `high`, `medium`, `low`). Automatically injects official Qwen 3.8 steering directives and auto-suppresses them when thinking is disabled.
|
| 261 |
-
> 3. **Cured Official 3.8 Empty Think Bug:** Fixed a regression in the official Qwen 3.8 template where removing `<think>` extraction caused blank `<think></think>` blocks to be prepended to real thoughts in chat history.
|
| 262 |
-
> 4. **Restored Fast Mode:** Replaced official 3.8's hard exception on `enable_thinking=false`, restoring full user freedom to disable reasoning via kwargs or `<|think_off|>`.
|
| 263 |
-
> 5. **llama.cpp `--reasoning-preserve` Alias:** Added native support for `preserve_reasoning` alongside `preserve_thinking`.
|
| 264 |
-
> 6. **Universal Tool Argument Handling:** Hardened tool call parsing to handle both dictionary structures and JSON-serialized strings without crashing.
|
| 265 |
-
> 7. **Pre-Scan Tag Gating:** Upgraded control tag pre-scanning to inspect both raw string content and multi-part content lists before assembling system instructions.
|
| 266 |
-
> 8. **Retained Reasoning in Error Recovery:** Maintained the `<think>\n` generation prompt during tool failure escalation so Qwen 3.8 can plan its corrected tool call.
|
| 267 |
-
|
| 268 |
-
> **2026-07-02 Update (v21.3): Optional JSON Tool Format Kwarg.** Added an optional `tool_call_format="json"` override for `chat_template_kwargs`.
|
| 269 |
-
|
| 270 |
-
> **2026-07-02 Update (v21.2): Reasoning Bypass Hallucination Fix.** Adjusted `<IMPORTANT>` block instructions to remove explicit mentions of `</think>` during tool definitions.
|
| 271 |
-
|
| 272 |
-
> **2026-07-02 Update (v21.1): Reliability Overhaul & XML Revert.** Reverted to native XML format for vLLM `qwen3_coder` compatibility and restored `preserve_thinking` default to `true`.
|
| 273 |
-
|
| 274 |
-
> **2026-06-05 Update (v20): The Architect Patch.** Major structural update for agentic loops and C++ inference engines.
|
| 275 |
-
|
| 276 |
-
> **2026-05-18 Update (v19): The Agentic Loop Cure.** Abolished "Empty Think" poisoning and restored Universal Synthesis instructions.
|
| 277 |
-
|
| 278 |
-
</details>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/v20_chat_template.jinja
DELETED
|
@@ -1,287 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.6-froggeric-v20" %}
|
| 2 |
-
{%- set image_count = namespace(value=0) %}
|
| 3 |
-
{%- set video_count = namespace(value=0) %}
|
| 4 |
-
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 5 |
-
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 6 |
-
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 7 |
-
{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else false %}
|
| 8 |
-
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 9 |
-
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 10 |
-
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 11 |
-
{%- set ns_state = namespace(thinking=enable_thinking) %}
|
| 12 |
-
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 13 |
-
{%- set ns_state.thinking = false %}
|
| 14 |
-
{%- endif %}
|
| 15 |
-
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 16 |
-
{%- if content is string %}
|
| 17 |
-
{{- content }}
|
| 18 |
-
{%- elif content is iterable and content is not mapping %}
|
| 19 |
-
{%- for item in content %}
|
| 20 |
-
{%- if item is mapping %}
|
| 21 |
-
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 22 |
-
{%- if is_system_content %}
|
| 23 |
-
{{- raise_exception('System message cannot contain images.') }}
|
| 24 |
-
{%- endif %}
|
| 25 |
-
{%- if do_vision_count %}
|
| 26 |
-
{%- set image_count.value = image_count.value + 1 %}
|
| 27 |
-
{%- endif %}
|
| 28 |
-
{%- if add_vision_id %}
|
| 29 |
-
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 30 |
-
{%- endif %}
|
| 31 |
-
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 32 |
-
{%- elif item.type == 'video' or 'video' in item %}
|
| 33 |
-
{%- if is_system_content %}
|
| 34 |
-
{{- raise_exception('System message cannot contain videos.') }}
|
| 35 |
-
{%- endif %}
|
| 36 |
-
{%- if do_vision_count %}
|
| 37 |
-
{%- set video_count.value = video_count.value + 1 %}
|
| 38 |
-
{%- endif %}
|
| 39 |
-
{%- if add_vision_id %}
|
| 40 |
-
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 41 |
-
{%- endif %}
|
| 42 |
-
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 43 |
-
{%- elif 'text' in item %}
|
| 44 |
-
{{- item.text }}
|
| 45 |
-
{%- else %}
|
| 46 |
-
{{- raise_exception('Unexpected item type in content.') }}
|
| 47 |
-
{%- endif %}
|
| 48 |
-
{%- else %}
|
| 49 |
-
{{- item | string }}
|
| 50 |
-
{%- endif %}
|
| 51 |
-
{%- endfor %}
|
| 52 |
-
{%- elif content is none or content is undefined %}
|
| 53 |
-
{{- '' }}
|
| 54 |
-
{%- else %}
|
| 55 |
-
{{- raise_exception('Unexpected content type.') }}
|
| 56 |
-
{%- endif %}
|
| 57 |
-
{%- endmacro %}
|
| 58 |
-
{%- if not messages %}
|
| 59 |
-
{{- raise_exception('No messages provided.') }}
|
| 60 |
-
{%- endif %}
|
| 61 |
-
{%- set _first_role = messages[0].role %}
|
| 62 |
-
{%- if _first_role == 'system' or _first_role == 'developer' %}
|
| 63 |
-
{%- set _sys_msg = messages[0] %}
|
| 64 |
-
{%- set _msgs = messages[1:] %}
|
| 65 |
-
{%- else %}
|
| 66 |
-
{%- set _sys_msg = none %}
|
| 67 |
-
{%- set _msgs = messages %}
|
| 68 |
-
{%- endif %}
|
| 69 |
-
{%- set _sc = '' %}
|
| 70 |
-
{%- if _sys_msg is not none %}
|
| 71 |
-
{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}
|
| 72 |
-
{%- if '<|think_off|>' in _sc %}
|
| 73 |
-
{%- set ns_state.thinking = false %}
|
| 74 |
-
{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}
|
| 75 |
-
{%- elif '<|think_on|>' in _sc %}
|
| 76 |
-
{%- set ns_state.thinking = true %}
|
| 77 |
-
{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}
|
| 78 |
-
{%- endif %}
|
| 79 |
-
{%- endif %}
|
| 80 |
-
{%- if _has_tools %}
|
| 81 |
-
{{- '<|im_start|>system\n' }}
|
| 82 |
-
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 83 |
-
{%- for tool in tools %}
|
| 84 |
-
{{- '\n' }}
|
| 85 |
-
{{- tool | tojson }}
|
| 86 |
-
{%- endfor %}
|
| 87 |
-
{{- '\n</tools>' }}
|
| 88 |
-
{%- set tool_instructions %}
|
| 89 |
-
If you choose to call a function ONLY reply in the following format with NO suffix:
|
| 90 |
-
|
| 91 |
-
<think>
|
| 92 |
-
Brief explanation of tool call
|
| 93 |
-
</think>
|
| 94 |
-
<tool_call>
|
| 95 |
-
<function=example_function_name>
|
| 96 |
-
<parameter=example_parameter_1>
|
| 97 |
-
value_1
|
| 98 |
-
</parameter>
|
| 99 |
-
<parameter=example_parameter_2>
|
| 100 |
-
This is the value for the second parameter
|
| 101 |
-
that can span
|
| 102 |
-
multiple lines
|
| 103 |
-
</parameter>
|
| 104 |
-
</function>
|
| 105 |
-
</tool_call>
|
| 106 |
-
|
| 107 |
-
<IMPORTANT>
|
| 108 |
-
Reminder:
|
| 109 |
-
- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.
|
| 110 |
-
- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.
|
| 111 |
-
- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.
|
| 112 |
-
- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after closing </think>. Do NOT output any conversational text before the tool call.
|
| 113 |
-
- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.
|
| 114 |
-
- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.
|
| 115 |
-
- If you have gathered all necessary data and do not need to call a tool, answer the question like normal and provide your final response to the user IMMEDIATELY after closing </think>.
|
| 116 |
-
</IMPORTANT>
|
| 117 |
-
{%- endset %}
|
| 118 |
-
{{- '\n\n' ~ tool_instructions | trim }}
|
| 119 |
-
{%- if _sc %}
|
| 120 |
-
{{- '\n\n' + _sc }}
|
| 121 |
-
{%- endif %}
|
| 122 |
-
{{- '<|im_end|>\n' }}
|
| 123 |
-
{%- else %}
|
| 124 |
-
{%- if _sc %}
|
| 125 |
-
{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}
|
| 126 |
-
{%- endif %}
|
| 127 |
-
{%- endif %}
|
| 128 |
-
{%- set _last_idx = _msgs | length - 1 %}
|
| 129 |
-
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 130 |
-
{%- for message in _msgs[::-1] %}
|
| 131 |
-
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 132 |
-
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 133 |
-
{%- set _rc = render_content(message.content, false) | trim %}
|
| 134 |
-
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 135 |
-
{%- set ns.multi_step_tool = false %}
|
| 136 |
-
{%- set ns.last_query_index = index %}
|
| 137 |
-
{%- endif %}
|
| 138 |
-
{%- endif %}
|
| 139 |
-
{%- endfor %}
|
| 140 |
-
{%- if ns.multi_step_tool %}
|
| 141 |
-
{%- if _last_idx > 50 %}
|
| 142 |
-
{%- set ns.last_query_index = _last_idx %}
|
| 143 |
-
{%- else %}
|
| 144 |
-
{%- set ns.last_query_index = 0 %}
|
| 145 |
-
{%- endif %}
|
| 146 |
-
{%- endif %}
|
| 147 |
-
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 148 |
-
{%- for message in _msgs %}
|
| 149 |
-
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 150 |
-
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 151 |
-
{%- if '<|think_off|>' in content %}
|
| 152 |
-
{%- set ns_state.thinking = false %}
|
| 153 |
-
{%- set content = content.split('<|think_off|>') | join('') | trim %}
|
| 154 |
-
{%- elif '<|think_on|>' in content %}
|
| 155 |
-
{%- set ns_state.thinking = true %}
|
| 156 |
-
{%- set content = content.split('<|think_on|>') | join('') | trim %}
|
| 157 |
-
{%- endif %}
|
| 158 |
-
{%- if is_system %}
|
| 159 |
-
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 160 |
-
{%- elif message.role == 'user' %}
|
| 161 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 162 |
-
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 163 |
-
{%- elif message.role == 'assistant' %}
|
| 164 |
-
{%- set reasoning_content = '' %}
|
| 165 |
-
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 166 |
-
{%- if message.reasoning_content is string %}
|
| 167 |
-
{%- set reasoning_content = message.reasoning_content %}
|
| 168 |
-
{%- else %}
|
| 169 |
-
{%- set reasoning_content = message.reasoning_content | string %}
|
| 170 |
-
{%- endif %}
|
| 171 |
-
{%- else %}
|
| 172 |
-
{%- set _think_end = '' %}
|
| 173 |
-
{%- if '</think>' in content %}
|
| 174 |
-
{%- set _think_end = '</think>' %}
|
| 175 |
-
{%- elif '</thinking>' in content %}
|
| 176 |
-
{%- set _think_end = '</thinking>' %}
|
| 177 |
-
{%- elif '</ think>' in content %}
|
| 178 |
-
{%- set _think_end = '</ think>' %}
|
| 179 |
-
{%- elif '</think >' in content %}
|
| 180 |
-
{%- set _think_end = '</think >' %}
|
| 181 |
-
{%- endif %}
|
| 182 |
-
{%- if _think_end %}
|
| 183 |
-
{%- if _think_end == '</thinking>' %}
|
| 184 |
-
{%- set _think_start = '<thinking>' %}
|
| 185 |
-
{%- else %}
|
| 186 |
-
{%- set _think_start = '<think>' %}
|
| 187 |
-
{%- endif %}
|
| 188 |
-
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 189 |
-
{%- if _think_start in reasoning_content %}
|
| 190 |
-
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 191 |
-
{%- endif %}
|
| 192 |
-
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 193 |
-
{%- endif %}
|
| 194 |
-
{%- endif %}
|
| 195 |
-
{%- set reasoning_content = reasoning_content | trim %}
|
| 196 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 197 |
-
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 198 |
-
{%- else %}
|
| 199 |
-
{{- '<|im_start|>assistant\n' + content }}
|
| 200 |
-
{%- endif %}
|
| 201 |
-
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 202 |
-
{%- for tool_call in message.tool_calls %}
|
| 203 |
-
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 204 |
-
{%- set tc = tool_call.function %}
|
| 205 |
-
{%- else %}
|
| 206 |
-
{%- set tc = tool_call %}
|
| 207 |
-
{%- endif %}
|
| 208 |
-
{%- if loop.first %}
|
| 209 |
-
{%- if content | trim %}
|
| 210 |
-
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 211 |
-
{%- else %}
|
| 212 |
-
{{- '<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 213 |
-
{%- endif %}
|
| 214 |
-
{%- else %}
|
| 215 |
-
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 216 |
-
{%- endif %}
|
| 217 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 218 |
-
{%- if tc.arguments is mapping %}
|
| 219 |
-
{%- for args_name, args_value in tc.arguments.items() %}
|
| 220 |
-
{{- '<parameter=' + args_name + '>\n' }}
|
| 221 |
-
{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
|
| 222 |
-
{%- set _av = args_value | tojson %}
|
| 223 |
-
{%- else %}
|
| 224 |
-
{%- set _av = args_value | string %}
|
| 225 |
-
{%- endif %}
|
| 226 |
-
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 227 |
-
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 228 |
-
{%- else %}
|
| 229 |
-
{{- _av }}
|
| 230 |
-
{%- endif %}
|
| 231 |
-
{{- '\n</parameter>\n' }}
|
| 232 |
-
{%- endfor %}
|
| 233 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 234 |
-
{{- tc.arguments }}
|
| 235 |
-
{%- endif %}
|
| 236 |
-
{%- endif %}
|
| 237 |
-
{%- if loop.last %}
|
| 238 |
-
{{- '</function>\n</tool_call>\n' }}
|
| 239 |
-
{%- else %}
|
| 240 |
-
{{- '</function>\n</tool_call>' }}
|
| 241 |
-
{%- endif %}
|
| 242 |
-
{%- endfor %}
|
| 243 |
-
{%- endif %}
|
| 244 |
-
{{- '<|im_end|>\n' }}
|
| 245 |
-
{%- elif message.role == 'tool' %}
|
| 246 |
-
{%- set _content_lower = content | lower %}
|
| 247 |
-
{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_lower or 'error:' in _content_lower or 'exception:' in _content_lower or 'traceback' in _content_lower or 'command not found' in _content_lower or 'invalid syntax' in _content_lower or 'failed to' in _content_lower) %}
|
| 248 |
-
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 249 |
-
{%- else %}
|
| 250 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 251 |
-
{%- endif %}
|
| 252 |
-
{%- if ns2.prev_role != 'tool' %}
|
| 253 |
-
{{- '<|im_start|>user' }}
|
| 254 |
-
{%- endif %}
|
| 255 |
-
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 256 |
-
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 257 |
-
{%- endif %}
|
| 258 |
-
{{- '\n<tool_response>\n' + content }}
|
| 259 |
-
{%- if ns2.consecutive_failures >= 2 %}
|
| 260 |
-
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 261 |
-
{%- elif ns2.consecutive_failures == 1 %}
|
| 262 |
-
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 263 |
-
{%- endif %}
|
| 264 |
-
{{- '\n</tool_response>' }}
|
| 265 |
-
{%- if loop.last %}
|
| 266 |
-
{{- '<|im_end|>\n' }}
|
| 267 |
-
{%- else %}
|
| 268 |
-
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 269 |
-
{%- if _next_role != 'tool' %}
|
| 270 |
-
{{- '<|im_end|>\n' }}
|
| 271 |
-
{%- endif %}
|
| 272 |
-
{%- endif %}
|
| 273 |
-
{%- else %}
|
| 274 |
-
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 275 |
-
{%- endif %}
|
| 276 |
-
{%- set ns2.prev_role = message.role %}
|
| 277 |
-
{%- endfor %}
|
| 278 |
-
{%- if add_generation_prompt %}
|
| 279 |
-
{{- '<|im_start|>assistant\n' }}
|
| 280 |
-
{%- if not ns_state.thinking %}
|
| 281 |
-
{{- '<think>\n</think>\n' }}
|
| 282 |
-
{%- elif ns2.consecutive_failures >= 2 %}
|
| 283 |
-
{{- '<think>\n</think>\n' }}
|
| 284 |
-
{%- else %}
|
| 285 |
-
{{- '<think>\n' }}
|
| 286 |
-
{%- endif %}
|
| 287 |
-
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/v20_chat_template_oneline.txt
DELETED
|
@@ -1 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.6-froggeric-v20" %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else false %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set ns_state = namespace(thinking=enable_thinking) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set _first_role = messages[0].role %}{%- if _first_role == 'system' or _first_role == 'developer' %}{%- set _sys_msg = messages[0] %}{%- set _msgs = messages[1:] %}{%- else %}{%- set _sys_msg = none %}{%- set _msgs = messages %}{%- endif %}{%- set _sc = '' %}{%- if _sys_msg is not none %}{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}{%- if '<|think_off|>' in _sc %}{%- set ns_state.thinking = false %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in _sc %}{%- set ns_state.thinking = true %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- set tool_instructions %} If you choose to call a function ONLY reply in the following format with NO suffix: <think> Brief explanation of tool call </think> <tool_call> <function=example_function_name> <parameter=example_parameter_1> value_1 </parameter> <parameter=example_parameter_2> This is the value for the second parameter that can span multiple lines </parameter> </function> </tool_call> <IMPORTANT> Reminder: - You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user. - ALL explanation and reasoning MUST be placed strictly inside the <think></think> block. - Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags. - If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after closing </think>. Do NOT output any conversational text before the tool call. - The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them. - To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks. - If you have gathered all necessary data and do not need to call a tool, answer the question like normal and provide your final response to the user IMMEDIATELY after closing </think>. </IMPORTANT> {%- endset %}{{- '\n\n' ~ tool_instructions | trim }}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if '<|think_off|>' in content %}{%- set ns_state.thinking = false %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in content %}{%- set ns_state.thinking = true %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if '</think>' in content %}{%- set _think_end = '</think>' %}{%- elif '</thinking>' in content %}{%- set _think_end = '</thinking>' %}{%- elif '</ think>' in content %}{%- set _think_end = '</ think>' %}{%- elif '</think >' in content %}{%- set _think_end = '</think >' %}{%- endif %}{%- if _think_end %}{%- if _think_end == '</thinking>' %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc.name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}{%- set _av = args_value | tojson %}{%- else %}{%- set _av = args_value | string %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{%- if loop.last %}{{- '</function>\n</tool_call>\n' }}{%- else %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_lower or 'error:' in _content_lower or 'exception:' in _content_lower or 'traceback' in _content_lower or 'command not found' in _content_lower or 'invalid syntax' in _content_lower or 'failed to' in _content_lower) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n</think>\n' }}{%- elif ns2.consecutive_failures >= 2 %}{{- '<think>\n</think>\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
|
|
|
|
|
archive/v21_chat_template.jinja
DELETED
|
@@ -1,329 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.6-froggeric-v21.3" %}
|
| 2 |
-
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
-
{%- set image_count = namespace(value=0) %}
|
| 4 |
-
{%- set video_count = namespace(value=0) %}
|
| 5 |
-
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 6 |
-
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 7 |
-
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 8 |
-
{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else true %}
|
| 9 |
-
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 10 |
-
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 11 |
-
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 12 |
-
{%- set ns_state = namespace(thinking=enable_thinking) %}
|
| 13 |
-
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 14 |
-
{%- set ns_state.thinking = false %}
|
| 15 |
-
{%- endif %}
|
| 16 |
-
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 17 |
-
{%- if content is string %}
|
| 18 |
-
{{- content }}
|
| 19 |
-
{%- elif content is iterable and content is not mapping %}
|
| 20 |
-
{%- for item in content %}
|
| 21 |
-
{%- if item is mapping %}
|
| 22 |
-
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 23 |
-
{%- if is_system_content %}
|
| 24 |
-
{{- raise_exception('System message cannot contain images.') }}
|
| 25 |
-
{%- endif %}
|
| 26 |
-
{%- if do_vision_count %}
|
| 27 |
-
{%- set image_count.value = image_count.value + 1 %}
|
| 28 |
-
{%- endif %}
|
| 29 |
-
{%- if add_vision_id %}
|
| 30 |
-
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 31 |
-
{%- endif %}
|
| 32 |
-
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 33 |
-
{%- elif item.type == 'video' or 'video' in item %}
|
| 34 |
-
{%- if is_system_content %}
|
| 35 |
-
{{- raise_exception('System message cannot contain videos.') }}
|
| 36 |
-
{%- endif %}
|
| 37 |
-
{%- if do_vision_count %}
|
| 38 |
-
{%- set video_count.value = video_count.value + 1 %}
|
| 39 |
-
{%- endif %}
|
| 40 |
-
{%- if add_vision_id %}
|
| 41 |
-
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 42 |
-
{%- endif %}
|
| 43 |
-
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 44 |
-
{%- elif 'text' in item %}
|
| 45 |
-
{{- item.text }}
|
| 46 |
-
{%- else %}
|
| 47 |
-
{{- raise_exception('Unexpected item type in content.') }}
|
| 48 |
-
{%- endif %}
|
| 49 |
-
{%- else %}
|
| 50 |
-
{{- item | string }}
|
| 51 |
-
{%- endif %}
|
| 52 |
-
{%- endfor %}
|
| 53 |
-
{%- elif content is none or content is undefined %}
|
| 54 |
-
{{- '' }}
|
| 55 |
-
{%- else %}
|
| 56 |
-
{{- raise_exception('Unexpected content type.') }}
|
| 57 |
-
{%- endif %}
|
| 58 |
-
{%- endmacro %}
|
| 59 |
-
{%- if not messages %}
|
| 60 |
-
{{- raise_exception('No messages provided.') }}
|
| 61 |
-
{%- endif %}
|
| 62 |
-
{%- set _first_role = messages[0].role %}
|
| 63 |
-
{%- if _first_role == 'system' or _first_role == 'developer' %}
|
| 64 |
-
{%- set _sys_msg = messages[0] %}
|
| 65 |
-
{%- set _msgs = messages[1:] %}
|
| 66 |
-
{%- else %}
|
| 67 |
-
{%- set _sys_msg = none %}
|
| 68 |
-
{%- set _msgs = messages %}
|
| 69 |
-
{%- endif %}
|
| 70 |
-
{%- set _sc = '' %}
|
| 71 |
-
{%- if _sys_msg is not none %}
|
| 72 |
-
{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}
|
| 73 |
-
{%- if '<|think_off|>' in _sc %}
|
| 74 |
-
{%- set ns_state.thinking = false %}
|
| 75 |
-
{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}
|
| 76 |
-
{%- elif '<|think_on|>' in _sc %}
|
| 77 |
-
{%- set ns_state.thinking = true %}
|
| 78 |
-
{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}
|
| 79 |
-
{%- endif %}
|
| 80 |
-
{%- endif %}
|
| 81 |
-
{%- if _has_tools %}
|
| 82 |
-
{{- '<|im_start|>system\n' }}
|
| 83 |
-
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 84 |
-
{%- for tool in tools %}
|
| 85 |
-
{{- '\n' }}
|
| 86 |
-
{{- tool | tojson }}
|
| 87 |
-
{%- endfor %}
|
| 88 |
-
{{- '\n</tools>' }}
|
| 89 |
-
{%- set tool_instructions %}
|
| 90 |
-
If you choose to call a function ONLY reply in the following format with NO suffix:
|
| 91 |
-
|
| 92 |
-
{%- if _tool_format == 'json' %}
|
| 93 |
-
<think>
|
| 94 |
-
Brief explanation of tool call
|
| 95 |
-
</think>
|
| 96 |
-
<tool_call>
|
| 97 |
-
{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}
|
| 98 |
-
</tool_call>
|
| 99 |
-
{%- else %}
|
| 100 |
-
<think>
|
| 101 |
-
Brief explanation of tool call
|
| 102 |
-
</think>
|
| 103 |
-
<tool_call>
|
| 104 |
-
<function=example_function_name>
|
| 105 |
-
<parameter=example_parameter_1>
|
| 106 |
-
value_1
|
| 107 |
-
</parameter>
|
| 108 |
-
<parameter=example_parameter_2>
|
| 109 |
-
This is the value for the second parameter
|
| 110 |
-
that can span
|
| 111 |
-
multiple lines
|
| 112 |
-
</parameter>
|
| 113 |
-
</function>
|
| 114 |
-
</tool_call>
|
| 115 |
-
{%- endif %}
|
| 116 |
-
|
| 117 |
-
<IMPORTANT>
|
| 118 |
-
Reminder:
|
| 119 |
-
- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.
|
| 120 |
-
- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.
|
| 121 |
-
{%- if _tool_format == 'json' %}
|
| 122 |
-
- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.
|
| 123 |
-
{%- else %}
|
| 124 |
-
- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.
|
| 125 |
-
{%- endif %}
|
| 126 |
-
- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.
|
| 127 |
-
{%- if _tool_format == 'json' %}
|
| 128 |
-
- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.
|
| 129 |
-
{%- else %}
|
| 130 |
-
- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.
|
| 131 |
-
{%- endif %}
|
| 132 |
-
- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.
|
| 133 |
-
- If you have all necessary data, provide your final answer directly to the user without any tool call.
|
| 134 |
-
</IMPORTANT>
|
| 135 |
-
{%- endset %}
|
| 136 |
-
{{- '\n\n' ~ tool_instructions | trim }}
|
| 137 |
-
{%- if _sc %}
|
| 138 |
-
{{- '\n\n' + _sc }}
|
| 139 |
-
{%- endif %}
|
| 140 |
-
{{- '<|im_end|>\n' }}
|
| 141 |
-
{%- else %}
|
| 142 |
-
{%- if _sc %}
|
| 143 |
-
{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}
|
| 144 |
-
{%- endif %}
|
| 145 |
-
{%- endif %}
|
| 146 |
-
{%- set _last_idx = _msgs | length - 1 %}
|
| 147 |
-
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 148 |
-
{%- for message in _msgs[::-1] %}
|
| 149 |
-
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 150 |
-
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 151 |
-
{%- set _rc = render_content(message.content, false) | trim %}
|
| 152 |
-
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 153 |
-
{%- set ns.multi_step_tool = false %}
|
| 154 |
-
{%- set ns.last_query_index = index %}
|
| 155 |
-
{%- endif %}
|
| 156 |
-
{%- endif %}
|
| 157 |
-
{%- endfor %}
|
| 158 |
-
{%- if ns.multi_step_tool %}
|
| 159 |
-
{%- if _last_idx > 50 %}
|
| 160 |
-
{%- set ns.last_query_index = _last_idx %}
|
| 161 |
-
{%- else %}
|
| 162 |
-
{%- set ns.last_query_index = 0 %}
|
| 163 |
-
{%- endif %}
|
| 164 |
-
{%- endif %}
|
| 165 |
-
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 166 |
-
{%- for message in _msgs %}
|
| 167 |
-
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 168 |
-
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 169 |
-
{%- if is_system or message.role == 'user' %}
|
| 170 |
-
{%- if '<|think_off|>' in content %}
|
| 171 |
-
{%- set ns_state.thinking = false %}
|
| 172 |
-
{%- set content = content.split('<|think_off|>') | join('') | trim %}
|
| 173 |
-
{%- elif '<|think_on|>' in content %}
|
| 174 |
-
{%- set ns_state.thinking = true %}
|
| 175 |
-
{%- set content = content.split('<|think_on|>') | join('') | trim %}
|
| 176 |
-
{%- endif %}
|
| 177 |
-
{%- endif %}
|
| 178 |
-
{%- if is_system %}
|
| 179 |
-
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 180 |
-
{%- elif message.role == 'user' %}
|
| 181 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 182 |
-
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 183 |
-
{%- elif message.role == 'assistant' %}
|
| 184 |
-
{%- set reasoning_content = '' %}
|
| 185 |
-
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 186 |
-
{%- if message.reasoning_content is string %}
|
| 187 |
-
{%- set reasoning_content = message.reasoning_content %}
|
| 188 |
-
{%- else %}
|
| 189 |
-
{%- set reasoning_content = message.reasoning_content | string %}
|
| 190 |
-
{%- endif %}
|
| 191 |
-
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 192 |
-
{%- if message.thinking is string %}
|
| 193 |
-
{%- set reasoning_content = message.thinking %}
|
| 194 |
-
{%- else %}
|
| 195 |
-
{%- set reasoning_content = message.thinking | string %}
|
| 196 |
-
{%- endif %}
|
| 197 |
-
{%- else %}
|
| 198 |
-
{%- set _think_end = '' %}
|
| 199 |
-
{%- if content.startswith('</think>') %}
|
| 200 |
-
{%- set _think_end = '</think>' %}
|
| 201 |
-
{%- elif content.startswith('</thinking>') %}
|
| 202 |
-
{%- set _think_end = '</thinking>' %}
|
| 203 |
-
{%- elif '\n</think>' in content %}
|
| 204 |
-
{%- set _think_end = '\n</think>' %}
|
| 205 |
-
{%- elif '\n</thinking>' in content %}
|
| 206 |
-
{%- set _think_end = '\n</thinking>' %}
|
| 207 |
-
{%- elif '\n</ think>' in content %}
|
| 208 |
-
{%- set _think_end = '\n</ think>' %}
|
| 209 |
-
{%- elif '\n</think >' in content %}
|
| 210 |
-
{%- set _think_end = '\n</think >' %}
|
| 211 |
-
{%- endif %}
|
| 212 |
-
{%- if _think_end %}
|
| 213 |
-
{%- if 'thinking' in _think_end %}
|
| 214 |
-
{%- set _think_start = '<thinking>' %}
|
| 215 |
-
{%- else %}
|
| 216 |
-
{%- set _think_start = '<think>' %}
|
| 217 |
-
{%- endif %}
|
| 218 |
-
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 219 |
-
{%- if _think_start in reasoning_content %}
|
| 220 |
-
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 221 |
-
{%- endif %}
|
| 222 |
-
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 223 |
-
{%- endif %}
|
| 224 |
-
{%- endif %}
|
| 225 |
-
{%- set reasoning_content = reasoning_content | trim %}
|
| 226 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 227 |
-
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 228 |
-
{%- else %}
|
| 229 |
-
{{- '<|im_start|>assistant\n' + content }}
|
| 230 |
-
{%- endif %}
|
| 231 |
-
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 232 |
-
{%- for tool_call in message.tool_calls %}
|
| 233 |
-
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 234 |
-
{%- set tc = tool_call.function %}
|
| 235 |
-
{%- else %}
|
| 236 |
-
{%- set tc = tool_call %}
|
| 237 |
-
{%- endif %}
|
| 238 |
-
{%- if _tool_format == 'json' %}
|
| 239 |
-
{%- if not loop.first or content | trim %}
|
| 240 |
-
{{- '\n\n' }}
|
| 241 |
-
{%- endif %}
|
| 242 |
-
{%- set _args = '{}' %}
|
| 243 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 244 |
-
{%- if tc.arguments is mapping %}
|
| 245 |
-
{%- set _args = tc.arguments | tojson %}
|
| 246 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 247 |
-
{%- set _args = tc.arguments %}
|
| 248 |
-
{%- endif %}
|
| 249 |
-
{%- endif %}
|
| 250 |
-
{{- '<tool_call>\n{"name": ' }}{{- tc.name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
| 251 |
-
{%- else %}
|
| 252 |
-
{%- if loop.first %}
|
| 253 |
-
{%- if content | trim %}
|
| 254 |
-
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 255 |
-
{%- else %}
|
| 256 |
-
{{- '<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 257 |
-
{%- endif %}
|
| 258 |
-
{%- else %}
|
| 259 |
-
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 260 |
-
{%- endif %}
|
| 261 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 262 |
-
{%- if tc.arguments is mapping %}
|
| 263 |
-
{%- for args_name, args_value in tc.arguments.items() %}
|
| 264 |
-
{{- '<parameter=' + args_name + '>\n' }}
|
| 265 |
-
{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
|
| 266 |
-
{%- set _av = args_value | tojson %}
|
| 267 |
-
{%- else %}
|
| 268 |
-
{%- set _av = args_value | string %}
|
| 269 |
-
{%- endif %}
|
| 270 |
-
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 271 |
-
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 272 |
-
{%- else %}
|
| 273 |
-
{{- _av }}
|
| 274 |
-
{%- endif %}
|
| 275 |
-
{{- '\n</parameter>\n' }}
|
| 276 |
-
{%- endfor %}
|
| 277 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 278 |
-
{{- tc.arguments }}
|
| 279 |
-
{%- endif %}
|
| 280 |
-
{%- endif %}
|
| 281 |
-
{{- '</function>\n</tool_call>' }}
|
| 282 |
-
{%- endif %}
|
| 283 |
-
{%- endfor %}
|
| 284 |
-
{%- endif %}
|
| 285 |
-
{{- '<|im_end|>\n' }}
|
| 286 |
-
{%- elif message.role == 'tool' %}
|
| 287 |
-
{%- set _content_lower = content | lower %}
|
| 288 |
-
{%- set _content_head = _content_lower[:80] %}
|
| 289 |
-
{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}
|
| 290 |
-
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 291 |
-
{%- else %}
|
| 292 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 293 |
-
{%- endif %}
|
| 294 |
-
{%- if ns2.prev_role != 'tool' %}
|
| 295 |
-
{{- '<|im_start|>user' }}
|
| 296 |
-
{%- endif %}
|
| 297 |
-
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 298 |
-
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 299 |
-
{%- endif %}
|
| 300 |
-
{{- '\n<tool_response>\n' + content }}
|
| 301 |
-
{%- if ns2.consecutive_failures >= 2 %}
|
| 302 |
-
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 303 |
-
{%- elif ns2.consecutive_failures == 1 %}
|
| 304 |
-
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 305 |
-
{%- endif %}
|
| 306 |
-
{{- '\n</tool_response>' }}
|
| 307 |
-
{%- if loop.last %}
|
| 308 |
-
{{- '<|im_end|>\n' }}
|
| 309 |
-
{%- else %}
|
| 310 |
-
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 311 |
-
{%- if _next_role != 'tool' %}
|
| 312 |
-
{{- '<|im_end|>\n' }}
|
| 313 |
-
{%- endif %}
|
| 314 |
-
{%- endif %}
|
| 315 |
-
{%- else %}
|
| 316 |
-
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 317 |
-
{%- endif %}
|
| 318 |
-
{%- set ns2.prev_role = message.role %}
|
| 319 |
-
{%- endfor %}
|
| 320 |
-
{%- if add_generation_prompt %}
|
| 321 |
-
{{- '<|im_start|>assistant\n' }}
|
| 322 |
-
{%- if not ns_state.thinking %}
|
| 323 |
-
{{- '<think>\n\n</think>\n\n' }}
|
| 324 |
-
{%- elif ns2.consecutive_failures >= 2 %}
|
| 325 |
-
{{- '<think>\n\n</think>\n\n' }}
|
| 326 |
-
{%- else %}
|
| 327 |
-
{{- '<think>\n' }}
|
| 328 |
-
{%- endif %}
|
| 329 |
-
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/v21_chat_template_oneline.txt
DELETED
|
@@ -1 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.6-froggeric-v21.3" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else true %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set ns_state = namespace(thinking=enable_thinking) %}{%- if auto_disable_thinking_with_tools and _has_tools %} {%- set ns_state.thinking = false %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %} {%- if content is string %} {{- content }} {%- elif content is iterable and content is not mapping %} {%- for item in content %} {%- if item is mapping %} {%- if item.type == 'image' or 'image' in item or 'image_url' in item %} {%- if is_system_content %} {{- raise_exception('System message cannot contain images.') }} {%- endif %} {%- if do_vision_count %} {%- set image_count.value = image_count.value + 1 %} {%- endif %} {%- if add_vision_id %} {{- 'Picture ' ~ image_count.value ~ ': ' }} {%- endif %} {{- '<|vision_start|><|image_pad|><|vision_end|>' }} {%- elif item.type == 'video' or 'video' in item %} {%- if is_system_content %} {{- raise_exception('System message cannot contain videos.') }} {%- endif %} {%- if do_vision_count %} {%- set video_count.value = video_count.value + 1 %} {%- endif %} {%- if add_vision_id %} {{- 'Video ' ~ video_count.value ~ ': ' }} {%- endif %} {{- '<|vision_start|><|video_pad|><|vision_end|>' }} {%- elif 'text' in item %} {{- item.text }} {%- else %} {{- raise_exception('Unexpected item type in content.') }} {%- endif %} {%- else %} {{- item | string }} {%- endif %} {%- endfor %} {%- elif content is none or content is undefined %} {{- '' }} {%- else %} {{- raise_exception('Unexpected content type.') }} {%- endif %}{%- endmacro %}{%- if not messages %} {{- raise_exception('No messages provided.') }}{%- endif %}{%- set _first_role = messages[0].role %}{%- if _first_role == 'system' or _first_role == 'developer' %} {%- set _sys_msg = messages[0] %} {%- set _msgs = messages[1:] %}{%- else %} {%- set _sys_msg = none %} {%- set _msgs = messages %}{%- endif %}{%- set _sc = '' %}{%- if _sys_msg is not none %} {%- set _sc = render_content(_sys_msg.content, false, true) | trim %} {%- if '<|think_off|>' in _sc %} {%- set ns_state.thinking = false %} {%- set _sc = _sc.split('<|think_off|>') | join('') | trim %} {%- elif '<|think_on|>' in _sc %} {%- set ns_state.thinking = true %} {%- set _sc = _sc.split('<|think_on|>') | join('') | trim %} {%- endif %}{%- endif %}{%- if _has_tools %} {{- '<|im_start|>system\n' }} {{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }} {%- for tool in tools %} {{- '\n' }} {{- tool | tojson }} {%- endfor %} {{- '\n</tools>' }} {%- set tool_instructions %}If you choose to call a function ONLY reply in the following format with NO suffix:{%- if _tool_format == 'json' %}<think>Brief explanation of tool call</think><tool_call>{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}</tool_call>{%- else %}<think>Brief explanation of tool call</think><tool_call><function=example_function_name><parameter=example_parameter_1>value_1</parameter><parameter=example_parameter_2>This is the value for the second parameterthat can spanmultiple lines</parameter></function></tool_call>{%- endif %}<IMPORTANT>Reminder:- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.{%- if _tool_format == 'json' %}- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.{%- else %}- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.{%- endif %}- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.{%- if _tool_format == 'json' %}- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.{%- else %}- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.{%- endif %}- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.- If you have all necessary data, provide your final answer directly to the user without any tool call.</IMPORTANT> {%- endset %} {{- '\n\n' ~ tool_instructions | trim }} {%- if _sc %} {{- '\n\n' + _sc }} {%- endif %} {{- '<|im_end|>\n' }}{%- else %} {%- if _sc %} {{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }} {%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %} {%- set index = (_msgs | length - 1) - loop.index0 %} {%- if ns.multi_step_tool and message.role == 'user' %} {%- set _rc = render_content(message.content, false) | trim %} {%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %} {%- set ns.multi_step_tool = false %} {%- set ns.last_query_index = index %} {%- endif %} {%- endif %}{%- endfor %}{%- if ns.multi_step_tool %} {%- if _last_idx > 50 %} {%- set ns.last_query_index = _last_idx %} {%- else %} {%- set ns.last_query_index = 0 %} {%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %} {%- set is_system = (message.role == "system" or message.role == "developer") %} {%- set content = render_content(message.content, true, is_system) | trim %} {%- if is_system or message.role == 'user' %} {%- if '<|think_off|>' in content %} {%- set ns_state.thinking = false %} {%- set content = content.split('<|think_off|>') | join('') | trim %} {%- elif '<|think_on|>' in content %} {%- set ns_state.thinking = true %} {%- set content = content.split('<|think_on|>') | join('') | trim %} {%- endif %} {%- endif %} {%- if is_system %} {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }} {%- elif message.role == 'user' %} {%- set ns2.consecutive_failures = 0 %} {{- '<|im_start|>user\n' + content + '<|im_end|>\n' }} {%- elif message.role == 'assistant' %} {%- set reasoning_content = '' %} {%- if message.reasoning_content is defined and message.reasoning_content is not none %} {%- if message.reasoning_content is string %} {%- set reasoning_content = message.reasoning_content %} {%- else %} {%- set reasoning_content = message.reasoning_content | string %} {%- endif %} {%- elif message.thinking is defined and message.thinking is not none %} {%- if message.thinking is string %} {%- set reasoning_content = message.thinking %} {%- else %} {%- set reasoning_content = message.thinking | string %} {%- endif %} {%- else %} {%- set _think_end = '' %} {%- if content.startswith('</think>') %} {%- set _think_end = '</think>' %} {%- elif content.startswith('</thinking>') %} {%- set _think_end = '</thinking>' %} {%- elif '\n</think>' in content %} {%- set _think_end = '\n</think>' %} {%- elif '\n</thinking>' in content %} {%- set _think_end = '\n</thinking>' %} {%- elif '\n</ think>' in content %} {%- set _think_end = '\n</ think>' %} {%- elif '\n</think >' in content %} {%- set _think_end = '\n</think >' %} {%- endif %} {%- if _think_end %} {%- if 'thinking' in _think_end %} {%- set _think_start = '<thinking>' %} {%- else %} {%- set _think_start = '<think>' %} {%- endif %} {%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %} {%- if _think_start in reasoning_content %} {%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %} {%- endif %} {%- set content = content.split(_think_end)[-1].lstrip('\n') %} {%- endif %} {%- endif %} {%- set reasoning_content = reasoning_content | trim %} {%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %} {{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }} {%- else %} {{- '<|im_start|>assistant\n' + content }} {%- endif %} {%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %} {%- for tool_call in message.tool_calls %} {%- if tool_call.function is defined and tool_call.function is not none %} {%- set tc = tool_call.function %} {%- else %} {%- set tc = tool_call %} {%- endif %} {%- if _tool_format == 'json' %} {%- if not loop.first or content | trim %} {{- '\n\n' }} {%- endif %} {%- set _args = '{}' %} {%- if tc.arguments is defined and tc.arguments is not none %} {%- if tc.arguments is mapping %} {%- set _args = tc.arguments | tojson %} {%- elif tc.arguments is string and tc.arguments %} {%- set _args = tc.arguments %} {%- endif %} {%- endif %} {{- '<tool_call>\n{"name": ' }}{{- tc.name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }} {%- else %} {%- if loop.first %} {%- if content | trim %} {{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }} {%- else %} {{- '<tool_call>\n<function=' + tc.name + '>\n' }} {%- endif %} {%- else %} {{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }} {%- endif %} {%- if tc.arguments is defined and tc.arguments is not none %} {%- if tc.arguments is mapping %} {%- for args_name, args_value in tc.arguments.items() %} {{- '<parameter=' + args_name + '>\n' }} {%- if args_value is mapping or (args_value is sequence and args_value is not string) %} {%- set _av = args_value | tojson %} {%- else %} {%- set _av = args_value | string %} {%- endif %} {%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %} {{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }} {%- else %} {{- _av }} {%- endif %} {{- '\n</parameter>\n' }} {%- endfor %} {%- elif tc.arguments is string and tc.arguments %} {{- tc.arguments }} {%- endif %} {%- endif %} {{- '</function>\n</tool_call>' }} {%- endif %} {%- endfor %} {%- endif %} {{- '<|im_end|>\n' }} {%- elif message.role == 'tool' %} {%- set _content_lower = content | lower %} {%- set _content_head = _content_lower[:80] %} {%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %} {%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %} {%- else %} {%- set ns2.consecutive_failures = 0 %} {%- endif %} {%- if ns2.prev_role != 'tool' %} {{- '<|im_start|>user' }} {%- endif %} {%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %} {%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %} {%- endif %} {{- '\n<tool_response>\n' + content }} {%- if ns2.consecutive_failures >= 2 %} {{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }} {%- elif ns2.consecutive_failures == 1 %} {{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }} {%- endif %} {{- '\n</tool_response>' }} {%- if loop.last %} {{- '<|im_end|>\n' }} {%- else %} {%- set _next_role = _msgs[loop.index0 + 1].role %} {%- if _next_role != 'tool' %} {{- '<|im_end|>\n' }} {%- endif %} {%- endif %} {%- else %} {{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }} {%- endif %} {%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %} {{- '<|im_start|>assistant\n' }} {%- if not ns_state.thinking %} {{- '<think>\n\n</think>\n\n' }} {%- elif ns2.consecutive_failures >= 2 %} {{- '<think>\n\n</think>\n\n' }} {%- else %} {{- '<think>\n' }} {%- endif %}{%- endif %}
|
|
|
|
|
|
archive/v22.1_chat_template.jinja
DELETED
|
@@ -1,363 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.1" %}
|
| 2 |
-
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
-
{%- set image_count = namespace(value=0) %}
|
| 4 |
-
{%- set video_count = namespace(value=0) %}
|
| 5 |
-
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 6 |
-
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 7 |
-
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 8 |
-
{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
|
| 9 |
-
{%- set _preserve_thinking = preserve_reasoning %}
|
| 10 |
-
{%- elif preserve_thinking is defined and preserve_thinking is not none %}
|
| 11 |
-
{%- set _preserve_thinking = preserve_thinking %}
|
| 12 |
-
{%- else %}
|
| 13 |
-
{%- set _preserve_thinking = true %}
|
| 14 |
-
{%- endif %}
|
| 15 |
-
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 16 |
-
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 17 |
-
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 18 |
-
{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}
|
| 19 |
-
{%- set _initial_thinking = enable_thinking %}
|
| 20 |
-
{%- set _initial_effort = 'medium' %}
|
| 21 |
-
{%- if _effort_raw == 'none' %}
|
| 22 |
-
{%- set _initial_thinking = false %}
|
| 23 |
-
{%- set _initial_effort = 'medium' %}
|
| 24 |
-
{%- elif _effort_raw == 'minimal' or _effort_raw == 'low' %}
|
| 25 |
-
{%- set _initial_effort = 'low' %}
|
| 26 |
-
{%- elif _effort_raw == 'high' or _effort_raw == 'xhigh' or _effort_raw == 'max' %}
|
| 27 |
-
{%- set _initial_effort = 'xhigh' %}
|
| 28 |
-
{%- else %}
|
| 29 |
-
{%- set _initial_effort = 'medium' %}
|
| 30 |
-
{%- endif %}
|
| 31 |
-
{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
|
| 32 |
-
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 33 |
-
{%- set ns_state.thinking = false %}
|
| 34 |
-
{%- endif %}
|
| 35 |
-
{%- for msg in messages %}
|
| 36 |
-
{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
|
| 37 |
-
{%- if msg.content is string %}
|
| 38 |
-
{%- if '<|think_off|>' in msg.content %}
|
| 39 |
-
{%- set ns_state.thinking = false %}
|
| 40 |
-
{%- elif '<|think_on|>' in msg.content %}
|
| 41 |
-
{%- set ns_state.thinking = true %}
|
| 42 |
-
{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content %}
|
| 43 |
-
{%- set ns_state.thinking = true %}
|
| 44 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 45 |
-
{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
|
| 46 |
-
{%- set ns_state.thinking = true %}
|
| 47 |
-
{%- set ns_state.effort = 'low' %}
|
| 48 |
-
{%- elif '<|think_medium|>' in msg.content %}
|
| 49 |
-
{%- set ns_state.thinking = true %}
|
| 50 |
-
{%- set ns_state.effort = 'medium' %}
|
| 51 |
-
{%- endif %}
|
| 52 |
-
{%- elif msg.content is iterable and msg.content is not mapping %}
|
| 53 |
-
{%- for item in msg.content %}
|
| 54 |
-
{%- if item is string %}
|
| 55 |
-
{%- set _item_text = item %}
|
| 56 |
-
{%- elif item is mapping and 'text' in item and item.text is string %}
|
| 57 |
-
{%- set _item_text = item.text %}
|
| 58 |
-
{%- else %}
|
| 59 |
-
{%- set _item_text = '' %}
|
| 60 |
-
{%- endif %}
|
| 61 |
-
{%- if _item_text %}
|
| 62 |
-
{%- if '<|think_off|>' in _item_text %}
|
| 63 |
-
{%- set ns_state.thinking = false %}
|
| 64 |
-
{%- elif '<|think_on|>' in _item_text %}
|
| 65 |
-
{%- set ns_state.thinking = true %}
|
| 66 |
-
{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text %}
|
| 67 |
-
{%- set ns_state.thinking = true %}
|
| 68 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 69 |
-
{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
|
| 70 |
-
{%- set ns_state.thinking = true %}
|
| 71 |
-
{%- set ns_state.effort = 'low' %}
|
| 72 |
-
{%- elif '<|think_medium|>' in _item_text %}
|
| 73 |
-
{%- set ns_state.thinking = true %}
|
| 74 |
-
{%- set ns_state.effort = 'medium' %}
|
| 75 |
-
{%- endif %}
|
| 76 |
-
{%- endif %}
|
| 77 |
-
{%- endfor %}
|
| 78 |
-
{%- endif %}
|
| 79 |
-
{%- endif %}
|
| 80 |
-
{%- endfor %}
|
| 81 |
-
{%- set reasoning_instructions = '' %}
|
| 82 |
-
{%- if ns_state.thinking %}
|
| 83 |
-
{%- if ns_state.effort == 'xhigh' %}
|
| 84 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
|
| 85 |
-
{%- elif ns_state.effort == 'low' %}
|
| 86 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
|
| 87 |
-
{%- endif %}
|
| 88 |
-
{%- endif %}
|
| 89 |
-
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 90 |
-
{%- if content is string %}
|
| 91 |
-
{{- content }}
|
| 92 |
-
{%- elif content is iterable and content is not mapping %}
|
| 93 |
-
{%- for item in content %}
|
| 94 |
-
{%- if item is mapping %}
|
| 95 |
-
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 96 |
-
{%- if is_system_content %}
|
| 97 |
-
{{- raise_exception('System message cannot contain images.') }}
|
| 98 |
-
{%- endif %}
|
| 99 |
-
{%- if do_vision_count %}
|
| 100 |
-
{%- set image_count.value = image_count.value + 1 %}
|
| 101 |
-
{%- endif %}
|
| 102 |
-
{%- if add_vision_id %}
|
| 103 |
-
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 104 |
-
{%- endif %}
|
| 105 |
-
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 106 |
-
{%- elif item.type == 'video' or 'video' in item %}
|
| 107 |
-
{%- if is_system_content %}
|
| 108 |
-
{{- raise_exception('System message cannot contain videos.') }}
|
| 109 |
-
{%- endif %}
|
| 110 |
-
{%- if do_vision_count %}
|
| 111 |
-
{%- set video_count.value = video_count.value + 1 %}
|
| 112 |
-
{%- endif %}
|
| 113 |
-
{%- if add_vision_id %}
|
| 114 |
-
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 115 |
-
{%- endif %}
|
| 116 |
-
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 117 |
-
{%- elif 'text' in item %}
|
| 118 |
-
{{- item.text }}
|
| 119 |
-
{%- else %}
|
| 120 |
-
{{- raise_exception('Unexpected item type in content.') }}
|
| 121 |
-
{%- endif %}
|
| 122 |
-
{%- else %}
|
| 123 |
-
{{- item | string }}
|
| 124 |
-
{%- endif %}
|
| 125 |
-
{%- endfor %}
|
| 126 |
-
{%- elif content is none or content is undefined %}
|
| 127 |
-
{{- '' }}
|
| 128 |
-
{%- else %}
|
| 129 |
-
{{- raise_exception('Unexpected content type.') }}
|
| 130 |
-
{%- endif %}
|
| 131 |
-
{%- endmacro %}
|
| 132 |
-
{%- if not messages %}
|
| 133 |
-
{{- raise_exception('No messages provided.') }}
|
| 134 |
-
{%- endif %}
|
| 135 |
-
{%- set _first_role = messages[0].role %}
|
| 136 |
-
{%- if _first_role == 'system' or _first_role == 'developer' %}
|
| 137 |
-
{%- set _sys_msg = messages[0] %}
|
| 138 |
-
{%- set _msgs = messages[1:] %}
|
| 139 |
-
{%- else %}
|
| 140 |
-
{%- set _sys_msg = none %}
|
| 141 |
-
{%- set _msgs = messages %}
|
| 142 |
-
{%- endif %}
|
| 143 |
-
{%- set _sc = '' %}
|
| 144 |
-
{%- if _sys_msg is not none %}
|
| 145 |
-
{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}
|
| 146 |
-
{%- if '<|think_off|>' in _sc %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 147 |
-
{%- if '<|think_on|>' in _sc %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 148 |
-
{%- if '<|think_xhigh|>' in _sc %}{%- set _sc = _sc.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 149 |
-
{%- if '<|think_high|>' in _sc %}{%- set _sc = _sc.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 150 |
-
{%- if '<|think_medium|>' in _sc %}{%- set _sc = _sc.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 151 |
-
{%- if '<|think_low|>' in _sc %}{%- set _sc = _sc.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 152 |
-
{%- if '<|think_minimal|>' in _sc %}{%- set _sc = _sc.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 153 |
-
{%- endif %}
|
| 154 |
-
{%- if _has_tools %}
|
| 155 |
-
{{- '<|im_start|>system\n' }}
|
| 156 |
-
{%- if reasoning_instructions %}
|
| 157 |
-
{{- reasoning_instructions + '\n\n' }}
|
| 158 |
-
{%- endif %}
|
| 159 |
-
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 160 |
-
{%- for tool in tools %}
|
| 161 |
-
{{- '\n' }}
|
| 162 |
-
{{- tool | tojson }}
|
| 163 |
-
{%- endfor %}
|
| 164 |
-
{{- '\n</tools>' }}
|
| 165 |
-
{%- if _tool_format == 'json' %}
|
| 166 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 167 |
-
{%- else %}
|
| 168 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 169 |
-
{%- endif %}
|
| 170 |
-
{%- if _sc %}
|
| 171 |
-
{{- '\n\n' + _sc }}
|
| 172 |
-
{%- endif %}
|
| 173 |
-
{{- '<|im_end|>\n' }}
|
| 174 |
-
{%- else %}
|
| 175 |
-
{%- if _sc %}
|
| 176 |
-
{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
|
| 177 |
-
{%- elif reasoning_instructions %}
|
| 178 |
-
{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
|
| 179 |
-
{%- endif %}
|
| 180 |
-
{%- endif %}
|
| 181 |
-
{%- set _last_idx = _msgs | length - 1 %}
|
| 182 |
-
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 183 |
-
{%- for message in _msgs[::-1] %}
|
| 184 |
-
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 185 |
-
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 186 |
-
{%- set _rc = render_content(message.content, false) | trim %}
|
| 187 |
-
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 188 |
-
{%- set ns.multi_step_tool = false %}
|
| 189 |
-
{%- set ns.last_query_index = index %}
|
| 190 |
-
{%- endif %}
|
| 191 |
-
{%- endif %}
|
| 192 |
-
{%- endfor %}
|
| 193 |
-
{%- if ns.multi_step_tool %}
|
| 194 |
-
{%- if _last_idx > 50 %}
|
| 195 |
-
{%- set ns.last_query_index = _last_idx %}
|
| 196 |
-
{%- else %}
|
| 197 |
-
{%- set ns.last_query_index = 0 %}
|
| 198 |
-
{%- endif %}
|
| 199 |
-
{%- endif %}
|
| 200 |
-
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 201 |
-
{%- for message in _msgs %}
|
| 202 |
-
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 203 |
-
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 204 |
-
{%- if is_system or message.role == 'user' %}
|
| 205 |
-
{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 206 |
-
{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 207 |
-
{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 208 |
-
{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 209 |
-
{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 210 |
-
{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 211 |
-
{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 212 |
-
{%- endif %}
|
| 213 |
-
{%- if is_system %}
|
| 214 |
-
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 215 |
-
{%- elif message.role == 'user' %}
|
| 216 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 217 |
-
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 218 |
-
{%- elif message.role == 'assistant' %}
|
| 219 |
-
{%- set reasoning_content = '' %}
|
| 220 |
-
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 221 |
-
{%- if message.reasoning_content is string %}
|
| 222 |
-
{%- set reasoning_content = message.reasoning_content %}
|
| 223 |
-
{%- else %}
|
| 224 |
-
{%- set reasoning_content = message.reasoning_content | string %}
|
| 225 |
-
{%- endif %}
|
| 226 |
-
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 227 |
-
{%- if message.thinking is string %}
|
| 228 |
-
{%- set reasoning_content = message.thinking %}
|
| 229 |
-
{%- else %}
|
| 230 |
-
{%- set reasoning_content = message.thinking | string %}
|
| 231 |
-
{%- endif %}
|
| 232 |
-
{%- else %}
|
| 233 |
-
{%- set _think_end = '' %}
|
| 234 |
-
{%- if content.startswith('</think>') %}
|
| 235 |
-
{%- set _think_end = '</think>' %}
|
| 236 |
-
{%- elif content.startswith('</thinking>') %}
|
| 237 |
-
{%- set _think_end = '</thinking>' %}
|
| 238 |
-
{%- elif '\n</think>' in content %}
|
| 239 |
-
{%- set _think_end = '\n</think>' %}
|
| 240 |
-
{%- elif '\n</thinking>' in content %}
|
| 241 |
-
{%- set _think_end = '\n</thinking>' %}
|
| 242 |
-
{%- elif '\n</ think>' in content %}
|
| 243 |
-
{%- set _think_end = '\n</ think>' %}
|
| 244 |
-
{%- elif '\n</think >' in content %}
|
| 245 |
-
{%- set _think_end = '\n</think >' %}
|
| 246 |
-
{%- endif %}
|
| 247 |
-
{%- if _think_end %}
|
| 248 |
-
{%- if 'thinking' in _think_end %}
|
| 249 |
-
{%- set _think_start = '<thinking>' %}
|
| 250 |
-
{%- else %}
|
| 251 |
-
{%- set _think_start = '<think>' %}
|
| 252 |
-
{%- endif %}
|
| 253 |
-
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 254 |
-
{%- if _think_start in reasoning_content %}
|
| 255 |
-
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 256 |
-
{%- endif %}
|
| 257 |
-
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 258 |
-
{%- endif %}
|
| 259 |
-
{%- endif %}
|
| 260 |
-
{%- set reasoning_content = reasoning_content | trim %}
|
| 261 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 262 |
-
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 263 |
-
{%- else %}
|
| 264 |
-
{{- '<|im_start|>assistant\n' + content }}
|
| 265 |
-
{%- endif %}
|
| 266 |
-
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 267 |
-
{%- for tool_call in message.tool_calls %}
|
| 268 |
-
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 269 |
-
{%- set tc = tool_call.function %}
|
| 270 |
-
{%- else %}
|
| 271 |
-
{%- set tc = tool_call %}
|
| 272 |
-
{%- endif %}
|
| 273 |
-
{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
|
| 274 |
-
{%- if _tool_format == 'json' %}
|
| 275 |
-
{%- if not loop.first or content | trim %}
|
| 276 |
-
{{- '\n\n' }}
|
| 277 |
-
{%- endif %}
|
| 278 |
-
{%- set _args = '{}' %}
|
| 279 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 280 |
-
{%- if tc.arguments is mapping %}
|
| 281 |
-
{%- set _args = tc.arguments | tojson %}
|
| 282 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 283 |
-
{%- set _args = tc.arguments %}
|
| 284 |
-
{%- endif %}
|
| 285 |
-
{%- endif %}
|
| 286 |
-
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
| 287 |
-
{%- else %}
|
| 288 |
-
{%- if loop.first %}
|
| 289 |
-
{%- if content | trim %}
|
| 290 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 291 |
-
{%- else %}
|
| 292 |
-
{{- '<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 293 |
-
{%- endif %}
|
| 294 |
-
{%- else %}
|
| 295 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 296 |
-
{%- endif %}
|
| 297 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 298 |
-
{%- if tc.arguments is mapping %}
|
| 299 |
-
{%- for args_name, args_value in tc.arguments.items() %}
|
| 300 |
-
{{- '<parameter=' + args_name + '>\n' }}
|
| 301 |
-
{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
|
| 302 |
-
{%- set _av = args_value | tojson %}
|
| 303 |
-
{%- else %}
|
| 304 |
-
{%- set _av = args_value | string %}
|
| 305 |
-
{%- endif %}
|
| 306 |
-
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 307 |
-
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 308 |
-
{%- else %}
|
| 309 |
-
{{- _av }}
|
| 310 |
-
{%- endif %}
|
| 311 |
-
{{- '\n</parameter>\n' }}
|
| 312 |
-
{%- endfor %}
|
| 313 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 314 |
-
{{- tc.arguments }}
|
| 315 |
-
{%- endif %}
|
| 316 |
-
{%- endif %}
|
| 317 |
-
{{- '</function>\n</tool_call>' }}
|
| 318 |
-
{%- endif %}
|
| 319 |
-
{%- endfor %}
|
| 320 |
-
{%- endif %}
|
| 321 |
-
{{- '<|im_end|>\n' }}
|
| 322 |
-
{%- elif message.role == 'tool' %}
|
| 323 |
-
{%- set _content_lower = content | lower %}
|
| 324 |
-
{%- set _content_head = _content_lower[:80] %}
|
| 325 |
-
{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}
|
| 326 |
-
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 327 |
-
{%- else %}
|
| 328 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 329 |
-
{%- endif %}
|
| 330 |
-
{%- if ns2.prev_role != 'tool' %}
|
| 331 |
-
{{- '<|im_start|>user' }}
|
| 332 |
-
{%- endif %}
|
| 333 |
-
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 334 |
-
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 335 |
-
{%- endif %}
|
| 336 |
-
{{- '\n<tool_response>\n' + content }}
|
| 337 |
-
{%- if ns2.consecutive_failures >= 2 %}
|
| 338 |
-
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 339 |
-
{%- elif ns2.consecutive_failures == 1 %}
|
| 340 |
-
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 341 |
-
{%- endif %}
|
| 342 |
-
{{- '\n</tool_response>' }}
|
| 343 |
-
{%- if loop.last %}
|
| 344 |
-
{{- '<|im_end|>\n' }}
|
| 345 |
-
{%- else %}
|
| 346 |
-
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 347 |
-
{%- if _next_role != 'tool' %}
|
| 348 |
-
{{- '<|im_end|>\n' }}
|
| 349 |
-
{%- endif %}
|
| 350 |
-
{%- endif %}
|
| 351 |
-
{%- else %}
|
| 352 |
-
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 353 |
-
{%- endif %}
|
| 354 |
-
{%- set ns2.prev_role = message.role %}
|
| 355 |
-
{%- endfor %}
|
| 356 |
-
{%- if add_generation_prompt %}
|
| 357 |
-
{{- '<|im_start|>assistant\n' }}
|
| 358 |
-
{%- if not ns_state.thinking %}
|
| 359 |
-
{{- '<think>\n\n</think>\n\n' }}
|
| 360 |
-
{%- else %}
|
| 361 |
-
{{- '<think>\n' }}
|
| 362 |
-
{%- endif %}
|
| 363 |
-
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/v22.1_chat_template_oneline.txt
DELETED
|
@@ -1 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.1" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- if preserve_reasoning is defined and preserve_reasoning is not none %}{%- set _preserve_thinking = preserve_reasoning %}{%- elif preserve_thinking is defined and preserve_thinking is not none %}{%- set _preserve_thinking = preserve_thinking %}{%- else %}{%- set _preserve_thinking = true %}{%- endif %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}{%- set _initial_thinking = enable_thinking %}{%- set _initial_effort = 'medium' %}{%- if _effort_raw == 'none' %}{%- set _initial_thinking = false %}{%- set _initial_effort = 'medium' %}{%- elif _effort_raw == 'minimal' or _effort_raw == 'low' %}{%- set _initial_effort = 'low' %}{%- elif _effort_raw == 'high' or _effort_raw == 'xhigh' or _effort_raw == 'max' %}{%- set _initial_effort = 'xhigh' %}{%- else %}{%- set _initial_effort = 'medium' %}{%- endif %}{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- for msg in messages %}{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}{%- if msg.content is string %}{%- if '<|think_off|>' in msg.content %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in msg.content %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- elif msg.content is iterable and msg.content is not mapping %}{%- for item in msg.content %}{%- if item is string %}{%- set _item_text = item %}{%- elif item is mapping and 'text' in item and item.text is string %}{%- set _item_text = item.text %}{%- else %}{%- set _item_text = '' %}{%- endif %}{%- if _item_text %}{%- if '<|think_off|>' in _item_text %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in _item_text %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- endif %}{%- endfor %}{%- endif %}{%- endif %}{%- endfor %}{%- set reasoning_instructions = '' %}{%- if ns_state.thinking %}{%- if ns_state.effort == 'xhigh' %}{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}{%- elif ns_state.effort == 'low' %}{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}{%- endif %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set _first_role = messages[0].role %}{%- if _first_role == 'system' or _first_role == 'developer' %}{%- set _sys_msg = messages[0] %}{%- set _msgs = messages[1:] %}{%- else %}{%- set _sys_msg = none %}{%- set _msgs = messages %}{%- endif %}{%- set _sc = '' %}{%- if _sys_msg is not none %}{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}{%- if '<|think_off|>' in _sc %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in _sc %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in _sc %}{%- set _sc = _sc.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in _sc %}{%- set _sc = _sc.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in _sc %}{%- set _sc = _sc.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in _sc %}{%- set _sc = _sc.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in _sc %}{%- set _sc = _sc.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set reasoning_content = message.thinking %}{%- else %}{%- set reasoning_content = message.thinking | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if not loop.first or content | trim %}{{- '\n\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string and tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}{%- set _av = args_value | tojson %}{%- else %}{%- set _av = args_value | string %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:80] %}{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
|
|
|
|
|
archive/v22.2_chat_template.jinja
DELETED
|
@@ -1,380 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.2" %}
|
| 2 |
-
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
-
{%- set image_count = namespace(value=0) %}
|
| 4 |
-
{%- set video_count = namespace(value=0) %}
|
| 5 |
-
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 6 |
-
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 7 |
-
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 8 |
-
{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
|
| 9 |
-
{%- set _preserve_thinking = preserve_reasoning %}
|
| 10 |
-
{%- elif preserve_thinking is defined and preserve_thinking is not none %}
|
| 11 |
-
{%- set _preserve_thinking = preserve_thinking %}
|
| 12 |
-
{%- else %}
|
| 13 |
-
{%- set _preserve_thinking = true %}
|
| 14 |
-
{%- endif %}
|
| 15 |
-
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 16 |
-
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 17 |
-
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 18 |
-
{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}
|
| 19 |
-
{%- set _initial_thinking = enable_thinking %}
|
| 20 |
-
{%- set _initial_effort = 'medium' %}
|
| 21 |
-
{%- if _effort_raw in ('none', 'off') %}
|
| 22 |
-
{%- set _initial_thinking = false %}
|
| 23 |
-
{%- set _initial_effort = 'medium' %}
|
| 24 |
-
{%- elif _effort_raw in ('minimal', 'low') %}
|
| 25 |
-
{%- set _initial_effort = 'low' %}
|
| 26 |
-
{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
|
| 27 |
-
{%- set _initial_effort = 'xhigh' %}
|
| 28 |
-
{%- else %}
|
| 29 |
-
{%- set _initial_effort = 'medium' %}
|
| 30 |
-
{%- endif %}
|
| 31 |
-
{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
|
| 32 |
-
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 33 |
-
{%- set ns_state.thinking = false %}
|
| 34 |
-
{%- endif %}
|
| 35 |
-
{%- for msg in messages %}
|
| 36 |
-
{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
|
| 37 |
-
{%- if msg.content is string %}
|
| 38 |
-
{%- if '<|think_off|>' in msg.content %}
|
| 39 |
-
{%- set ns_state.thinking = false %}
|
| 40 |
-
{%- elif '<|think_on|>' in msg.content %}
|
| 41 |
-
{%- set ns_state.thinking = true %}
|
| 42 |
-
{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
|
| 43 |
-
{%- set ns_state.thinking = true %}
|
| 44 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 45 |
-
{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
|
| 46 |
-
{%- set ns_state.thinking = true %}
|
| 47 |
-
{%- set ns_state.effort = 'low' %}
|
| 48 |
-
{%- elif '<|think_medium|>' in msg.content %}
|
| 49 |
-
{%- set ns_state.thinking = true %}
|
| 50 |
-
{%- set ns_state.effort = 'medium' %}
|
| 51 |
-
{%- endif %}
|
| 52 |
-
{%- elif msg.content is iterable and msg.content is not mapping %}
|
| 53 |
-
{%- for item in msg.content %}
|
| 54 |
-
{%- if item is string %}
|
| 55 |
-
{%- set _item_text = item %}
|
| 56 |
-
{%- elif item is mapping and 'text' in item and item.text is string %}
|
| 57 |
-
{%- set _item_text = item.text %}
|
| 58 |
-
{%- else %}
|
| 59 |
-
{%- set _item_text = '' %}
|
| 60 |
-
{%- endif %}
|
| 61 |
-
{%- if _item_text %}
|
| 62 |
-
{%- if '<|think_off|>' in _item_text %}
|
| 63 |
-
{%- set ns_state.thinking = false %}
|
| 64 |
-
{%- elif '<|think_on|>' in _item_text %}
|
| 65 |
-
{%- set ns_state.thinking = true %}
|
| 66 |
-
{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
|
| 67 |
-
{%- set ns_state.thinking = true %}
|
| 68 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 69 |
-
{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
|
| 70 |
-
{%- set ns_state.thinking = true %}
|
| 71 |
-
{%- set ns_state.effort = 'low' %}
|
| 72 |
-
{%- elif '<|think_medium|>' in _item_text %}
|
| 73 |
-
{%- set ns_state.thinking = true %}
|
| 74 |
-
{%- set ns_state.effort = 'medium' %}
|
| 75 |
-
{%- endif %}
|
| 76 |
-
{%- endif %}
|
| 77 |
-
{%- endfor %}
|
| 78 |
-
{%- endif %}
|
| 79 |
-
{%- endif %}
|
| 80 |
-
{%- endfor %}
|
| 81 |
-
{%- set reasoning_instructions = '' %}
|
| 82 |
-
{%- if ns_state.thinking %}
|
| 83 |
-
{%- if ns_state.effort == 'xhigh' %}
|
| 84 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
|
| 85 |
-
{%- elif ns_state.effort == 'low' %}
|
| 86 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
|
| 87 |
-
{%- endif %}
|
| 88 |
-
{%- endif %}
|
| 89 |
-
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 90 |
-
{%- if content is string %}
|
| 91 |
-
{{- content }}
|
| 92 |
-
{%- elif content is iterable and content is not mapping %}
|
| 93 |
-
{%- for item in content %}
|
| 94 |
-
{%- if item is mapping %}
|
| 95 |
-
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 96 |
-
{%- if is_system_content %}
|
| 97 |
-
{{- raise_exception('System message cannot contain images.') }}
|
| 98 |
-
{%- endif %}
|
| 99 |
-
{%- if do_vision_count %}
|
| 100 |
-
{%- set image_count.value = image_count.value + 1 %}
|
| 101 |
-
{%- endif %}
|
| 102 |
-
{%- if add_vision_id %}
|
| 103 |
-
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 104 |
-
{%- endif %}
|
| 105 |
-
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 106 |
-
{%- elif item.type == 'video' or 'video' in item %}
|
| 107 |
-
{%- if is_system_content %}
|
| 108 |
-
{{- raise_exception('System message cannot contain videos.') }}
|
| 109 |
-
{%- endif %}
|
| 110 |
-
{%- if do_vision_count %}
|
| 111 |
-
{%- set video_count.value = video_count.value + 1 %}
|
| 112 |
-
{%- endif %}
|
| 113 |
-
{%- if add_vision_id %}
|
| 114 |
-
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 115 |
-
{%- endif %}
|
| 116 |
-
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 117 |
-
{%- elif 'text' in item %}
|
| 118 |
-
{{- item.text }}
|
| 119 |
-
{%- else %}
|
| 120 |
-
{{- raise_exception('Unexpected item type in content.') }}
|
| 121 |
-
{%- endif %}
|
| 122 |
-
{%- else %}
|
| 123 |
-
{{- item | string }}
|
| 124 |
-
{%- endif %}
|
| 125 |
-
{%- endfor %}
|
| 126 |
-
{%- elif content is none or content is undefined %}
|
| 127 |
-
{{- '' }}
|
| 128 |
-
{%- else %}
|
| 129 |
-
{{- raise_exception('Unexpected content type.') }}
|
| 130 |
-
{%- endif %}
|
| 131 |
-
{%- endmacro %}
|
| 132 |
-
{%- if not messages %}
|
| 133 |
-
{{- raise_exception('No messages provided.') }}
|
| 134 |
-
{%- endif %}
|
| 135 |
-
{%- set head = namespace(count=0, seen_non_system=false) %}
|
| 136 |
-
{%- for message in messages %}
|
| 137 |
-
{%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}
|
| 138 |
-
{%- if _is_sys and not head.seen_non_system %}
|
| 139 |
-
{%- set head.count = head.count + 1 %}
|
| 140 |
-
{%- else %}
|
| 141 |
-
{%- set head.seen_non_system = true %}
|
| 142 |
-
{%- endif %}
|
| 143 |
-
{%- endfor %}
|
| 144 |
-
{%- set sys_state = namespace(content='') %}
|
| 145 |
-
{%- for message in messages[:head.count] %}
|
| 146 |
-
{%- set _part = render_content(message.content, false, true) | trim %}
|
| 147 |
-
{%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 148 |
-
{%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 149 |
-
{%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 150 |
-
{%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 151 |
-
{%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
|
| 152 |
-
{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
|
| 153 |
-
{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
|
| 154 |
-
{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 155 |
-
{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 156 |
-
{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 157 |
-
{%- if _part %}
|
| 158 |
-
{%- if sys_state.content %}
|
| 159 |
-
{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
|
| 160 |
-
{%- else %}
|
| 161 |
-
{%- set sys_state.content = _part %}
|
| 162 |
-
{%- endif %}
|
| 163 |
-
{%- endif %}
|
| 164 |
-
{%- endfor %}
|
| 165 |
-
{%- set _sc = sys_state.content %}
|
| 166 |
-
{%- set _msgs = messages[head.count:] %}
|
| 167 |
-
{%- if _has_tools %}
|
| 168 |
-
{{- '<|im_start|>system\n' }}
|
| 169 |
-
{%- if reasoning_instructions %}
|
| 170 |
-
{{- reasoning_instructions + '\n\n' }}
|
| 171 |
-
{%- endif %}
|
| 172 |
-
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 173 |
-
{%- for tool in tools %}
|
| 174 |
-
{{- '\n' }}
|
| 175 |
-
{{- tool | tojson }}
|
| 176 |
-
{%- endfor %}
|
| 177 |
-
{{- '\n</tools>' }}
|
| 178 |
-
{%- if _tool_format == 'json' %}
|
| 179 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 180 |
-
{%- else %}
|
| 181 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 182 |
-
{%- endif %}
|
| 183 |
-
{%- if _sc %}
|
| 184 |
-
{{- '\n\n' + _sc }}
|
| 185 |
-
{%- endif %}
|
| 186 |
-
{{- '<|im_end|>\n' }}
|
| 187 |
-
{%- else %}
|
| 188 |
-
{%- if _sc %}
|
| 189 |
-
{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
|
| 190 |
-
{%- elif reasoning_instructions %}
|
| 191 |
-
{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
|
| 192 |
-
{%- endif %}
|
| 193 |
-
{%- endif %}
|
| 194 |
-
{%- set _last_idx = _msgs | length - 1 %}
|
| 195 |
-
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 196 |
-
{%- for message in _msgs[::-1] %}
|
| 197 |
-
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 198 |
-
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 199 |
-
{%- set _rc = render_content(message.content, false) | trim %}
|
| 200 |
-
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 201 |
-
{%- set ns.multi_step_tool = false %}
|
| 202 |
-
{%- set ns.last_query_index = index %}
|
| 203 |
-
{%- endif %}
|
| 204 |
-
{%- endif %}
|
| 205 |
-
{%- endfor %}
|
| 206 |
-
{%- if ns.multi_step_tool %}
|
| 207 |
-
{%- if _last_idx > 50 %}
|
| 208 |
-
{%- set ns.last_query_index = _last_idx %}
|
| 209 |
-
{%- else %}
|
| 210 |
-
{%- set ns.last_query_index = 0 %}
|
| 211 |
-
{%- endif %}
|
| 212 |
-
{%- endif %}
|
| 213 |
-
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 214 |
-
{%- for message in _msgs %}
|
| 215 |
-
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 216 |
-
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 217 |
-
{%- if is_system or message.role == 'user' %}
|
| 218 |
-
{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 219 |
-
{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 220 |
-
{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 221 |
-
{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 222 |
-
{%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
|
| 223 |
-
{%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
|
| 224 |
-
{%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}
|
| 225 |
-
{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 226 |
-
{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 227 |
-
{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 228 |
-
{%- endif %}
|
| 229 |
-
{%- if is_system %}
|
| 230 |
-
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 231 |
-
{%- elif message.role == 'user' %}
|
| 232 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 233 |
-
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 234 |
-
{%- elif message.role == 'assistant' %}
|
| 235 |
-
{%- set reasoning_content = '' %}
|
| 236 |
-
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 237 |
-
{%- if message.reasoning_content is string %}
|
| 238 |
-
{%- set reasoning_content = message.reasoning_content %}
|
| 239 |
-
{%- else %}
|
| 240 |
-
{%- set reasoning_content = message.reasoning_content | string %}
|
| 241 |
-
{%- endif %}
|
| 242 |
-
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 243 |
-
{%- if message.thinking is string %}
|
| 244 |
-
{%- set reasoning_content = message.thinking %}
|
| 245 |
-
{%- else %}
|
| 246 |
-
{%- set reasoning_content = message.thinking | string %}
|
| 247 |
-
{%- endif %}
|
| 248 |
-
{%- else %}
|
| 249 |
-
{%- set _think_end = '' %}
|
| 250 |
-
{%- if content.startswith('</think>') %}
|
| 251 |
-
{%- set _think_end = '</think>' %}
|
| 252 |
-
{%- elif content.startswith('</thinking>') %}
|
| 253 |
-
{%- set _think_end = '</thinking>' %}
|
| 254 |
-
{%- elif '\n</think>' in content %}
|
| 255 |
-
{%- set _think_end = '\n</think>' %}
|
| 256 |
-
{%- elif '\n</thinking>' in content %}
|
| 257 |
-
{%- set _think_end = '\n</thinking>' %}
|
| 258 |
-
{%- elif '\n</ think>' in content %}
|
| 259 |
-
{%- set _think_end = '\n</ think>' %}
|
| 260 |
-
{%- elif '\n</think >' in content %}
|
| 261 |
-
{%- set _think_end = '\n</think >' %}
|
| 262 |
-
{%- endif %}
|
| 263 |
-
{%- if _think_end %}
|
| 264 |
-
{%- if 'thinking' in _think_end %}
|
| 265 |
-
{%- set _think_start = '<thinking>' %}
|
| 266 |
-
{%- else %}
|
| 267 |
-
{%- set _think_start = '<think>' %}
|
| 268 |
-
{%- endif %}
|
| 269 |
-
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 270 |
-
{%- if _think_start in reasoning_content %}
|
| 271 |
-
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 272 |
-
{%- endif %}
|
| 273 |
-
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 274 |
-
{%- endif %}
|
| 275 |
-
{%- endif %}
|
| 276 |
-
{%- set reasoning_content = reasoning_content | trim %}
|
| 277 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 278 |
-
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 279 |
-
{%- else %}
|
| 280 |
-
{{- '<|im_start|>assistant\n' + content }}
|
| 281 |
-
{%- endif %}
|
| 282 |
-
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 283 |
-
{%- for tool_call in message.tool_calls %}
|
| 284 |
-
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 285 |
-
{%- set tc = tool_call.function %}
|
| 286 |
-
{%- else %}
|
| 287 |
-
{%- set tc = tool_call %}
|
| 288 |
-
{%- endif %}
|
| 289 |
-
{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
|
| 290 |
-
{%- if _tool_format == 'json' %}
|
| 291 |
-
{%- if not loop.first or content | trim %}
|
| 292 |
-
{{- '\n\n' }}
|
| 293 |
-
{%- endif %}
|
| 294 |
-
{%- set _args = '{}' %}
|
| 295 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 296 |
-
{%- if tc.arguments is mapping %}
|
| 297 |
-
{%- set _args = tc.arguments | tojson %}
|
| 298 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 299 |
-
{%- set _args = tc.arguments %}
|
| 300 |
-
{%- endif %}
|
| 301 |
-
{%- endif %}
|
| 302 |
-
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
| 303 |
-
{%- else %}
|
| 304 |
-
{%- if loop.first %}
|
| 305 |
-
{%- if content | trim %}
|
| 306 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 307 |
-
{%- else %}
|
| 308 |
-
{{- '<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 309 |
-
{%- endif %}
|
| 310 |
-
{%- else %}
|
| 311 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 312 |
-
{%- endif %}
|
| 313 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 314 |
-
{%- if tc.arguments is mapping %}
|
| 315 |
-
{%- for args_name, args_value in tc.arguments.items() %}
|
| 316 |
-
{{- '<parameter=' + args_name + '>\n' }}
|
| 317 |
-
{%- if args_value is string %}
|
| 318 |
-
{%- set _av = args_value %}
|
| 319 |
-
{%- else %}
|
| 320 |
-
{%- set _av = args_value | tojson %}
|
| 321 |
-
{%- endif %}
|
| 322 |
-
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 323 |
-
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 324 |
-
{%- else %}
|
| 325 |
-
{{- _av }}
|
| 326 |
-
{%- endif %}
|
| 327 |
-
{{- '\n</parameter>\n' }}
|
| 328 |
-
{%- endfor %}
|
| 329 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 330 |
-
{{- tc.arguments }}
|
| 331 |
-
{%- endif %}
|
| 332 |
-
{%- endif %}
|
| 333 |
-
{{- '</function>\n</tool_call>' }}
|
| 334 |
-
{%- endif %}
|
| 335 |
-
{%- endfor %}
|
| 336 |
-
{%- endif %}
|
| 337 |
-
{{- '<|im_end|>\n' }}
|
| 338 |
-
{%- elif message.role == 'tool' %}
|
| 339 |
-
{%- set _content_lower = content | lower %}
|
| 340 |
-
{%- set _content_head = _content_lower[:120] %}
|
| 341 |
-
{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
|
| 342 |
-
{%- if not _is_code_or_grep and content | length < 600 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or _content_head.startswith('exception:') or _content_head.startswith('failed to ') or 'exit code: ' in _content_head or 'process exited with code' in _content_head) %}
|
| 343 |
-
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 344 |
-
{%- else %}
|
| 345 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 346 |
-
{%- endif %}
|
| 347 |
-
{%- if ns2.prev_role != 'tool' %}
|
| 348 |
-
{{- '<|im_start|>user' }}
|
| 349 |
-
{%- endif %}
|
| 350 |
-
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 351 |
-
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 352 |
-
{%- endif %}
|
| 353 |
-
{{- '\n<tool_response>\n' + content }}
|
| 354 |
-
{%- if ns2.consecutive_failures >= 2 %}
|
| 355 |
-
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 356 |
-
{%- elif ns2.consecutive_failures == 1 %}
|
| 357 |
-
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 358 |
-
{%- endif %}
|
| 359 |
-
{{- '\n</tool_response>' }}
|
| 360 |
-
{%- if loop.last %}
|
| 361 |
-
{{- '<|im_end|>\n' }}
|
| 362 |
-
{%- else %}
|
| 363 |
-
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 364 |
-
{%- if _next_role != 'tool' %}
|
| 365 |
-
{{- '<|im_end|>\n' }}
|
| 366 |
-
{%- endif %}
|
| 367 |
-
{%- endif %}
|
| 368 |
-
{%- else %}
|
| 369 |
-
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 370 |
-
{%- endif %}
|
| 371 |
-
{%- set ns2.prev_role = message.role %}
|
| 372 |
-
{%- endfor %}
|
| 373 |
-
{%- if add_generation_prompt %}
|
| 374 |
-
{{- '<|im_start|>assistant\n' }}
|
| 375 |
-
{%- if not ns_state.thinking %}
|
| 376 |
-
{{- '<think>\n\n</think>\n\n' }}
|
| 377 |
-
{%- else %}
|
| 378 |
-
{{- '<think>\n' }}
|
| 379 |
-
{%- endif %}
|
| 380 |
-
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/v22.2_chat_template_oneline.txt
DELETED
|
@@ -1 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.2" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- if preserve_reasoning is defined and preserve_reasoning is not none %}{%- set _preserve_thinking = preserve_reasoning %}{%- elif preserve_thinking is defined and preserve_thinking is not none %}{%- set _preserve_thinking = preserve_thinking %}{%- else %}{%- set _preserve_thinking = true %}{%- endif %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}{%- set _initial_thinking = enable_thinking %}{%- set _initial_effort = 'medium' %}{%- if _effort_raw in ('none', 'off') %}{%- set _initial_thinking = false %}{%- set _initial_effort = 'medium' %}{%- elif _effort_raw in ('minimal', 'low') %}{%- set _initial_effort = 'low' %}{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}{%- set _initial_effort = 'xhigh' %}{%- else %}{%- set _initial_effort = 'medium' %}{%- endif %}{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- for msg in messages %}{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}{%- if msg.content is string %}{%- if '<|think_off|>' in msg.content %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in msg.content %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- elif msg.content is iterable and msg.content is not mapping %}{%- for item in msg.content %}{%- if item is string %}{%- set _item_text = item %}{%- elif item is mapping and 'text' in item and item.text is string %}{%- set _item_text = item.text %}{%- else %}{%- set _item_text = '' %}{%- endif %}{%- if _item_text %}{%- if '<|think_off|>' in _item_text %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in _item_text %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- endif %}{%- endfor %}{%- endif %}{%- endif %}{%- endfor %}{%- set reasoning_instructions = '' %}{%- if ns_state.thinking %}{%- if ns_state.effort == 'xhigh' %}{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}{%- elif ns_state.effort == 'low' %}{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}{%- endif %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set head = namespace(count=0, seen_non_system=false) %}{%- for message in messages %}{%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}{%- if _is_sys and not head.seen_non_system %}{%- set head.count = head.count + 1 %}{%- else %}{%- set head.seen_non_system = true %}{%- endif %}{%- endfor %}{%- set sys_state = namespace(content='') %}{%- for message in messages[:head.count] %}{%- set _part = render_content(message.content, false, true) | trim %}{%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- if _part %}{%- if sys_state.content %}{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}{%- else %}{%- set sys_state.content = _part %}{%- endif %}{%- endif %}{%- endfor %}{%- set _sc = sys_state.content %}{%- set _msgs = messages[head.count:] %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set reasoning_content = message.thinking %}{%- else %}{%- set reasoning_content = message.thinking | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if not loop.first or content | trim %}{{- '\n\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string and tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is string %}{%- set _av = args_value %}{%- else %}{%- set _av = args_value | tojson %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:120] %}{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}{%- if not _is_code_or_grep and content | length < 600 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or _content_head.startswith('exception:') or _content_head.startswith('failed to ') or 'exit code: ' in _content_head or 'process exited with code' in _content_head) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
|
|
|
|
|
archive/v22.3_chat_template.jinja
DELETED
|
@@ -1,421 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.3" %}
|
| 2 |
-
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
-
{%- set image_count = namespace(value=0) %}
|
| 4 |
-
{%- set video_count = namespace(value=0) %}
|
| 5 |
-
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 6 |
-
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 7 |
-
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 8 |
-
{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
|
| 9 |
-
{%- set _preserve_thinking = preserve_reasoning %}
|
| 10 |
-
{%- elif preserve_thinking is defined and preserve_thinking is not none %}
|
| 11 |
-
{%- set _preserve_thinking = preserve_thinking %}
|
| 12 |
-
{%- else %}
|
| 13 |
-
{%- set _preserve_thinking = true %}
|
| 14 |
-
{%- endif %}
|
| 15 |
-
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 16 |
-
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 17 |
-
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 18 |
-
{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}
|
| 19 |
-
{%- set _initial_thinking = enable_thinking %}
|
| 20 |
-
{%- set _initial_effort = 'medium' %}
|
| 21 |
-
{%- if _effort_raw in ('none', 'off') %}
|
| 22 |
-
{%- set _initial_thinking = false %}
|
| 23 |
-
{%- set _initial_effort = 'medium' %}
|
| 24 |
-
{%- elif _effort_raw in ('minimal', 'low') %}
|
| 25 |
-
{%- set _initial_effort = 'low' %}
|
| 26 |
-
{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
|
| 27 |
-
{%- set _initial_effort = 'xhigh' %}
|
| 28 |
-
{%- else %}
|
| 29 |
-
{%- set _initial_effort = 'medium' %}
|
| 30 |
-
{%- endif %}
|
| 31 |
-
{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
|
| 32 |
-
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 33 |
-
{%- set ns_state.thinking = false %}
|
| 34 |
-
{%- endif %}
|
| 35 |
-
{%- for msg in messages %}
|
| 36 |
-
{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
|
| 37 |
-
{%- if msg.content is string %}
|
| 38 |
-
{%- if '<|think_off|>' in msg.content %}
|
| 39 |
-
{%- set ns_state.thinking = false %}
|
| 40 |
-
{%- elif '<|think_on|>' in msg.content %}
|
| 41 |
-
{%- set ns_state.thinking = true %}
|
| 42 |
-
{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
|
| 43 |
-
{%- set ns_state.thinking = true %}
|
| 44 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 45 |
-
{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
|
| 46 |
-
{%- set ns_state.thinking = true %}
|
| 47 |
-
{%- set ns_state.effort = 'low' %}
|
| 48 |
-
{%- elif '<|think_medium|>' in msg.content %}
|
| 49 |
-
{%- set ns_state.thinking = true %}
|
| 50 |
-
{%- set ns_state.effort = 'medium' %}
|
| 51 |
-
{%- endif %}
|
| 52 |
-
{%- elif msg.content is iterable and msg.content is not mapping %}
|
| 53 |
-
{%- for item in msg.content %}
|
| 54 |
-
{%- if item is string %}
|
| 55 |
-
{%- set _item_text = item %}
|
| 56 |
-
{%- elif item is mapping and 'text' in item and item.text is string %}
|
| 57 |
-
{%- set _item_text = item.text %}
|
| 58 |
-
{%- else %}
|
| 59 |
-
{%- set _item_text = '' %}
|
| 60 |
-
{%- endif %}
|
| 61 |
-
{%- if _item_text %}
|
| 62 |
-
{%- if '<|think_off|>' in _item_text %}
|
| 63 |
-
{%- set ns_state.thinking = false %}
|
| 64 |
-
{%- elif '<|think_on|>' in _item_text %}
|
| 65 |
-
{%- set ns_state.thinking = true %}
|
| 66 |
-
{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
|
| 67 |
-
{%- set ns_state.thinking = true %}
|
| 68 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 69 |
-
{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
|
| 70 |
-
{%- set ns_state.thinking = true %}
|
| 71 |
-
{%- set ns_state.effort = 'low' %}
|
| 72 |
-
{%- elif '<|think_medium|>' in _item_text %}
|
| 73 |
-
{%- set ns_state.thinking = true %}
|
| 74 |
-
{%- set ns_state.effort = 'medium' %}
|
| 75 |
-
{%- endif %}
|
| 76 |
-
{%- endif %}
|
| 77 |
-
{%- endfor %}
|
| 78 |
-
{%- endif %}
|
| 79 |
-
{%- endif %}
|
| 80 |
-
{%- endfor %}
|
| 81 |
-
{%- set reasoning_instructions = '' %}
|
| 82 |
-
{%- if ns_state.thinking %}
|
| 83 |
-
{%- if ns_state.effort == 'xhigh' %}
|
| 84 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
|
| 85 |
-
{%- elif ns_state.effort == 'low' %}
|
| 86 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
|
| 87 |
-
{%- endif %}
|
| 88 |
-
{%- endif %}
|
| 89 |
-
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 90 |
-
{%- if content is string %}
|
| 91 |
-
{{- content }}
|
| 92 |
-
{%- elif content is iterable and content is not mapping %}
|
| 93 |
-
{%- for item in content %}
|
| 94 |
-
{%- if item is mapping %}
|
| 95 |
-
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 96 |
-
{%- if is_system_content %}
|
| 97 |
-
{{- raise_exception('System message cannot contain images.') }}
|
| 98 |
-
{%- endif %}
|
| 99 |
-
{%- if do_vision_count %}
|
| 100 |
-
{%- set image_count.value = image_count.value + 1 %}
|
| 101 |
-
{%- endif %}
|
| 102 |
-
{%- if add_vision_id %}
|
| 103 |
-
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 104 |
-
{%- endif %}
|
| 105 |
-
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 106 |
-
{%- elif item.type == 'video' or 'video' in item %}
|
| 107 |
-
{%- if is_system_content %}
|
| 108 |
-
{{- raise_exception('System message cannot contain videos.') }}
|
| 109 |
-
{%- endif %}
|
| 110 |
-
{%- if do_vision_count %}
|
| 111 |
-
{%- set video_count.value = video_count.value + 1 %}
|
| 112 |
-
{%- endif %}
|
| 113 |
-
{%- if add_vision_id %}
|
| 114 |
-
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 115 |
-
{%- endif %}
|
| 116 |
-
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 117 |
-
{%- elif 'text' in item %}
|
| 118 |
-
{{- item.text }}
|
| 119 |
-
{%- else %}
|
| 120 |
-
{{- raise_exception('Unexpected item type in content.') }}
|
| 121 |
-
{%- endif %}
|
| 122 |
-
{%- else %}
|
| 123 |
-
{{- item | string }}
|
| 124 |
-
{%- endif %}
|
| 125 |
-
{%- endfor %}
|
| 126 |
-
{%- elif content is none or content is undefined %}
|
| 127 |
-
{{- '' }}
|
| 128 |
-
{%- else %}
|
| 129 |
-
{{- raise_exception('Unexpected content type.') }}
|
| 130 |
-
{%- endif %}
|
| 131 |
-
{%- endmacro %}
|
| 132 |
-
{%- if not messages %}
|
| 133 |
-
{{- raise_exception('No messages provided.') }}
|
| 134 |
-
{%- endif %}
|
| 135 |
-
{%- set head = namespace(count=0, seen_non_system=false) %}
|
| 136 |
-
{%- for message in messages %}
|
| 137 |
-
{%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}
|
| 138 |
-
{%- if _is_sys and not head.seen_non_system %}
|
| 139 |
-
{%- set head.count = head.count + 1 %}
|
| 140 |
-
{%- else %}
|
| 141 |
-
{%- set head.seen_non_system = true %}
|
| 142 |
-
{%- endif %}
|
| 143 |
-
{%- endfor %}
|
| 144 |
-
{%- set sys_state = namespace(content='') %}
|
| 145 |
-
{%- for message in messages[:head.count] %}
|
| 146 |
-
{%- set _part = render_content(message.content, false, true) | trim %}
|
| 147 |
-
{%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 148 |
-
{%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 149 |
-
{%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 150 |
-
{%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 151 |
-
{%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
|
| 152 |
-
{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
|
| 153 |
-
{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
|
| 154 |
-
{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 155 |
-
{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 156 |
-
{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 157 |
-
{%- if _part %}
|
| 158 |
-
{%- if sys_state.content %}
|
| 159 |
-
{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
|
| 160 |
-
{%- else %}
|
| 161 |
-
{%- set sys_state.content = _part %}
|
| 162 |
-
{%- endif %}
|
| 163 |
-
{%- endif %}
|
| 164 |
-
{%- endfor %}
|
| 165 |
-
{%- set _sc = sys_state.content %}
|
| 166 |
-
{%- set _msgs = messages[head.count:] %}
|
| 167 |
-
{%- if _has_tools %}
|
| 168 |
-
{{- '<|im_start|>system\n' }}
|
| 169 |
-
{%- if reasoning_instructions %}
|
| 170 |
-
{{- reasoning_instructions + '\n\n' }}
|
| 171 |
-
{%- endif %}
|
| 172 |
-
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 173 |
-
{%- for tool in tools %}
|
| 174 |
-
{{- '\n' }}
|
| 175 |
-
{{- tool | tojson }}
|
| 176 |
-
{%- endfor %}
|
| 177 |
-
{{- '\n</tools>' }}
|
| 178 |
-
{%- if _tool_format == 'json' %}
|
| 179 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 180 |
-
{%- else %}
|
| 181 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 182 |
-
{%- endif %}
|
| 183 |
-
{%- if _sc %}
|
| 184 |
-
{{- '\n\n' + _sc }}
|
| 185 |
-
{%- endif %}
|
| 186 |
-
{{- '<|im_end|>\n' }}
|
| 187 |
-
{%- else %}
|
| 188 |
-
{%- if _sc %}
|
| 189 |
-
{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
|
| 190 |
-
{%- elif reasoning_instructions %}
|
| 191 |
-
{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
|
| 192 |
-
{%- endif %}
|
| 193 |
-
{%- endif %}
|
| 194 |
-
{%- set _last_idx = _msgs | length - 1 %}
|
| 195 |
-
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 196 |
-
{%- for message in _msgs[::-1] %}
|
| 197 |
-
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 198 |
-
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 199 |
-
{%- set _rc = render_content(message.content, false) | trim %}
|
| 200 |
-
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 201 |
-
{%- set ns.multi_step_tool = false %}
|
| 202 |
-
{%- set ns.last_query_index = index %}
|
| 203 |
-
{%- endif %}
|
| 204 |
-
{%- endif %}
|
| 205 |
-
{%- endfor %}
|
| 206 |
-
{%- if ns.multi_step_tool %}
|
| 207 |
-
{%- if _last_idx > 50 %}
|
| 208 |
-
{%- set ns.last_query_index = _last_idx %}
|
| 209 |
-
{%- else %}
|
| 210 |
-
{%- set ns.last_query_index = 0 %}
|
| 211 |
-
{%- endif %}
|
| 212 |
-
{%- endif %}
|
| 213 |
-
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 214 |
-
{%- for message in _msgs %}
|
| 215 |
-
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 216 |
-
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 217 |
-
{%- if is_system or message.role == 'user' %}
|
| 218 |
-
{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
|
| 219 |
-
{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
|
| 220 |
-
{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
|
| 221 |
-
{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
|
| 222 |
-
{%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
|
| 223 |
-
{%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
|
| 224 |
-
{%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}
|
| 225 |
-
{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 226 |
-
{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 227 |
-
{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 228 |
-
{%- endif %}
|
| 229 |
-
{%- if is_system %}
|
| 230 |
-
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 231 |
-
{%- elif message.role == 'user' %}
|
| 232 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 233 |
-
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 234 |
-
{%- elif message.role == 'assistant' %}
|
| 235 |
-
{%- set reasoning_content = '' %}
|
| 236 |
-
{%- set _explicit_reasoning = '' %}
|
| 237 |
-
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 238 |
-
{%- if message.reasoning_content is string %}
|
| 239 |
-
{%- set _explicit_reasoning = message.reasoning_content %}
|
| 240 |
-
{%- else %}
|
| 241 |
-
{%- set _explicit_reasoning = message.reasoning_content | string %}
|
| 242 |
-
{%- endif %}
|
| 243 |
-
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 244 |
-
{%- if message.thinking is string %}
|
| 245 |
-
{%- set _explicit_reasoning = message.thinking %}
|
| 246 |
-
{%- else %}
|
| 247 |
-
{%- set _explicit_reasoning = message.thinking | string %}
|
| 248 |
-
{%- endif %}
|
| 249 |
-
{%- endif %}
|
| 250 |
-
{%- if _explicit_reasoning %}
|
| 251 |
-
{%- set _lead_end = '' %}
|
| 252 |
-
{%- if content.startswith('<think>') and '</think>' in content %}
|
| 253 |
-
{%- set _lead_end = '</think>' %}
|
| 254 |
-
{%- elif content.startswith('<thinking>') and '</thinking>' in content %}
|
| 255 |
-
{%- set _lead_end = '</thinking>' %}
|
| 256 |
-
{%- elif content.startswith('</think>') %}
|
| 257 |
-
{%- set _lead_end = '</think>' %}
|
| 258 |
-
{%- elif content.startswith('</thinking>') %}
|
| 259 |
-
{%- set _lead_end = '</thinking>' %}
|
| 260 |
-
{%- endif %}
|
| 261 |
-
{%- if _lead_end %}
|
| 262 |
-
{%- set content = content.split(_lead_end)[-1].lstrip('\n') %}
|
| 263 |
-
{%- endif %}
|
| 264 |
-
{%- set reasoning_content = _explicit_reasoning %}
|
| 265 |
-
{%- else %}
|
| 266 |
-
{%- set _think_end = '' %}
|
| 267 |
-
{%- if content.startswith('</think>') %}
|
| 268 |
-
{%- set _think_end = '</think>' %}
|
| 269 |
-
{%- elif content.startswith('</thinking>') %}
|
| 270 |
-
{%- set _think_end = '</thinking>' %}
|
| 271 |
-
{%- elif '\n</think>' in content %}
|
| 272 |
-
{%- set _think_end = '\n</think>' %}
|
| 273 |
-
{%- elif '\n</thinking>' in content %}
|
| 274 |
-
{%- set _think_end = '\n</thinking>' %}
|
| 275 |
-
{%- elif '\n</ think>' in content %}
|
| 276 |
-
{%- set _think_end = '\n</ think>' %}
|
| 277 |
-
{%- elif '\n</think >' in content %}
|
| 278 |
-
{%- set _think_end = '\n</think >' %}
|
| 279 |
-
{%- elif content.startswith('<think>') and '</think>' in content %}
|
| 280 |
-
{%- set _think_end = '</think>' %}
|
| 281 |
-
{%- elif content.startswith('<thinking>') and '</thinking>' in content %}
|
| 282 |
-
{%- set _think_end = '</thinking>' %}
|
| 283 |
-
{%- endif %}
|
| 284 |
-
{%- if _think_end %}
|
| 285 |
-
{%- if 'thinking' in _think_end %}
|
| 286 |
-
{%- set _think_start = '<thinking>' %}
|
| 287 |
-
{%- else %}
|
| 288 |
-
{%- set _think_start = '<think>' %}
|
| 289 |
-
{%- endif %}
|
| 290 |
-
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 291 |
-
{%- if _think_start in reasoning_content %}
|
| 292 |
-
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 293 |
-
{%- endif %}
|
| 294 |
-
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 295 |
-
{%- endif %}
|
| 296 |
-
{%- endif %}
|
| 297 |
-
{%- set reasoning_content = reasoning_content | trim %}
|
| 298 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}
|
| 299 |
-
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 300 |
-
{%- else %}
|
| 301 |
-
{{- '<|im_start|>assistant\n' + content }}
|
| 302 |
-
{%- endif %}
|
| 303 |
-
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 304 |
-
{%- for tool_call in message.tool_calls %}
|
| 305 |
-
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 306 |
-
{%- set tc = tool_call.function %}
|
| 307 |
-
{%- else %}
|
| 308 |
-
{%- set tc = tool_call %}
|
| 309 |
-
{%- endif %}
|
| 310 |
-
{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
|
| 311 |
-
{%- if _tool_format == 'json' %}
|
| 312 |
-
{%- if not loop.first or content | trim %}
|
| 313 |
-
{{- '\n\n' }}
|
| 314 |
-
{%- endif %}
|
| 315 |
-
{%- set _args = '{}' %}
|
| 316 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 317 |
-
{%- if tc.arguments is mapping %}
|
| 318 |
-
{%- set _args = tc.arguments | tojson %}
|
| 319 |
-
{%- elif tc.arguments is string %}
|
| 320 |
-
{%- if tc.arguments %}
|
| 321 |
-
{%- set _args = tc.arguments %}
|
| 322 |
-
{%- endif %}
|
| 323 |
-
{%- else %}
|
| 324 |
-
{%- set _args = tc.arguments | tojson %}
|
| 325 |
-
{%- endif %}
|
| 326 |
-
{%- endif %}
|
| 327 |
-
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
| 328 |
-
{%- else %}
|
| 329 |
-
{%- if loop.first %}
|
| 330 |
-
{%- if content | trim %}
|
| 331 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 332 |
-
{%- else %}
|
| 333 |
-
{{- '<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 334 |
-
{%- endif %}
|
| 335 |
-
{%- else %}
|
| 336 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 337 |
-
{%- endif %}
|
| 338 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 339 |
-
{%- if tc.arguments is mapping %}
|
| 340 |
-
{%- for args_name, args_value in tc.arguments.items() %}
|
| 341 |
-
{{- '<parameter=' + args_name + '>\n' }}
|
| 342 |
-
{%- if args_value is string %}
|
| 343 |
-
{%- set _av = args_value %}
|
| 344 |
-
{%- else %}
|
| 345 |
-
{%- set _av = args_value | tojson %}
|
| 346 |
-
{%- endif %}
|
| 347 |
-
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 348 |
-
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 349 |
-
{%- else %}
|
| 350 |
-
{{- _av }}
|
| 351 |
-
{%- endif %}
|
| 352 |
-
{{- '\n</parameter>\n' }}
|
| 353 |
-
{%- endfor %}
|
| 354 |
-
{%- else %}
|
| 355 |
-
{%- if tc.arguments is string %}
|
| 356 |
-
{%- set _raw_args = tc.arguments %}
|
| 357 |
-
{%- else %}
|
| 358 |
-
{%- set _raw_args = tc.arguments | tojson %}
|
| 359 |
-
{%- endif %}
|
| 360 |
-
{%- if _raw_args %}
|
| 361 |
-
{%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}
|
| 362 |
-
{{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}
|
| 363 |
-
{%- else %}
|
| 364 |
-
{{- _raw_args }}
|
| 365 |
-
{%- endif %}
|
| 366 |
-
{%- endif %}
|
| 367 |
-
{%- endif %}
|
| 368 |
-
{%- endif %}
|
| 369 |
-
{{- '</function>\n</tool_call>' }}
|
| 370 |
-
{%- endif %}
|
| 371 |
-
{%- endfor %}
|
| 372 |
-
{%- endif %}
|
| 373 |
-
{{- '<|im_end|>\n' }}
|
| 374 |
-
{%- elif message.role == 'tool' %}
|
| 375 |
-
{%- set _content_lower = content | lower %}
|
| 376 |
-
{%- set _content_head = _content_lower[:120] %}
|
| 377 |
-
{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
|
| 378 |
-
{%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}
|
| 379 |
-
{%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}
|
| 380 |
-
{%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}
|
| 381 |
-
{%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}
|
| 382 |
-
{%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}
|
| 383 |
-
{%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}
|
| 384 |
-
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 385 |
-
{%- else %}
|
| 386 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 387 |
-
{%- endif %}
|
| 388 |
-
{%- if ns2.prev_role != 'tool' %}
|
| 389 |
-
{{- '<|im_start|>user' }}
|
| 390 |
-
{%- endif %}
|
| 391 |
-
{%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 392 |
-
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 393 |
-
{%- endif %}
|
| 394 |
-
{{- '\n<tool_response>\n' + content }}
|
| 395 |
-
{%- if ns2.consecutive_failures >= 2 %}
|
| 396 |
-
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 397 |
-
{%- elif ns2.consecutive_failures == 1 %}
|
| 398 |
-
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 399 |
-
{%- endif %}
|
| 400 |
-
{{- '\n</tool_response>' }}
|
| 401 |
-
{%- if loop.last %}
|
| 402 |
-
{{- '<|im_end|>\n' }}
|
| 403 |
-
{%- else %}
|
| 404 |
-
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 405 |
-
{%- if _next_role != 'tool' %}
|
| 406 |
-
{{- '<|im_end|>\n' }}
|
| 407 |
-
{%- endif %}
|
| 408 |
-
{%- endif %}
|
| 409 |
-
{%- else %}
|
| 410 |
-
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 411 |
-
{%- endif %}
|
| 412 |
-
{%- set ns2.prev_role = message.role %}
|
| 413 |
-
{%- endfor %}
|
| 414 |
-
{%- if add_generation_prompt %}
|
| 415 |
-
{{- '<|im_start|>assistant\n' }}
|
| 416 |
-
{%- if not ns_state.thinking %}
|
| 417 |
-
{{- '<think>\n\n</think>\n\n' }}
|
| 418 |
-
{%- else %}
|
| 419 |
-
{{- '<think>\n' }}
|
| 420 |
-
{%- endif %}
|
| 421 |
-
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/v22.3_chat_template_oneline.txt
DELETED
|
@@ -1 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.3" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- if preserve_reasoning is defined and preserve_reasoning is not none %}{%- set _preserve_thinking = preserve_reasoning %}{%- elif preserve_thinking is defined and preserve_thinking is not none %}{%- set _preserve_thinking = preserve_thinking %}{%- else %}{%- set _preserve_thinking = true %}{%- endif %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else 'medium' %}{%- set _initial_thinking = enable_thinking %}{%- set _initial_effort = 'medium' %}{%- if _effort_raw in ('none', 'off') %}{%- set _initial_thinking = false %}{%- set _initial_effort = 'medium' %}{%- elif _effort_raw in ('minimal', 'low') %}{%- set _initial_effort = 'low' %}{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}{%- set _initial_effort = 'xhigh' %}{%- else %}{%- set _initial_effort = 'medium' %}{%- endif %}{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- for msg in messages %}{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}{%- if msg.content is string %}{%- if '<|think_off|>' in msg.content %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in msg.content %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- elif msg.content is iterable and msg.content is not mapping %}{%- for item in msg.content %}{%- if item is string %}{%- set _item_text = item %}{%- elif item is mapping and 'text' in item and item.text is string %}{%- set _item_text = item.text %}{%- else %}{%- set _item_text = '' %}{%- endif %}{%- if _item_text %}{%- if '<|think_off|>' in _item_text %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in _item_text %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- endif %}{%- endfor %}{%- endif %}{%- endif %}{%- endfor %}{%- set reasoning_instructions = '' %}{%- if ns_state.thinking %}{%- if ns_state.effort == 'xhigh' %}{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}{%- elif ns_state.effort == 'low' %}{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}{%- endif %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set head = namespace(count=0, seen_non_system=false) %}{%- for message in messages %}{%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}{%- if _is_sys and not head.seen_non_system %}{%- set head.count = head.count + 1 %}{%- else %}{%- set head.seen_non_system = true %}{%- endif %}{%- endfor %}{%- set sys_state = namespace(content='') %}{%- for message in messages[:head.count] %}{%- set _part = render_content(message.content, false, true) | trim %}{%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- if _part %}{%- if sys_state.content %}{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}{%- else %}{%- set sys_state.content = _part %}{%- endif %}{%- endif %}{%- endfor %}{%- set _sc = sys_state.content %}{%- set _msgs = messages[head.count:] %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- set _explicit_reasoning = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set _explicit_reasoning = message.reasoning_content %}{%- else %}{%- set _explicit_reasoning = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set _explicit_reasoning = message.thinking %}{%- else %}{%- set _explicit_reasoning = message.thinking | string %}{%- endif %}{%- endif %}{%- if _explicit_reasoning %}{%- set _lead_end = '' %}{%- if content.startswith('<think>') and '</think>' in content %}{%- set _lead_end = '</think>' %}{%- elif content.startswith('<thinking>') and '</thinking>' in content %}{%- set _lead_end = '</thinking>' %}{%- elif content.startswith('</think>') %}{%- set _lead_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _lead_end = '</thinking>' %}{%- endif %}{%- if _lead_end %}{%- set content = content.split(_lead_end)[-1].lstrip('\n') %}{%- endif %}{%- set reasoning_content = _explicit_reasoning %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- elif content.startswith('<think>') and '</think>' in content %}{%- set _think_end = '</think>' %}{%- elif content.startswith('<thinking>') and '</thinking>' in content %}{%- set _think_end = '</thinking>' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if not loop.first or content | trim %}{{- '\n\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string %}{%- if tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- else %}{%- set _args = tc.arguments | tojson %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is string %}{%- set _av = args_value %}{%- else %}{%- set _av = args_value | tojson %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- else %}{%- if tc.arguments is string %}{%- set _raw_args = tc.arguments %}{%- else %}{%- set _raw_args = tc.arguments | tojson %}{%- endif %}{%- if _raw_args %}{%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}{{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}{%- else %}{{- _raw_args }}{%- endif %}{%- endif %}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:120] %}{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}{%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}{%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}{%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}{%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}{%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}{%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
|
|
|
|
|
archive/v22_chat_template.jinja
DELETED
|
@@ -1,331 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22" %}
|
| 2 |
-
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
-
{%- set image_count = namespace(value=0) %}
|
| 4 |
-
{%- set video_count = namespace(value=0) %}
|
| 5 |
-
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 6 |
-
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 7 |
-
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 8 |
-
{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
|
| 9 |
-
{%- set _preserve_thinking = preserve_reasoning %}
|
| 10 |
-
{%- elif preserve_thinking is defined and preserve_thinking is not none %}
|
| 11 |
-
{%- set _preserve_thinking = preserve_thinking %}
|
| 12 |
-
{%- else %}
|
| 13 |
-
{%- set _preserve_thinking = true %}
|
| 14 |
-
{%- endif %}
|
| 15 |
-
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 16 |
-
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 17 |
-
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 18 |
-
{%- set ns_state = namespace(thinking=enable_thinking) %}
|
| 19 |
-
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 20 |
-
{%- set ns_state.thinking = false %}
|
| 21 |
-
{%- endif %}
|
| 22 |
-
{%- for msg in messages %}
|
| 23 |
-
{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
|
| 24 |
-
{%- if msg.content is string %}
|
| 25 |
-
{%- if '<|think_off|>' in msg.content %}
|
| 26 |
-
{%- set ns_state.thinking = false %}
|
| 27 |
-
{%- elif '<|think_on|>' in msg.content %}
|
| 28 |
-
{%- set ns_state.thinking = true %}
|
| 29 |
-
{%- endif %}
|
| 30 |
-
{%- elif msg.content is iterable and msg.content is not mapping %}
|
| 31 |
-
{%- for item in msg.content %}
|
| 32 |
-
{%- if item is mapping and 'text' in item and item.text is string %}
|
| 33 |
-
{%- if '<|think_off|>' in item.text %}
|
| 34 |
-
{%- set ns_state.thinking = false %}
|
| 35 |
-
{%- elif '<|think_on|>' in item.text %}
|
| 36 |
-
{%- set ns_state.thinking = true %}
|
| 37 |
-
{%- endif %}
|
| 38 |
-
{%- endif %}
|
| 39 |
-
{%- endfor %}
|
| 40 |
-
{%- endif %}
|
| 41 |
-
{%- endif %}
|
| 42 |
-
{%- endfor %}
|
| 43 |
-
{%- set _effort_raw = reasoning_effort if reasoning_effort is defined else 'xhigh' %}
|
| 44 |
-
{%- if _effort_raw == 'high' or _effort_raw == 'xhigh' %}
|
| 45 |
-
{%- set _reasoning_effort = 'xhigh' %}
|
| 46 |
-
{%- elif _effort_raw == 'low' %}
|
| 47 |
-
{%- set _reasoning_effort = 'low' %}
|
| 48 |
-
{%- elif _effort_raw == 'medium' %}
|
| 49 |
-
{%- set _reasoning_effort = 'medium' %}
|
| 50 |
-
{%- else %}
|
| 51 |
-
{%- set _reasoning_effort = 'xhigh' %}
|
| 52 |
-
{%- endif %}
|
| 53 |
-
{%- set reasoning_instructions = '' %}
|
| 54 |
-
{%- if ns_state.thinking %}
|
| 55 |
-
{%- if _reasoning_effort == 'xhigh' %}
|
| 56 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
|
| 57 |
-
{%- elif _reasoning_effort == 'low' %}
|
| 58 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
|
| 59 |
-
{%- endif %}
|
| 60 |
-
{%- endif %}
|
| 61 |
-
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 62 |
-
{%- if content is string %}
|
| 63 |
-
{{- content }}
|
| 64 |
-
{%- elif content is iterable and content is not mapping %}
|
| 65 |
-
{%- for item in content %}
|
| 66 |
-
{%- if item is mapping %}
|
| 67 |
-
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 68 |
-
{%- if is_system_content %}
|
| 69 |
-
{{- raise_exception('System message cannot contain images.') }}
|
| 70 |
-
{%- endif %}
|
| 71 |
-
{%- if do_vision_count %}
|
| 72 |
-
{%- set image_count.value = image_count.value + 1 %}
|
| 73 |
-
{%- endif %}
|
| 74 |
-
{%- if add_vision_id %}
|
| 75 |
-
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 76 |
-
{%- endif %}
|
| 77 |
-
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 78 |
-
{%- elif item.type == 'video' or 'video' in item %}
|
| 79 |
-
{%- if is_system_content %}
|
| 80 |
-
{{- raise_exception('System message cannot contain videos.') }}
|
| 81 |
-
{%- endif %}
|
| 82 |
-
{%- if do_vision_count %}
|
| 83 |
-
{%- set video_count.value = video_count.value + 1 %}
|
| 84 |
-
{%- endif %}
|
| 85 |
-
{%- if add_vision_id %}
|
| 86 |
-
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 87 |
-
{%- endif %}
|
| 88 |
-
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 89 |
-
{%- elif 'text' in item %}
|
| 90 |
-
{{- item.text }}
|
| 91 |
-
{%- else %}
|
| 92 |
-
{{- raise_exception('Unexpected item type in content.') }}
|
| 93 |
-
{%- endif %}
|
| 94 |
-
{%- else %}
|
| 95 |
-
{{- item | string }}
|
| 96 |
-
{%- endif %}
|
| 97 |
-
{%- endfor %}
|
| 98 |
-
{%- elif content is none or content is undefined %}
|
| 99 |
-
{{- '' }}
|
| 100 |
-
{%- else %}
|
| 101 |
-
{{- raise_exception('Unexpected content type.') }}
|
| 102 |
-
{%- endif %}
|
| 103 |
-
{%- endmacro %}
|
| 104 |
-
{%- if not messages %}
|
| 105 |
-
{{- raise_exception('No messages provided.') }}
|
| 106 |
-
{%- endif %}
|
| 107 |
-
{%- set _first_role = messages[0].role %}
|
| 108 |
-
{%- if _first_role == 'system' or _first_role == 'developer' %}
|
| 109 |
-
{%- set _sys_msg = messages[0] %}
|
| 110 |
-
{%- set _msgs = messages[1:] %}
|
| 111 |
-
{%- else %}
|
| 112 |
-
{%- set _sys_msg = none %}
|
| 113 |
-
{%- set _msgs = messages %}
|
| 114 |
-
{%- endif %}
|
| 115 |
-
{%- set _sc = '' %}
|
| 116 |
-
{%- if _sys_msg is not none %}
|
| 117 |
-
{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}
|
| 118 |
-
{%- if '<|think_off|>' in _sc %}
|
| 119 |
-
{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}
|
| 120 |
-
{%- elif '<|think_on|>' in _sc %}
|
| 121 |
-
{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}
|
| 122 |
-
{%- endif %}
|
| 123 |
-
{%- endif %}
|
| 124 |
-
{%- if _has_tools %}
|
| 125 |
-
{{- '<|im_start|>system\n' }}
|
| 126 |
-
{%- if reasoning_instructions %}
|
| 127 |
-
{{- reasoning_instructions + '\n\n' }}
|
| 128 |
-
{%- endif %}
|
| 129 |
-
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 130 |
-
{%- for tool in tools %}
|
| 131 |
-
{{- '\n' }}
|
| 132 |
-
{{- tool | tojson }}
|
| 133 |
-
{%- endfor %}
|
| 134 |
-
{{- '\n</tools>' }}
|
| 135 |
-
{%- if _tool_format == 'json' %}
|
| 136 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 137 |
-
{%- else %}
|
| 138 |
-
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
|
| 139 |
-
{%- endif %}
|
| 140 |
-
{%- if _sc %}
|
| 141 |
-
{{- '\n\n' + _sc }}
|
| 142 |
-
{%- endif %}
|
| 143 |
-
{{- '<|im_end|>\n' }}
|
| 144 |
-
{%- else %}
|
| 145 |
-
{%- if _sc %}
|
| 146 |
-
{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
|
| 147 |
-
{%- elif reasoning_instructions %}
|
| 148 |
-
{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
|
| 149 |
-
{%- endif %}
|
| 150 |
-
{%- endif %}
|
| 151 |
-
{%- set _last_idx = _msgs | length - 1 %}
|
| 152 |
-
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 153 |
-
{%- for message in _msgs[::-1] %}
|
| 154 |
-
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 155 |
-
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 156 |
-
{%- set _rc = render_content(message.content, false) | trim %}
|
| 157 |
-
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 158 |
-
{%- set ns.multi_step_tool = false %}
|
| 159 |
-
{%- set ns.last_query_index = index %}
|
| 160 |
-
{%- endif %}
|
| 161 |
-
{%- endif %}
|
| 162 |
-
{%- endfor %}
|
| 163 |
-
{%- if ns.multi_step_tool %}
|
| 164 |
-
{%- if _last_idx > 50 %}
|
| 165 |
-
{%- set ns.last_query_index = _last_idx %}
|
| 166 |
-
{%- else %}
|
| 167 |
-
{%- set ns.last_query_index = 0 %}
|
| 168 |
-
{%- endif %}
|
| 169 |
-
{%- endif %}
|
| 170 |
-
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 171 |
-
{%- for message in _msgs %}
|
| 172 |
-
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 173 |
-
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 174 |
-
{%- if is_system or message.role == 'user' %}
|
| 175 |
-
{%- if '<|think_off|>' in content %}
|
| 176 |
-
{%- set content = content.split('<|think_off|>') | join('') | trim %}
|
| 177 |
-
{%- elif '<|think_on|>' in content %}
|
| 178 |
-
{%- set content = content.split('<|think_on|>') | join('') | trim %}
|
| 179 |
-
{%- endif %}
|
| 180 |
-
{%- endif %}
|
| 181 |
-
{%- if is_system %}
|
| 182 |
-
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 183 |
-
{%- elif message.role == 'user' %}
|
| 184 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 185 |
-
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 186 |
-
{%- elif message.role == 'assistant' %}
|
| 187 |
-
{%- set reasoning_content = '' %}
|
| 188 |
-
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 189 |
-
{%- if message.reasoning_content is string %}
|
| 190 |
-
{%- set reasoning_content = message.reasoning_content %}
|
| 191 |
-
{%- else %}
|
| 192 |
-
{%- set reasoning_content = message.reasoning_content | string %}
|
| 193 |
-
{%- endif %}
|
| 194 |
-
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 195 |
-
{%- if message.thinking is string %}
|
| 196 |
-
{%- set reasoning_content = message.thinking %}
|
| 197 |
-
{%- else %}
|
| 198 |
-
{%- set reasoning_content = message.thinking | string %}
|
| 199 |
-
{%- endif %}
|
| 200 |
-
{%- else %}
|
| 201 |
-
{%- set _think_end = '' %}
|
| 202 |
-
{%- if content.startswith('</think>') %}
|
| 203 |
-
{%- set _think_end = '</think>' %}
|
| 204 |
-
{%- elif content.startswith('</thinking>') %}
|
| 205 |
-
{%- set _think_end = '</thinking>' %}
|
| 206 |
-
{%- elif '\n</think>' in content %}
|
| 207 |
-
{%- set _think_end = '\n</think>' %}
|
| 208 |
-
{%- elif '\n</thinking>' in content %}
|
| 209 |
-
{%- set _think_end = '\n</thinking>' %}
|
| 210 |
-
{%- elif '\n</ think>' in content %}
|
| 211 |
-
{%- set _think_end = '\n</ think>' %}
|
| 212 |
-
{%- elif '\n</think >' in content %}
|
| 213 |
-
{%- set _think_end = '\n</think >' %}
|
| 214 |
-
{%- endif %}
|
| 215 |
-
{%- if _think_end %}
|
| 216 |
-
{%- if 'thinking' in _think_end %}
|
| 217 |
-
{%- set _think_start = '<thinking>' %}
|
| 218 |
-
{%- else %}
|
| 219 |
-
{%- set _think_start = '<think>' %}
|
| 220 |
-
{%- endif %}
|
| 221 |
-
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 222 |
-
{%- if _think_start in reasoning_content %}
|
| 223 |
-
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 224 |
-
{%- endif %}
|
| 225 |
-
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 226 |
-
{%- endif %}
|
| 227 |
-
{%- endif %}
|
| 228 |
-
{%- set reasoning_content = reasoning_content | trim %}
|
| 229 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 230 |
-
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 231 |
-
{%- else %}
|
| 232 |
-
{{- '<|im_start|>assistant\n' + content }}
|
| 233 |
-
{%- endif %}
|
| 234 |
-
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 235 |
-
{%- for tool_call in message.tool_calls %}
|
| 236 |
-
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 237 |
-
{%- set tc = tool_call.function %}
|
| 238 |
-
{%- else %}
|
| 239 |
-
{%- set tc = tool_call %}
|
| 240 |
-
{%- endif %}
|
| 241 |
-
{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
|
| 242 |
-
{%- if _tool_format == 'json' %}
|
| 243 |
-
{%- if not loop.first or content | trim %}
|
| 244 |
-
{{- '\n\n' }}
|
| 245 |
-
{%- endif %}
|
| 246 |
-
{%- set _args = '{}' %}
|
| 247 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 248 |
-
{%- if tc.arguments is mapping %}
|
| 249 |
-
{%- set _args = tc.arguments | tojson %}
|
| 250 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 251 |
-
{%- set _args = tc.arguments %}
|
| 252 |
-
{%- endif %}
|
| 253 |
-
{%- endif %}
|
| 254 |
-
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
| 255 |
-
{%- else %}
|
| 256 |
-
{%- if loop.first %}
|
| 257 |
-
{%- if content | trim %}
|
| 258 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 259 |
-
{%- else %}
|
| 260 |
-
{{- '<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 261 |
-
{%- endif %}
|
| 262 |
-
{%- else %}
|
| 263 |
-
{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
|
| 264 |
-
{%- endif %}
|
| 265 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 266 |
-
{%- if tc.arguments is mapping %}
|
| 267 |
-
{%- for args_name, args_value in tc.arguments.items() %}
|
| 268 |
-
{{- '<parameter=' + args_name + '>\n' }}
|
| 269 |
-
{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
|
| 270 |
-
{%- set _av = args_value | tojson %}
|
| 271 |
-
{%- else %}
|
| 272 |
-
{%- set _av = args_value | string %}
|
| 273 |
-
{%- endif %}
|
| 274 |
-
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 275 |
-
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 276 |
-
{%- else %}
|
| 277 |
-
{{- _av }}
|
| 278 |
-
{%- endif %}
|
| 279 |
-
{{- '\n</parameter>\n' }}
|
| 280 |
-
{%- endfor %}
|
| 281 |
-
{%- elif tc.arguments is string and tc.arguments %}
|
| 282 |
-
{{- tc.arguments }}
|
| 283 |
-
{%- endif %}
|
| 284 |
-
{%- endif %}
|
| 285 |
-
{{- '</function>\n</tool_call>' }}
|
| 286 |
-
{%- endif %}
|
| 287 |
-
{%- endfor %}
|
| 288 |
-
{%- endif %}
|
| 289 |
-
{{- '<|im_end|>\n' }}
|
| 290 |
-
{%- elif message.role == 'tool' %}
|
| 291 |
-
{%- set _content_lower = content | lower %}
|
| 292 |
-
{%- set _content_head = _content_lower[:80] %}
|
| 293 |
-
{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}
|
| 294 |
-
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 295 |
-
{%- else %}
|
| 296 |
-
{%- set ns2.consecutive_failures = 0 %}
|
| 297 |
-
{%- endif %}
|
| 298 |
-
{%- if ns2.prev_role != 'tool' %}
|
| 299 |
-
{{- '<|im_start|>user' }}
|
| 300 |
-
{%- endif %}
|
| 301 |
-
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 302 |
-
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 303 |
-
{%- endif %}
|
| 304 |
-
{{- '\n<tool_response>\n' + content }}
|
| 305 |
-
{%- if ns2.consecutive_failures >= 2 %}
|
| 306 |
-
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 307 |
-
{%- elif ns2.consecutive_failures == 1 %}
|
| 308 |
-
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 309 |
-
{%- endif %}
|
| 310 |
-
{{- '\n</tool_response>' }}
|
| 311 |
-
{%- if loop.last %}
|
| 312 |
-
{{- '<|im_end|>\n' }}
|
| 313 |
-
{%- else %}
|
| 314 |
-
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 315 |
-
{%- if _next_role != 'tool' %}
|
| 316 |
-
{{- '<|im_end|>\n' }}
|
| 317 |
-
{%- endif %}
|
| 318 |
-
{%- endif %}
|
| 319 |
-
{%- else %}
|
| 320 |
-
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 321 |
-
{%- endif %}
|
| 322 |
-
{%- set ns2.prev_role = message.role %}
|
| 323 |
-
{%- endfor %}
|
| 324 |
-
{%- if add_generation_prompt %}
|
| 325 |
-
{{- '<|im_start|>assistant\n' }}
|
| 326 |
-
{%- if not ns_state.thinking %}
|
| 327 |
-
{{- '<think>\n\n</think>\n\n' }}
|
| 328 |
-
{%- else %}
|
| 329 |
-
{{- '<think>\n' }}
|
| 330 |
-
{%- endif %}
|
| 331 |
-
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
archive/v22_chat_template_oneline.txt
DELETED
|
@@ -1 +0,0 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- if preserve_reasoning is defined and preserve_reasoning is not none %}{%- set _preserve_thinking = preserve_reasoning %}{%- elif preserve_thinking is defined and preserve_thinking is not none %}{%- set _preserve_thinking = preserve_thinking %}{%- else %}{%- set _preserve_thinking = true %}{%- endif %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set ns_state = namespace(thinking=enable_thinking) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- for msg in messages %}{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}{%- if msg.content is string %}{%- if '<|think_off|>' in msg.content %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in msg.content %}{%- set ns_state.thinking = true %}{%- endif %}{%- elif msg.content is iterable and msg.content is not mapping %}{%- for item in msg.content %}{%- if item is mapping and 'text' in item and item.text is string %}{%- if '<|think_off|>' in item.text %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in item.text %}{%- set ns_state.thinking = true %}{%- endif %}{%- endif %}{%- endfor %}{%- endif %}{%- endif %}{%- endfor %}{%- set _effort_raw = reasoning_effort if reasoning_effort is defined else 'xhigh' %}{%- if _effort_raw == 'high' or _effort_raw == 'xhigh' %}{%- set _reasoning_effort = 'xhigh' %}{%- elif _effort_raw == 'low' %}{%- set _reasoning_effort = 'low' %}{%- elif _effort_raw == 'medium' %}{%- set _reasoning_effort = 'medium' %}{%- else %}{%- set _reasoning_effort = 'xhigh' %}{%- endif %}{%- set reasoning_instructions = '' %}{%- if ns_state.thinking %}{%- if _reasoning_effort == 'xhigh' %}{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}{%- elif _reasoning_effort == 'low' %}{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}{%- endif %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set _first_role = messages[0].role %}{%- if _first_role == 'system' or _first_role == 'developer' %}{%- set _sys_msg = messages[0] %}{%- set _msgs = messages[1:] %}{%- else %}{%- set _sys_msg = none %}{%- set _msgs = messages %}{%- endif %}{%- set _sc = '' %}{%- if _sys_msg is not none %}{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}{%- if '<|think_off|>' in _sc %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in _sc %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set reasoning_content = message.thinking %}{%- else %}{%- set reasoning_content = message.thinking | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if not loop.first or content | trim %}{{- '\n\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string and tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}{%- set _av = args_value | tojson %}{%- else %}{%- set _av = args_value | string %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:80] %}{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_head or 'error:' in _content_head or 'err!' in _content_head or 'fatal:' in _content_head or 'exception:' in _content_head or 'traceback' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'failed to' in _content_head) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
|
|
|
|
|
chat_template.jinja
CHANGED
|
@@ -1,91 +1,17 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.
|
| 2 |
-
{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
|
| 3 |
{%- set image_count = namespace(value=0) %}
|
| 4 |
{%- set video_count = namespace(value=0) %}
|
| 5 |
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 6 |
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 7 |
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 8 |
-
{%-
|
| 9 |
-
{%- set _preserve_thinking = preserve_reasoning %}
|
| 10 |
-
{%- elif preserve_thinking is defined and preserve_thinking is not none %}
|
| 11 |
-
{%- set _preserve_thinking = preserve_thinking %}
|
| 12 |
-
{%- else %}
|
| 13 |
-
{%- set _preserve_thinking = true %}
|
| 14 |
-
{%- endif %}
|
| 15 |
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 16 |
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 17 |
-
{%- set _default_reasoning_effort = 'medium' %}
|
| 18 |
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 19 |
-
{%- set
|
| 20 |
-
{%- set _initial_thinking = enable_thinking %}
|
| 21 |
-
{%- if _effort_raw in ('none', 'off') %}
|
| 22 |
-
{%- set _initial_thinking = false %}
|
| 23 |
-
{%- set _initial_effort = 'medium' %}
|
| 24 |
-
{%- elif _effort_raw in ('minimal', 'low') %}
|
| 25 |
-
{%- set _initial_effort = 'low' %}
|
| 26 |
-
{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
|
| 27 |
-
{%- set _initial_effort = 'xhigh' %}
|
| 28 |
-
{%- else %}
|
| 29 |
-
{%- set _initial_effort = 'medium' %}
|
| 30 |
-
{%- endif %}
|
| 31 |
-
{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
|
| 32 |
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 33 |
{%- set ns_state.thinking = false %}
|
| 34 |
{%- endif %}
|
| 35 |
-
{%- for msg in messages %}
|
| 36 |
-
{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
|
| 37 |
-
{%- if msg.content is string %}
|
| 38 |
-
{%- if '<|think_off|>' in msg.content %}
|
| 39 |
-
{%- set ns_state.thinking = false %}
|
| 40 |
-
{%- elif '<|think_on|>' in msg.content %}
|
| 41 |
-
{%- set ns_state.thinking = true %}
|
| 42 |
-
{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
|
| 43 |
-
{%- set ns_state.thinking = true %}
|
| 44 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 45 |
-
{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
|
| 46 |
-
{%- set ns_state.thinking = true %}
|
| 47 |
-
{%- set ns_state.effort = 'low' %}
|
| 48 |
-
{%- elif '<|think_medium|>' in msg.content %}
|
| 49 |
-
{%- set ns_state.thinking = true %}
|
| 50 |
-
{%- set ns_state.effort = 'medium' %}
|
| 51 |
-
{%- endif %}
|
| 52 |
-
{%- elif msg.content is iterable and msg.content is not mapping %}
|
| 53 |
-
{%- for item in msg.content %}
|
| 54 |
-
{%- if item is string %}
|
| 55 |
-
{%- set _item_text = item %}
|
| 56 |
-
{%- elif item is mapping and 'text' in item and item.text is string %}
|
| 57 |
-
{%- set _item_text = item.text %}
|
| 58 |
-
{%- else %}
|
| 59 |
-
{%- set _item_text = '' %}
|
| 60 |
-
{%- endif %}
|
| 61 |
-
{%- if _item_text %}
|
| 62 |
-
{%- if '<|think_off|>' in _item_text %}
|
| 63 |
-
{%- set ns_state.thinking = false %}
|
| 64 |
-
{%- elif '<|think_on|>' in _item_text %}
|
| 65 |
-
{%- set ns_state.thinking = true %}
|
| 66 |
-
{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
|
| 67 |
-
{%- set ns_state.thinking = true %}
|
| 68 |
-
{%- set ns_state.effort = 'xhigh' %}
|
| 69 |
-
{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
|
| 70 |
-
{%- set ns_state.thinking = true %}
|
| 71 |
-
{%- set ns_state.effort = 'low' %}
|
| 72 |
-
{%- elif '<|think_medium|>' in _item_text %}
|
| 73 |
-
{%- set ns_state.thinking = true %}
|
| 74 |
-
{%- set ns_state.effort = 'medium' %}
|
| 75 |
-
{%- endif %}
|
| 76 |
-
{%- endif %}
|
| 77 |
-
{%- endfor %}
|
| 78 |
-
{%- endif %}
|
| 79 |
-
{%- endif %}
|
| 80 |
-
{%- endfor %}
|
| 81 |
-
{%- set reasoning_instructions = '' %}
|
| 82 |
-
{%- if ns_state.thinking %}
|
| 83 |
-
{%- if ns_state.effort == 'xhigh' %}
|
| 84 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
|
| 85 |
-
{%- elif ns_state.effort == 'low' %}
|
| 86 |
-
{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
|
| 87 |
-
{%- endif %}
|
| 88 |
-
{%- endif %}
|
| 89 |
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 90 |
{%- if content is string %}
|
| 91 |
{{- content }}
|
|
@@ -132,63 +58,71 @@
|
|
| 132 |
{%- if not messages %}
|
| 133 |
{{- raise_exception('No messages provided.') }}
|
| 134 |
{%- endif %}
|
| 135 |
-
{%- set
|
| 136 |
-
{%-
|
| 137 |
-
{%- set
|
| 138 |
-
{%-
|
| 139 |
-
|
| 140 |
-
{%-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
{%-
|
| 144 |
-
{%-
|
| 145 |
-
{%-
|
| 146 |
-
{%-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
{%-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
|
| 153 |
-
{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
|
| 154 |
-
{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 155 |
-
{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 156 |
-
{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 157 |
-
{%- if _part %}
|
| 158 |
-
{%- if sys_state.content %}
|
| 159 |
-
{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
|
| 160 |
-
{%- else %}
|
| 161 |
-
{%- set sys_state.content = _part %}
|
| 162 |
-
{%- endif %}
|
| 163 |
{%- endif %}
|
| 164 |
-
{%-
|
| 165 |
-
{%- set _sc = sys_state.content %}
|
| 166 |
-
{%- set _msgs = messages[head.count:] %}
|
| 167 |
{%- if _has_tools %}
|
| 168 |
{{- '<|im_start|>system\n' }}
|
| 169 |
-
{%- if reasoning_instructions %}
|
| 170 |
-
{{- reasoning_instructions + '\n\n' }}
|
| 171 |
-
{%- endif %}
|
| 172 |
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 173 |
{%- for tool in tools %}
|
| 174 |
{{- '\n' }}
|
| 175 |
{{- tool | tojson }}
|
| 176 |
{%- endfor %}
|
| 177 |
{{- '\n</tools>' }}
|
| 178 |
-
{%-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 183 |
{%- if _sc %}
|
| 184 |
{{- '\n\n' + _sc }}
|
| 185 |
{%- endif %}
|
| 186 |
{{- '<|im_end|>\n' }}
|
| 187 |
{%- else %}
|
| 188 |
{%- if _sc %}
|
| 189 |
-
{{- '<|im_start|>system\n' +
|
| 190 |
-
{%- elif reasoning_instructions %}
|
| 191 |
-
{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
|
| 192 |
{%- endif %}
|
| 193 |
{%- endif %}
|
| 194 |
{%- set _last_idx = _msgs | length - 1 %}
|
|
@@ -215,16 +149,13 @@
|
|
| 215 |
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 216 |
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 217 |
{%- if is_system or message.role == 'user' %}
|
| 218 |
-
{%- if '<|think_off|>' in content %}
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
{%-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
{%-
|
| 225 |
-
{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
|
| 226 |
-
{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
|
| 227 |
-
{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
|
| 228 |
{%- endif %}
|
| 229 |
{%- if is_system %}
|
| 230 |
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
|
@@ -233,62 +164,25 @@
|
|
| 233 |
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 234 |
{%- elif message.role == 'assistant' %}
|
| 235 |
{%- set reasoning_content = '' %}
|
| 236 |
-
{%- set _explicit_reasoning = '' %}
|
| 237 |
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 238 |
{%- if message.reasoning_content is string %}
|
| 239 |
-
{%- set
|
| 240 |
{%- else %}
|
| 241 |
-
{%- set
|
| 242 |
-
{%- endif %}
|
| 243 |
-
{%- elif message.thinking is defined and message.thinking is not none %}
|
| 244 |
-
{%- if message.thinking is string %}
|
| 245 |
-
{%- set _explicit_reasoning = message.thinking %}
|
| 246 |
-
{%- else %}
|
| 247 |
-
{%- set _explicit_reasoning = message.thinking | string %}
|
| 248 |
-
{%- endif %}
|
| 249 |
-
{%- elif message.reasoning is defined and message.reasoning is not none %}
|
| 250 |
-
{%- if message.reasoning is string %}
|
| 251 |
-
{%- set _explicit_reasoning = message.reasoning %}
|
| 252 |
-
{%- else %}
|
| 253 |
-
{%- set _explicit_reasoning = message.reasoning | string %}
|
| 254 |
-
{%- endif %}
|
| 255 |
-
{%- endif %}
|
| 256 |
-
{%- if _explicit_reasoning %}
|
| 257 |
-
{%- set _lead_end = '' %}
|
| 258 |
-
{%- if content.startswith('<think>') and '</think>' in content %}
|
| 259 |
-
{%- set _lead_end = '</think>' %}
|
| 260 |
-
{%- elif content.startswith('<thinking>') and '</thinking>' in content %}
|
| 261 |
-
{%- set _lead_end = '</thinking>' %}
|
| 262 |
-
{%- elif content.startswith('</think>') %}
|
| 263 |
-
{%- set _lead_end = '</think>' %}
|
| 264 |
-
{%- elif content.startswith('</thinking>') %}
|
| 265 |
-
{%- set _lead_end = '</thinking>' %}
|
| 266 |
{%- endif %}
|
| 267 |
-
{%- if _lead_end %}
|
| 268 |
-
{%- set content = content.split(_lead_end)[-1].lstrip('\n') %}
|
| 269 |
-
{%- endif %}
|
| 270 |
-
{%- set reasoning_content = _explicit_reasoning %}
|
| 271 |
{%- else %}
|
| 272 |
{%- set _think_end = '' %}
|
| 273 |
-
{%- if
|
| 274 |
-
{%- set _think_end = '</think>' %}
|
| 275 |
-
{%- elif content.startswith('</thinking>') %}
|
| 276 |
-
{%- set _think_end = '</thinking>' %}
|
| 277 |
-
{%- elif '\n</think>' in content %}
|
| 278 |
-
{%- set _think_end = '\n</think>' %}
|
| 279 |
-
{%- elif '\n</thinking>' in content %}
|
| 280 |
-
{%- set _think_end = '\n</thinking>' %}
|
| 281 |
-
{%- elif '\n</ think>' in content %}
|
| 282 |
-
{%- set _think_end = '\n</ think>' %}
|
| 283 |
-
{%- elif '\n</think >' in content %}
|
| 284 |
-
{%- set _think_end = '\n</think >' %}
|
| 285 |
-
{%- elif content.startswith('<think>') and '</think>' in content %}
|
| 286 |
{%- set _think_end = '</think>' %}
|
| 287 |
-
{%- elif
|
| 288 |
{%- set _think_end = '</thinking>' %}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 289 |
{%- endif %}
|
| 290 |
{%- if _think_end %}
|
| 291 |
-
{%- if 'thinking'
|
| 292 |
{%- set _think_start = '<thinking>' %}
|
| 293 |
{%- else %}
|
| 294 |
{%- set _think_start = '<think>' %}
|
|
@@ -301,7 +195,7 @@
|
|
| 301 |
{%- endif %}
|
| 302 |
{%- endif %}
|
| 303 |
{%- set reasoning_content = reasoning_content | trim %}
|
| 304 |
-
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}
|
| 305 |
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 306 |
{%- else %}
|
| 307 |
{{- '<|im_start|>assistant\n' + content }}
|
|
@@ -313,69 +207,38 @@
|
|
| 313 |
{%- else %}
|
| 314 |
{%- set tc = tool_call %}
|
| 315 |
{%- endif %}
|
| 316 |
-
{%-
|
| 317 |
-
|
| 318 |
-
|
| 319 |
-
{%- if content | trim %}
|
| 320 |
-
{{- '\n\n' }}
|
| 321 |
-
{%- endif %}
|
| 322 |
{%- else %}
|
| 323 |
-
{{- '\n' }}
|
| 324 |
{%- endif %}
|
| 325 |
-
{%- set _args = '{}' %}
|
| 326 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 327 |
-
{%- if tc.arguments is mapping %}
|
| 328 |
-
{%- set _args = tc.arguments | tojson %}
|
| 329 |
-
{%- elif tc.arguments is string %}
|
| 330 |
-
{%- if tc.arguments %}
|
| 331 |
-
{%- set _args = tc.arguments %}
|
| 332 |
-
{%- endif %}
|
| 333 |
-
{%- else %}
|
| 334 |
-
{%- set _args = tc.arguments | tojson %}
|
| 335 |
-
{%- endif %}
|
| 336 |
-
{%- endif %}
|
| 337 |
-
{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
|
| 338 |
{%- else %}
|
| 339 |
-
{
|
| 340 |
-
|
| 341 |
-
|
| 342 |
-
|
| 343 |
-
|
| 344 |
-
|
| 345 |
-
|
| 346 |
-
|
| 347 |
-
{%- endif %}
|
| 348 |
-
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 349 |
-
{%- if tc.arguments is mapping %}
|
| 350 |
-
{%- for args_name, args_value in tc.arguments.items() %}
|
| 351 |
-
{{- '<parameter=' + args_name + '>\n' }}
|
| 352 |
-
{%- if args_value is string %}
|
| 353 |
-
{%- set _av = args_value %}
|
| 354 |
-
{%- else %}
|
| 355 |
-
{%- set _av = args_value | tojson %}
|
| 356 |
-
{%- endif %}
|
| 357 |
-
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 358 |
-
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 359 |
-
{%- else %}
|
| 360 |
-
{{- _av }}
|
| 361 |
-
{%- endif %}
|
| 362 |
-
{{- '\n</parameter>\n' }}
|
| 363 |
-
{%- endfor %}
|
| 364 |
-
{%- else %}
|
| 365 |
-
{%- if tc.arguments is string %}
|
| 366 |
-
{%- set _raw_args = tc.arguments %}
|
| 367 |
{%- else %}
|
| 368 |
-
{%- set
|
| 369 |
{%- endif %}
|
| 370 |
-
{%- if
|
| 371 |
-
{
|
| 372 |
-
|
| 373 |
-
{
|
| 374 |
-
{{- _raw_args }}
|
| 375 |
-
{%- endif %}
|
| 376 |
{%- endif %}
|
| 377 |
-
|
|
|
|
|
|
|
|
|
|
| 378 |
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 379 |
{{- '</function>\n</tool_call>' }}
|
| 380 |
{%- endif %}
|
| 381 |
{%- endfor %}
|
|
@@ -383,14 +246,7 @@
|
|
| 383 |
{{- '<|im_end|>\n' }}
|
| 384 |
{%- elif message.role == 'tool' %}
|
| 385 |
{%- set _content_lower = content | lower %}
|
| 386 |
-
{%-
|
| 387 |
-
{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
|
| 388 |
-
{%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}
|
| 389 |
-
{%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}
|
| 390 |
-
{%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}
|
| 391 |
-
{%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}
|
| 392 |
-
{%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}
|
| 393 |
-
{%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}
|
| 394 |
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 395 |
{%- else %}
|
| 396 |
{%- set ns2.consecutive_failures = 0 %}
|
|
@@ -398,8 +254,8 @@
|
|
| 398 |
{%- if ns2.prev_role != 'tool' %}
|
| 399 |
{{- '<|im_start|>user' }}
|
| 400 |
{%- endif %}
|
| 401 |
-
{%- if
|
| 402 |
-
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED
|
| 403 |
{%- endif %}
|
| 404 |
{{- '\n<tool_response>\n' + content }}
|
| 405 |
{%- if ns2.consecutive_failures >= 2 %}
|
|
@@ -424,7 +280,9 @@
|
|
| 424 |
{%- if add_generation_prompt %}
|
| 425 |
{{- '<|im_start|>assistant\n' }}
|
| 426 |
{%- if not ns_state.thinking %}
|
| 427 |
-
{{- '<think>\n
|
|
|
|
|
|
|
| 428 |
{%- else %}
|
| 429 |
{{- '<think>\n' }}
|
| 430 |
{%- endif %}
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.6-froggeric-v20" %}
|
|
|
|
| 2 |
{%- set image_count = namespace(value=0) %}
|
| 3 |
{%- set video_count = namespace(value=0) %}
|
| 4 |
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 5 |
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 6 |
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 7 |
+
{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else false %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 9 |
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
|
|
|
| 10 |
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 11 |
+
{%- set ns_state = namespace(thinking=enable_thinking) %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 13 |
{%- set ns_state.thinking = false %}
|
| 14 |
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 16 |
{%- if content is string %}
|
| 17 |
{{- content }}
|
|
|
|
| 58 |
{%- if not messages %}
|
| 59 |
{{- raise_exception('No messages provided.') }}
|
| 60 |
{%- endif %}
|
| 61 |
+
{%- set _first_role = messages[0].role %}
|
| 62 |
+
{%- if _first_role == 'system' or _first_role == 'developer' %}
|
| 63 |
+
{%- set _sys_msg = messages[0] %}
|
| 64 |
+
{%- set _msgs = messages[1:] %}
|
| 65 |
+
{%- else %}
|
| 66 |
+
{%- set _sys_msg = none %}
|
| 67 |
+
{%- set _msgs = messages %}
|
| 68 |
+
{%- endif %}
|
| 69 |
+
{%- set _sc = '' %}
|
| 70 |
+
{%- if _sys_msg is not none %}
|
| 71 |
+
{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}
|
| 72 |
+
{%- if '<|think_off|>' in _sc %}
|
| 73 |
+
{%- set ns_state.thinking = false %}
|
| 74 |
+
{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}
|
| 75 |
+
{%- elif '<|think_on|>' in _sc %}
|
| 76 |
+
{%- set ns_state.thinking = true %}
|
| 77 |
+
{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
{%- endif %}
|
| 79 |
+
{%- endif %}
|
|
|
|
|
|
|
| 80 |
{%- if _has_tools %}
|
| 81 |
{{- '<|im_start|>system\n' }}
|
|
|
|
|
|
|
|
|
|
| 82 |
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 83 |
{%- for tool in tools %}
|
| 84 |
{{- '\n' }}
|
| 85 |
{{- tool | tojson }}
|
| 86 |
{%- endfor %}
|
| 87 |
{{- '\n</tools>' }}
|
| 88 |
+
{%- set tool_instructions %}
|
| 89 |
+
If you choose to call a function ONLY reply in the following format with NO suffix:
|
| 90 |
+
|
| 91 |
+
<think>
|
| 92 |
+
Brief explanation of tool call
|
| 93 |
+
</think>
|
| 94 |
+
<tool_call>
|
| 95 |
+
<function=example_function_name>
|
| 96 |
+
<parameter=example_parameter_1>
|
| 97 |
+
value_1
|
| 98 |
+
</parameter>
|
| 99 |
+
<parameter=example_parameter_2>
|
| 100 |
+
This is the value for the second parameter
|
| 101 |
+
that can span
|
| 102 |
+
multiple lines
|
| 103 |
+
</parameter>
|
| 104 |
+
</function>
|
| 105 |
+
</tool_call>
|
| 106 |
+
|
| 107 |
+
<IMPORTANT>
|
| 108 |
+
Reminder:
|
| 109 |
+
- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.
|
| 110 |
+
- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.
|
| 111 |
+
- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.
|
| 112 |
+
- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after closing </think>. Do NOT output any conversational text before the tool call.
|
| 113 |
+
- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.
|
| 114 |
+
- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.
|
| 115 |
+
- If you have gathered all necessary data and do not need to call a tool, answer the question like normal and provide your final response to the user IMMEDIATELY after closing </think>.
|
| 116 |
+
</IMPORTANT>
|
| 117 |
+
{%- endset %}
|
| 118 |
+
{{- '\n\n' ~ tool_instructions | trim }}
|
| 119 |
{%- if _sc %}
|
| 120 |
{{- '\n\n' + _sc }}
|
| 121 |
{%- endif %}
|
| 122 |
{{- '<|im_end|>\n' }}
|
| 123 |
{%- else %}
|
| 124 |
{%- if _sc %}
|
| 125 |
+
{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}
|
|
|
|
|
|
|
| 126 |
{%- endif %}
|
| 127 |
{%- endif %}
|
| 128 |
{%- set _last_idx = _msgs | length - 1 %}
|
|
|
|
| 149 |
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 150 |
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 151 |
{%- if is_system or message.role == 'user' %}
|
| 152 |
+
{%- if '<|think_off|>' in content %}
|
| 153 |
+
{%- set ns_state.thinking = false %}
|
| 154 |
+
{%- set content = content.split('<|think_off|>') | join('') | trim %}
|
| 155 |
+
{%- elif '<|think_on|>' in content %}
|
| 156 |
+
{%- set ns_state.thinking = true %}
|
| 157 |
+
{%- set content = content.split('<|think_on|>') | join('') | trim %}
|
| 158 |
+
{%- endif %}
|
|
|
|
|
|
|
|
|
|
| 159 |
{%- endif %}
|
| 160 |
{%- if is_system %}
|
| 161 |
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
|
|
|
| 164 |
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 165 |
{%- elif message.role == 'assistant' %}
|
| 166 |
{%- set reasoning_content = '' %}
|
|
|
|
| 167 |
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 168 |
{%- if message.reasoning_content is string %}
|
| 169 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 170 |
{%- else %}
|
| 171 |
+
{%- set reasoning_content = message.reasoning_content | string %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 172 |
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 173 |
{%- else %}
|
| 174 |
{%- set _think_end = '' %}
|
| 175 |
+
{%- if '</think>' in content %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 176 |
{%- set _think_end = '</think>' %}
|
| 177 |
+
{%- elif '</thinking>' in content %}
|
| 178 |
{%- set _think_end = '</thinking>' %}
|
| 179 |
+
{%- elif '</ think>' in content %}
|
| 180 |
+
{%- set _think_end = '</ think>' %}
|
| 181 |
+
{%- elif '</think >' in content %}
|
| 182 |
+
{%- set _think_end = '</think >' %}
|
| 183 |
{%- endif %}
|
| 184 |
{%- if _think_end %}
|
| 185 |
+
{%- if _think_end == '</thinking>' %}
|
| 186 |
{%- set _think_start = '<thinking>' %}
|
| 187 |
{%- else %}
|
| 188 |
{%- set _think_start = '<think>' %}
|
|
|
|
| 195 |
{%- endif %}
|
| 196 |
{%- endif %}
|
| 197 |
{%- set reasoning_content = reasoning_content | trim %}
|
| 198 |
+
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 199 |
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 200 |
{%- else %}
|
| 201 |
{{- '<|im_start|>assistant\n' + content }}
|
|
|
|
| 207 |
{%- else %}
|
| 208 |
{%- set tc = tool_call %}
|
| 209 |
{%- endif %}
|
| 210 |
+
{%- if loop.first %}
|
| 211 |
+
{%- if content | trim %}
|
| 212 |
+
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
|
|
|
|
|
|
|
|
|
| 213 |
{%- else %}
|
| 214 |
+
{{- '<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 215 |
{%- endif %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 216 |
{%- else %}
|
| 217 |
+
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 218 |
+
{%- endif %}
|
| 219 |
+
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 220 |
+
{%- if tc.arguments is mapping %}
|
| 221 |
+
{%- for args_name, args_value in tc.arguments.items() %}
|
| 222 |
+
{{- '<parameter=' + args_name + '>\n' }}
|
| 223 |
+
{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
|
| 224 |
+
{%- set _av = args_value | tojson %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 225 |
{%- else %}
|
| 226 |
+
{%- set _av = args_value | string %}
|
| 227 |
{%- endif %}
|
| 228 |
+
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 229 |
+
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 230 |
+
{%- else %}
|
| 231 |
+
{{- _av }}
|
|
|
|
|
|
|
| 232 |
{%- endif %}
|
| 233 |
+
{{- '\n</parameter>\n' }}
|
| 234 |
+
{%- endfor %}
|
| 235 |
+
{%- elif tc.arguments is string and tc.arguments %}
|
| 236 |
+
{{- tc.arguments }}
|
| 237 |
{%- endif %}
|
| 238 |
+
{%- endif %}
|
| 239 |
+
{%- if loop.last %}
|
| 240 |
+
{{- '</function>\n</tool_call>\n' }}
|
| 241 |
+
{%- else %}
|
| 242 |
{{- '</function>\n</tool_call>' }}
|
| 243 |
{%- endif %}
|
| 244 |
{%- endfor %}
|
|
|
|
| 246 |
{{- '<|im_end|>\n' }}
|
| 247 |
{%- elif message.role == 'tool' %}
|
| 248 |
{%- set _content_lower = content | lower %}
|
| 249 |
+
{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_lower or 'error:' in _content_lower or 'exception:' in _content_lower or 'traceback' in _content_lower or 'command not found' in _content_lower or 'invalid syntax' in _content_lower or 'failed to' in _content_lower) %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 250 |
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 251 |
{%- else %}
|
| 252 |
{%- set ns2.consecutive_failures = 0 %}
|
|
|
|
| 254 |
{%- if ns2.prev_role != 'tool' %}
|
| 255 |
{{- '<|im_start|>user' }}
|
| 256 |
{%- endif %}
|
| 257 |
+
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 258 |
+
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 259 |
{%- endif %}
|
| 260 |
{{- '\n<tool_response>\n' + content }}
|
| 261 |
{%- if ns2.consecutive_failures >= 2 %}
|
|
|
|
| 280 |
{%- if add_generation_prompt %}
|
| 281 |
{{- '<|im_start|>assistant\n' }}
|
| 282 |
{%- if not ns_state.thinking %}
|
| 283 |
+
{{- '<think>\n</think>\n' }}
|
| 284 |
+
{%- elif ns2.consecutive_failures >= 2 %}
|
| 285 |
+
{{- '<think>\n</think>\n' }}
|
| 286 |
{%- else %}
|
| 287 |
{{- '<think>\n' }}
|
| 288 |
{%- endif %}
|
chat_template_oneline.txt
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{%- set template_version = "qwen3.8-froggeric-v22.4" %}{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- if preserve_reasoning is defined and preserve_reasoning is not none %}{%- set _preserve_thinking = preserve_reasoning %}{%- elif preserve_thinking is defined and preserve_thinking is not none %}{%- set _preserve_thinking = preserve_thinking %}{%- else %}{%- set _preserve_thinking = true %}{%- endif %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _default_reasoning_effort = 'medium' %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else _default_reasoning_effort %}{%- set _initial_thinking = enable_thinking %}{%- if _effort_raw in ('none', 'off') %}{%- set _initial_thinking = false %}{%- set _initial_effort = 'medium' %}{%- elif _effort_raw in ('minimal', 'low') %}{%- set _initial_effort = 'low' %}{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}{%- set _initial_effort = 'xhigh' %}{%- else %}{%- set _initial_effort = 'medium' %}{%- endif %}{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- for msg in messages %}{%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}{%- if msg.content is string %}{%- if '<|think_off|>' in msg.content %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in msg.content %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in msg.content %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- elif msg.content is iterable and msg.content is not mapping %}{%- for item in msg.content %}{%- if item is string %}{%- set _item_text = item %}{%- elif item is mapping and 'text' in item and item.text is string %}{%- set _item_text = item.text %}{%- else %}{%- set _item_text = '' %}{%- endif %}{%- if _item_text %}{%- if '<|think_off|>' in _item_text %}{%- set ns_state.thinking = false %}{%- elif '<|think_on|>' in _item_text %}{%- set ns_state.thinking = true %}{%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'xhigh' %}{%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'low' %}{%- elif '<|think_medium|>' in _item_text %}{%- set ns_state.thinking = true %}{%- set ns_state.effort = 'medium' %}{%- endif %}{%- endif %}{%- endfor %}{%- endif %}{%- endif %}{%- endfor %}{%- set reasoning_instructions = '' %}{%- if ns_state.thinking %}{%- if ns_state.effort == 'xhigh' %}{%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}{%- elif ns_state.effort == 'low' %}{%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}{%- endif %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set head = namespace(count=0, seen_non_system=false) %}{%- for message in messages %}{%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}{%- if _is_sys and not head.seen_non_system %}{%- set head.count = head.count + 1 %}{%- else %}{%- set head.seen_non_system = true %}{%- endif %}{%- endfor %}{%- set sys_state = namespace(content='') %}{%- for message in messages[:head.count] %}{%- set _part = render_content(message.content, false, true) | trim %}{%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- if _part %}{%- if sys_state.content %}{%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}{%- else %}{%- set sys_state.content = _part %}{%- endif %}{%- endif %}{%- endfor %}{%- set _sc = sys_state.content %}{%- set _msgs = messages[head.count:] %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{%- if reasoning_instructions %}{{- reasoning_instructions + '\n\n' }}{%- endif %}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- if _tool_format == 'json' %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- else %}{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<think>\nBrief explanation of tool call\n</think>\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}{%- endif %}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}{%- elif reasoning_instructions %}{{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}{%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}{%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}{%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}{%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}{%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}{%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}{%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}{%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- set _explicit_reasoning = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set _explicit_reasoning = message.reasoning_content %}{%- else %}{%- set _explicit_reasoning = message.reasoning_content | string %}{%- endif %}{%- elif message.thinking is defined and message.thinking is not none %}{%- if message.thinking is string %}{%- set _explicit_reasoning = message.thinking %}{%- else %}{%- set _explicit_reasoning = message.thinking | string %}{%- endif %}{%- elif message.reasoning is defined and message.reasoning is not none %}{%- if message.reasoning is string %}{%- set _explicit_reasoning = message.reasoning %}{%- else %}{%- set _explicit_reasoning = message.reasoning | string %}{%- endif %}{%- endif %}{%- if _explicit_reasoning %}{%- set _lead_end = '' %}{%- if content.startswith('<think>') and '</think>' in content %}{%- set _lead_end = '</think>' %}{%- elif content.startswith('<thinking>') and '</thinking>' in content %}{%- set _lead_end = '</thinking>' %}{%- elif content.startswith('</think>') %}{%- set _lead_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _lead_end = '</thinking>' %}{%- endif %}{%- if _lead_end %}{%- set content = content.split(_lead_end)[-1].lstrip('\n') %}{%- endif %}{%- set reasoning_content = _explicit_reasoning %}{%- else %}{%- set _think_end = '' %}{%- if content.startswith('</think>') %}{%- set _think_end = '</think>' %}{%- elif content.startswith('</thinking>') %}{%- set _think_end = '</thinking>' %}{%- elif '\n</think>' in content %}{%- set _think_end = '\n</think>' %}{%- elif '\n</thinking>' in content %}{%- set _think_end = '\n</thinking>' %}{%- elif '\n</ think>' in content %}{%- set _think_end = '\n</ think>' %}{%- elif '\n</think >' in content %}{%- set _think_end = '\n</think >' %}{%- elif content.startswith('<think>') and '</think>' in content %}{%- set _think_end = '</think>' %}{%- elif content.startswith('<thinking>') and '</thinking>' in content %}{%- set _think_end = '</thinking>' %}{%- endif %}{%- if _think_end %}{%- if 'thinking' in _think_end %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}{%- if _tool_format == 'json' %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n' }}{%- endif %}{%- else %}{{- '\n' }}{%- endif %}{%- set _args = '{}' %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- set _args = tc.arguments | tojson %}{%- elif tc.arguments is string %}{%- if tc.arguments %}{%- set _args = tc.arguments %}{%- endif %}{%- else %}{%- set _args = tc.arguments | tojson %}{%- endif %}{%- endif %}{{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}{%- else %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- else %}{{- '\n<tool_call>\n<function=' + tc_name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is string %}{%- set _av = args_value %}{%- else %}{%- set _av = args_value | tojson %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- else %}{%- if tc.arguments is string %}{%- set _raw_args = tc.arguments %}{%- else %}{%- set _raw_args = tc.arguments | tojson %}{%- endif %}{%- if _raw_args %}{%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}{{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}{%- else %}{{- _raw_args }}{%- endif %}{%- endif %}{%- endif %}{%- endif %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- set _content_head = _content_lower[:120] %}{%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}{%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}{%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}{%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}{%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}{%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}{%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n\n</think>\n\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.6-froggeric-v20" %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else false %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set ns_state = namespace(thinking=enable_thinking) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set _first_role = messages[0].role %}{%- if _first_role == 'system' or _first_role == 'developer' %}{%- set _sys_msg = messages[0] %}{%- set _msgs = messages[1:] %}{%- else %}{%- set _sys_msg = none %}{%- set _msgs = messages %}{%- endif %}{%- set _sc = '' %}{%- if _sys_msg is not none %}{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}{%- if '<|think_off|>' in _sc %}{%- set ns_state.thinking = false %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in _sc %}{%- set ns_state.thinking = true %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- set tool_instructions %} If you choose to call a function ONLY reply in the following format with NO suffix: <think> Brief explanation of tool call </think> <tool_call> <function=example_function_name> <parameter=example_parameter_1> value_1 </parameter> <parameter=example_parameter_2> This is the value for the second parameter that can span multiple lines </parameter> </function> </tool_call> <IMPORTANT> Reminder: - You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user. - ALL explanation and reasoning MUST be placed strictly inside the <think></think> block. - Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags. - If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after closing </think>. Do NOT output any conversational text before the tool call. - The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them. - To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks. - If you have gathered all necessary data and do not need to call a tool, answer the question like normal and provide your final response to the user IMMEDIATELY after closing </think>. </IMPORTANT> {%- endset %}{{- '\n\n' ~ tool_instructions | trim }}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if is_system or message.role == 'user' %}{%- if '<|think_off|>' in content %}{%- set ns_state.thinking = false %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in content %}{%- set ns_state.thinking = true %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if '</think>' in content %}{%- set _think_end = '</think>' %}{%- elif '</thinking>' in content %}{%- set _think_end = '</thinking>' %}{%- elif '</ think>' in content %}{%- set _think_end = '</ think>' %}{%- elif '</think >' in content %}{%- set _think_end = '</think >' %}{%- endif %}{%- if _think_end %}{%- if _think_end == '</thinking>' %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc.name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}{%- set _av = args_value | tojson %}{%- else %}{%- set _av = args_value | string %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{%- if loop.last %}{{- '</function>\n</tool_call>\n' }}{%- else %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_lower or 'error:' in _content_lower or 'exception:' in _content_lower or 'traceback' in _content_lower or 'command not found' in _content_lower or 'invalid syntax' in _content_lower or 'failed to' in _content_lower) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n</think>\n' }}{%- elif ns2.consecutive_failures >= 2 %}{{- '<think>\n</think>\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
scripts/check_applied.py
DELETED
|
@@ -1,139 +0,0 @@
|
|
| 1 |
-
#!/usr/bin/env python3
|
| 2 |
-
"""Diagnostic utility to verify which chat template is active on a model directory or GGUF file.
|
| 3 |
-
|
| 4 |
-
Usage:
|
| 5 |
-
python3 scripts/check_applied.py [path_to_model_dir_or_gguf]
|
| 6 |
-
|
| 7 |
-
Examples:
|
| 8 |
-
python3 scripts/check_applied.py .
|
| 9 |
-
python3 scripts/check_applied.py /models/Qwen3.8-27B-Instruct
|
| 10 |
-
python3 scripts/check_applied.py /models/qwen3.8-27b.gguf
|
| 11 |
-
"""
|
| 12 |
-
|
| 13 |
-
import sys
|
| 14 |
-
import os
|
| 15 |
-
import re
|
| 16 |
-
import json
|
| 17 |
-
|
| 18 |
-
GREEN = "\033[92m"
|
| 19 |
-
RED = "\033[91m"
|
| 20 |
-
YELLOW = "\033[93m"
|
| 21 |
-
CYAN = "\033[96m"
|
| 22 |
-
BOLD = "\033[1m"
|
| 23 |
-
RESET = "\033[0m"
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
def extract_template_version(content: str) -> str:
|
| 27 |
-
if not content:
|
| 28 |
-
return "None"
|
| 29 |
-
m = re.search(r'template_version\s*=\s*["\']([^"\']+)["\']', content)
|
| 30 |
-
if m:
|
| 31 |
-
return m.group(1)
|
| 32 |
-
if "qwen" in content.lower() or "im_start" in content:
|
| 33 |
-
return "Stock / Unknown Qwen Template"
|
| 34 |
-
return "Unknown"
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
def inspect_gguf(path: str):
|
| 38 |
-
try:
|
| 39 |
-
import gguf
|
| 40 |
-
reader = gguf.GGUFReader(path)
|
| 41 |
-
for field in reader.fields.values():
|
| 42 |
-
if field.name == "tokenizer.chat_template":
|
| 43 |
-
parts = field.parts
|
| 44 |
-
for idx in field.data:
|
| 45 |
-
val = str(parts[idx])
|
| 46 |
-
if "im_start" in val:
|
| 47 |
-
return val
|
| 48 |
-
return None
|
| 49 |
-
except ImportError:
|
| 50 |
-
# Fallback: simple binary scan for template string
|
| 51 |
-
try:
|
| 52 |
-
with open(path, "rb") as f:
|
| 53 |
-
# Read first 10MB of GGUF header
|
| 54 |
-
buf = f.read(10 * 1024 * 1024).decode("utf-8", errors="ignore")
|
| 55 |
-
m = re.search(r'template_version\s*=\s*["\']([^"\']+)["\']', buf)
|
| 56 |
-
if m:
|
| 57 |
-
return f'{{%- set template_version = "{m.group(1)}" %}}'
|
| 58 |
-
except Exception:
|
| 59 |
-
pass
|
| 60 |
-
return None
|
| 61 |
-
except Exception as e:
|
| 62 |
-
return None
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
def main():
|
| 66 |
-
target = sys.argv[1] if len(sys.argv) > 1 else "."
|
| 67 |
-
|
| 68 |
-
print(f"\n{BOLD}{CYAN}=== Qwen Chat Template Diagnostic Utility ==={RESET}")
|
| 69 |
-
print(f"Target: {os.path.abspath(target)}\n")
|
| 70 |
-
|
| 71 |
-
sources = {}
|
| 72 |
-
|
| 73 |
-
if os.path.isfile(target):
|
| 74 |
-
if target.endswith(".gguf"):
|
| 75 |
-
gguf_template = inspect_gguf(target)
|
| 76 |
-
sources["gguf"] = gguf_template
|
| 77 |
-
elif target.endswith(".jinja"):
|
| 78 |
-
with open(target, "r", encoding="utf-8") as f:
|
| 79 |
-
sources["chat_template.jinja"] = f.read()
|
| 80 |
-
elif target.endswith(".json"):
|
| 81 |
-
with open(target, "r", encoding="utf-8") as f:
|
| 82 |
-
data = json.load(f)
|
| 83 |
-
sources["tokenizer_config.json"] = data.get("chat_template", "")
|
| 84 |
-
elif os.path.isdir(target):
|
| 85 |
-
jinja_path = os.path.join(target, "chat_template.jinja")
|
| 86 |
-
json_path = os.path.join(target, "tokenizer_config.json")
|
| 87 |
-
|
| 88 |
-
if os.path.exists(jinja_path):
|
| 89 |
-
with open(jinja_path, "r", encoding="utf-8") as f:
|
| 90 |
-
sources["chat_template.jinja"] = f.read()
|
| 91 |
-
|
| 92 |
-
if os.path.exists(json_path):
|
| 93 |
-
try:
|
| 94 |
-
with open(json_path, "r", encoding="utf-8") as f:
|
| 95 |
-
data = json.load(f)
|
| 96 |
-
sources["tokenizer_config.json"] = data.get("chat_template", "")
|
| 97 |
-
except Exception as e:
|
| 98 |
-
sources["tokenizer_config.json"] = f"Error reading JSON: {e}"
|
| 99 |
-
|
| 100 |
-
if not sources:
|
| 101 |
-
print(f"{RED}❌ No template files found in {target}{RESET}")
|
| 102 |
-
print("Expected `chat_template.jinja`, `tokenizer_config.json`, or a `.gguf` file.\n")
|
| 103 |
-
sys.exit(1)
|
| 104 |
-
|
| 105 |
-
versions = {}
|
| 106 |
-
for name, content in sources.items():
|
| 107 |
-
ver = extract_template_version(content)
|
| 108 |
-
versions[name] = ver
|
| 109 |
-
status_color = GREEN if "froggeric" in ver else YELLOW
|
| 110 |
-
print(f" [{name}]")
|
| 111 |
-
print(f" Version detected: {status_color}{ver}{RESET}")
|
| 112 |
-
print(f" Size: {len(content) if content else 0} characters")
|
| 113 |
-
|
| 114 |
-
# Check for mismatches
|
| 115 |
-
unique_versions = set(versions.values())
|
| 116 |
-
print("\n" + "-" * 50)
|
| 117 |
-
|
| 118 |
-
if len(sources) > 1 and len(unique_versions) > 1:
|
| 119 |
-
print(f"{YELLOW}⚠️ WARNING: TEMPLATE SOURCE MISMATCH DETECTED!{RESET}")
|
| 120 |
-
print("Your runtime may pick a different template depending on precedence:")
|
| 121 |
-
print(" - Transformers ≥ 4.51 / LM Studio: prefers `chat_template.jinja`")
|
| 122 |
-
print(" - oMLX / legacy engines: reads `tokenizer_config.json`")
|
| 123 |
-
print(f"\n{BOLD}Recommendation:{RESET} Overwrite both sources with `chat_template.jinja` and `chat_template_oneline.txt`.\n")
|
| 124 |
-
sys.exit(1)
|
| 125 |
-
else:
|
| 126 |
-
active_ver = list(unique_versions)[0]
|
| 127 |
-
if "froggeric-v22.3" in active_ver:
|
| 128 |
-
print(f"{GREEN}✅ SUCCESS: Verified v22.3 template is cleanly installed!{RESET}\n")
|
| 129 |
-
sys.exit(0)
|
| 130 |
-
elif "froggeric" in active_ver:
|
| 131 |
-
print(f"{CYAN}ℹ️ INFO: Found earlier fixed template version ({active_ver}). Upgrade to v22.3 recommended.{RESET}\n")
|
| 132 |
-
sys.exit(0)
|
| 133 |
-
else:
|
| 134 |
-
print(f"{YELLOW}⚠️ Stock / unpatched template detected.{RESET}\n")
|
| 135 |
-
sys.exit(1)
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
if __name__ == "__main__":
|
| 139 |
-
main()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
scripts/fuzz_template.py
DELETED
|
@@ -1,290 +0,0 @@
|
|
| 1 |
-
"""Deterministic property-based fuzzer for the fixed Qwen chat templates.
|
| 2 |
-
|
| 3 |
-
Generates structurally valid conversations (system merging, multipart vision
|
| 4 |
-
content, tool loops with mixed argument shapes, explicit and in-content
|
| 5 |
-
reasoning, planted error payloads) and asserts nine invariants:
|
| 6 |
-
|
| 7 |
-
1. render Rendering never raises on valid input.
|
| 8 |
-
2. parity chat_template.jinja and chat_template_oneline.txt render
|
| 9 |
-
byte-identically.
|
| 10 |
-
3. balance <|im_start|> and <|im_end|> counts match
|
| 11 |
-
(add_generation_prompt=False).
|
| 12 |
-
4. content Planted user text, assistant answers, and (when preserved)
|
| 13 |
-
reasoning appear verbatim in the output.
|
| 14 |
-
5. xml-fidelity In XML mode every mapping tool-argument key appears as
|
| 15 |
-
<parameter=key>; values appear verbatim when truncation
|
| 16 |
-
is disabled.
|
| 17 |
-
6. json-validity In JSON mode every emitted <tool_call> body parses as JSON.
|
| 18 |
-
7. warning The tool-error warning appears iff an error payload was
|
| 19 |
-
planted (no false positives, no false negatives).
|
| 20 |
-
8. prefix render(messages[:k]) is a strict prefix of
|
| 21 |
-
render(messages[:k+1]) at every generation boundary under
|
| 22 |
-
default preserve_thinking. Prefixes that split a merged
|
| 23 |
-
system block or a consecutive tool-result batch are
|
| 24 |
-
skipped: those intermediate states are never rendered in
|
| 25 |
-
real serving, where the model generates only after the
|
| 26 |
-
full tool-result batch is appended.
|
| 27 |
-
9. prefill enable_thinking=False ends the generation prompt with the
|
| 28 |
-
empty think prefill.
|
| 29 |
-
|
| 30 |
-
Usage:
|
| 31 |
-
python3 scripts/fuzz_template.py [--cases 500] [--seed 0] [--template chat_template.jinja]
|
| 32 |
-
|
| 33 |
-
Exits nonzero on any invariant failure and prints a JSON repro for each.
|
| 34 |
-
"""
|
| 35 |
-
import argparse
|
| 36 |
-
import json
|
| 37 |
-
import os
|
| 38 |
-
import random
|
| 39 |
-
import sys
|
| 40 |
-
import traceback
|
| 41 |
-
|
| 42 |
-
from jinja2 import Environment, FileSystemLoader, StrictUndefined
|
| 43 |
-
|
| 44 |
-
# Alphabet deliberately excludes every error-detector keyword, control-tag
|
| 45 |
-
# fragment, and special-token prefix so invariant 7 stays two-sided.
|
| 46 |
-
WORDS = ['alpha', 'bravo', 'delta', 'gamma', 'lumen', 'quartz', 'river',
|
| 47 |
-
'stone', 'matrix', 'vector', 'naïve', 'café', '数据', '結果']
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
def _phrase(rnd, lo=2, hi=5):
|
| 51 |
-
sep = '\n' if rnd.random() < 0.1 else ' '
|
| 52 |
-
return sep.join(rnd.choice(WORDS) + str(rnd.randint(0, 99))
|
| 53 |
-
for _ in range(rnd.randint(lo, hi)))
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
def _gen_case(rnd):
|
| 57 |
-
kwargs = {}
|
| 58 |
-
fmt = 'json' if rnd.random() < 0.3 else 'xml'
|
| 59 |
-
if fmt == 'json':
|
| 60 |
-
kwargs['tool_call_format'] = 'json'
|
| 61 |
-
if rnd.random() < 0.3:
|
| 62 |
-
kwargs['reasoning_effort'] = rnd.choice(['low', 'medium', 'high', 'xhigh'])
|
| 63 |
-
no_think = rnd.random() < 0.15
|
| 64 |
-
if no_think:
|
| 65 |
-
kwargs['enable_thinking'] = False
|
| 66 |
-
preserve = True
|
| 67 |
-
if rnd.random() < 0.2:
|
| 68 |
-
kwargs['preserve_thinking'] = False
|
| 69 |
-
preserve = False
|
| 70 |
-
if rnd.random() < 0.2:
|
| 71 |
-
kwargs['add_vision_id'] = True
|
| 72 |
-
trunc_args = False
|
| 73 |
-
if fmt == 'xml' and rnd.random() < 0.2:
|
| 74 |
-
kwargs['max_tool_arg_chars'] = rnd.choice([8, 40])
|
| 75 |
-
trunc_args = True
|
| 76 |
-
if fmt == 'xml' and rnd.random() < 0.15:
|
| 77 |
-
kwargs['max_tool_response_chars'] = 80
|
| 78 |
-
|
| 79 |
-
msgs = []
|
| 80 |
-
user_texts, answers, reasonings, xml_keys, xml_vals = [], [], [], [], []
|
| 81 |
-
planted_error = False
|
| 82 |
-
|
| 83 |
-
for _ in range(rnd.randint(0, 2)):
|
| 84 |
-
msgs.append({'role': 'system', 'content': _phrase(rnd)})
|
| 85 |
-
|
| 86 |
-
for _ in range(rnd.randint(1, 3)):
|
| 87 |
-
text = _phrase(rnd)
|
| 88 |
-
if rnd.random() < 0.25:
|
| 89 |
-
parts = [{'type': 'image', 'image': 'x'}
|
| 90 |
-
for _ in range(rnd.randint(1, 2))]
|
| 91 |
-
parts.append({'type': 'text', 'text': text})
|
| 92 |
-
msgs.append({'role': 'user', 'content': parts})
|
| 93 |
-
else:
|
| 94 |
-
msgs.append({'role': 'user', 'content': text})
|
| 95 |
-
user_texts.append(text)
|
| 96 |
-
|
| 97 |
-
for _ in range(rnd.randint(0, 2)):
|
| 98 |
-
calls = []
|
| 99 |
-
for c in range(rnd.randint(1, 2)):
|
| 100 |
-
roll = rnd.random()
|
| 101 |
-
if roll < 0.5:
|
| 102 |
-
value = _phrase(rnd)
|
| 103 |
-
args = {'k%d' % c: value}
|
| 104 |
-
xml_keys.append('k%d' % c)
|
| 105 |
-
if fmt == 'xml' and not trunc_args:
|
| 106 |
-
xml_vals.append(value)
|
| 107 |
-
elif roll < 0.7:
|
| 108 |
-
args = json.dumps({'q': _phrase(rnd)})
|
| 109 |
-
elif roll < 0.8:
|
| 110 |
-
args = rnd.randint(0, 999)
|
| 111 |
-
elif roll < 0.9:
|
| 112 |
-
args = [1, 2, 3]
|
| 113 |
-
else:
|
| 114 |
-
args = {}
|
| 115 |
-
fn = {'name': 'fn%d' % c, 'arguments': args}
|
| 116 |
-
calls.append({'type': 'function', 'function': fn}
|
| 117 |
-
if rnd.random() < 0.5 else dict(fn))
|
| 118 |
-
amsg = {'role': 'assistant',
|
| 119 |
-
'content': _phrase(rnd) if rnd.random() < 0.4 else '',
|
| 120 |
-
'tool_calls': calls}
|
| 121 |
-
if rnd.random() < 0.5:
|
| 122 |
-
rz = _phrase(rnd)
|
| 123 |
-
r_pick = rnd.random()
|
| 124 |
-
if r_pick < 0.34:
|
| 125 |
-
amsg['content'] = '<think>\n' + rz + '\n</think>\n\n' + amsg['content']
|
| 126 |
-
elif r_pick < 0.67:
|
| 127 |
-
amsg['reasoning_content'] = rz
|
| 128 |
-
else:
|
| 129 |
-
amsg['reasoning'] = rz
|
| 130 |
-
if preserve:
|
| 131 |
-
reasonings.append(rz)
|
| 132 |
-
msgs.append(amsg)
|
| 133 |
-
for _ in calls:
|
| 134 |
-
if rnd.random() < 0.12:
|
| 135 |
-
msgs.append({'role': 'tool', 'content': '{"error": "boom"}'})
|
| 136 |
-
planted_error = True
|
| 137 |
-
else:
|
| 138 |
-
msgs.append({'role': 'tool', 'content': 'result ' + _phrase(rnd)})
|
| 139 |
-
|
| 140 |
-
ans = _phrase(rnd)
|
| 141 |
-
amsg = {'role': 'assistant', 'content': ans}
|
| 142 |
-
if rnd.random() < 0.6:
|
| 143 |
-
rz = _phrase(rnd)
|
| 144 |
-
r_pick = rnd.random()
|
| 145 |
-
if r_pick < 0.34:
|
| 146 |
-
amsg['content'] = '<think>\n' + rz + '\n</think>\n\n' + ans
|
| 147 |
-
elif r_pick < 0.67:
|
| 148 |
-
amsg['reasoning_content'] = rz
|
| 149 |
-
else:
|
| 150 |
-
amsg['reasoning'] = rz
|
| 151 |
-
if preserve:
|
| 152 |
-
reasonings.append(rz)
|
| 153 |
-
msgs.append(amsg)
|
| 154 |
-
answers.append(ans)
|
| 155 |
-
|
| 156 |
-
return dict(msgs=msgs, kwargs=kwargs, fmt=fmt, preserve=preserve,
|
| 157 |
-
no_think=no_think, user_texts=user_texts, answers=answers,
|
| 158 |
-
reasonings=reasonings, xml_keys=xml_keys, xml_vals=xml_vals,
|
| 159 |
-
planted_error=planted_error)
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
def _check(case, tpl, other, failures, idx):
|
| 163 |
-
msgs, kw = case['msgs'], case['kwargs']
|
| 164 |
-
|
| 165 |
-
def fail(inv, detail):
|
| 166 |
-
failures.append({
|
| 167 |
-
'case': idx, 'invariant': inv, 'detail': detail,
|
| 168 |
-
'repro': json.dumps({'messages': msgs, 'kwargs': kw},
|
| 169 |
-
ensure_ascii=False, default=str)})
|
| 170 |
-
|
| 171 |
-
try:
|
| 172 |
-
out = tpl.render(messages=msgs, add_generation_prompt=False, **kw)
|
| 173 |
-
except Exception:
|
| 174 |
-
fail('render', traceback.format_exc().strip().splitlines()[-1])
|
| 175 |
-
return
|
| 176 |
-
try:
|
| 177 |
-
out_b = other.render(messages=msgs, add_generation_prompt=False, **kw)
|
| 178 |
-
if out != out_b:
|
| 179 |
-
i = next((j for j in range(min(len(out), len(out_b)))
|
| 180 |
-
if out[j] != out_b[j]), min(len(out), len(out_b)))
|
| 181 |
-
fail('parity', 'first diff at char %d: %r vs %r'
|
| 182 |
-
% (i, out[i:i + 40], out_b[i:i + 40]))
|
| 183 |
-
except Exception:
|
| 184 |
-
fail('parity', traceback.format_exc().strip().splitlines()[-1])
|
| 185 |
-
|
| 186 |
-
if out.count('<|im_start|>') != out.count('<|im_end|>'):
|
| 187 |
-
fail('balance', '%d starts vs %d ends'
|
| 188 |
-
% (out.count('<|im_start|>'), out.count('<|im_end|>')))
|
| 189 |
-
|
| 190 |
-
for text in case['user_texts'] + case['answers'] + case['reasonings']:
|
| 191 |
-
if text not in out:
|
| 192 |
-
fail('content', 'missing %r' % text[:60])
|
| 193 |
-
break
|
| 194 |
-
|
| 195 |
-
if case['fmt'] == 'xml':
|
| 196 |
-
for key in case['xml_keys']:
|
| 197 |
-
if ('<parameter=%s>' % key) not in out:
|
| 198 |
-
fail('xml-fidelity', 'missing key %s' % key)
|
| 199 |
-
break
|
| 200 |
-
for value in case['xml_vals']:
|
| 201 |
-
if value not in out:
|
| 202 |
-
fail('xml-fidelity', 'missing value %r' % value[:60])
|
| 203 |
-
break
|
| 204 |
-
|
| 205 |
-
if case['fmt'] == 'json':
|
| 206 |
-
for block in out.split('<tool_call>\n')[1:]:
|
| 207 |
-
body = block.split('\n</tool_call>')[0]
|
| 208 |
-
try:
|
| 209 |
-
json.loads(body)
|
| 210 |
-
except Exception:
|
| 211 |
-
fail('json-validity', body[:80])
|
| 212 |
-
break
|
| 213 |
-
|
| 214 |
-
warned = '⚠️ SYSTEM WARNING' in out
|
| 215 |
-
if warned != case['planted_error']:
|
| 216 |
-
fail('warning', 'warned=%s planted=%s' % (warned, case['planted_error']))
|
| 217 |
-
|
| 218 |
-
if case['preserve']:
|
| 219 |
-
prev = None
|
| 220 |
-
for k in range(1, len(msgs) + 1):
|
| 221 |
-
if k < len(msgs) and msgs[k]['role'] == msgs[k - 1]['role'] \
|
| 222 |
-
and msgs[k]['role'] in ('system', 'tool'):
|
| 223 |
-
continue
|
| 224 |
-
cur = tpl.render(messages=msgs[:k], add_generation_prompt=False, **kw)
|
| 225 |
-
|
| 226 |
-
if k > 1 and msgs[k - 1].get('role') == 'assistant':
|
| 227 |
-
prompt = tpl.render(messages=msgs[:k - 1], add_generation_prompt=True, **kw)
|
| 228 |
-
if not cur.startswith(prompt):
|
| 229 |
-
if prompt.endswith('<think>\n\n</think>\n\n') and cur.startswith(prompt[:-11]):
|
| 230 |
-
# Fuzzer randomly injected reasoning into a non-thinking turn. KV cache naturally breaks here.
|
| 231 |
-
pass
|
| 232 |
-
else:
|
| 233 |
-
fail('prefix', 'generation prompt at turn %d not prefix of history at turn %d' % (k - 1, k))
|
| 234 |
-
break
|
| 235 |
-
|
| 236 |
-
if prev is not None and not cur.startswith(prev):
|
| 237 |
-
fail('prefix', 'history mutated at turn %d' % k)
|
| 238 |
-
break
|
| 239 |
-
prev = cur
|
| 240 |
-
|
| 241 |
-
if case['no_think']:
|
| 242 |
-
gen = tpl.render(messages=msgs, add_generation_prompt=True, **kw)
|
| 243 |
-
if not gen.endswith('<think>\n\n</think>\n\n'):
|
| 244 |
-
fail('prefill', repr(gen[-40:]))
|
| 245 |
-
|
| 246 |
-
|
| 247 |
-
def run_fuzz(cases=500, seed=0, template_dir=None,
|
| 248 |
-
template_file='chat_template.jinja', max_failures=10):
|
| 249 |
-
template_dir = template_dir or os.path.dirname(
|
| 250 |
-
os.path.dirname(os.path.abspath(__file__)))
|
| 251 |
-
env = Environment(loader=FileSystemLoader(template_dir),
|
| 252 |
-
undefined=StrictUndefined, keep_trailing_newline=True,
|
| 253 |
-
lstrip_blocks=True, trim_blocks=True)
|
| 254 |
-
env.globals['raise_exception'] = \
|
| 255 |
-
lambda m: (_ for _ in ()).throw(Exception(m))
|
| 256 |
-
tpl = env.get_template(template_file)
|
| 257 |
-
other_name = ('chat_template.jinja'
|
| 258 |
-
if template_file == 'chat_template_oneline.txt'
|
| 259 |
-
else 'chat_template_oneline.txt')
|
| 260 |
-
other = env.get_template(other_name)
|
| 261 |
-
rnd = random.Random(seed)
|
| 262 |
-
failures = []
|
| 263 |
-
for i in range(cases):
|
| 264 |
-
_check(_gen_case(rnd), tpl, other, failures, i)
|
| 265 |
-
if len(failures) >= max_failures:
|
| 266 |
-
break
|
| 267 |
-
return failures
|
| 268 |
-
|
| 269 |
-
|
| 270 |
-
def main():
|
| 271 |
-
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
| 272 |
-
parser.add_argument('--cases', type=int, default=500)
|
| 273 |
-
parser.add_argument('--seed', type=int, default=0)
|
| 274 |
-
parser.add_argument('--template', default='chat_template.jinja')
|
| 275 |
-
args = parser.parse_args()
|
| 276 |
-
failures = run_fuzz(cases=args.cases, seed=args.seed,
|
| 277 |
-
template_file=args.template)
|
| 278 |
-
if failures:
|
| 279 |
-
for f in failures:
|
| 280 |
-
print('FAIL case %d [%s]: %s' % (f['case'], f['invariant'], f['detail']))
|
| 281 |
-
print(' repro: %s' % f['repro'][:800])
|
| 282 |
-
print('\n%d invariant violation(s) in %d cases (seed %d)'
|
| 283 |
-
% (len(failures), args.cases, args.seed))
|
| 284 |
-
sys.exit(1)
|
| 285 |
-
print('All invariants held over %d generated conversations (seed %d).'
|
| 286 |
-
% (args.cases, args.seed))
|
| 287 |
-
|
| 288 |
-
|
| 289 |
-
if __name__ == '__main__':
|
| 290 |
-
main()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
scripts/{test_v21.py → test_v20.py}
RENAMED
|
@@ -93,7 +93,7 @@ execute_test(
|
|
| 93 |
messages=[{"role": "user", "content": "Hello!"}],
|
| 94 |
tools=[{"name": "test_tool"}],
|
| 95 |
kwargs={"auto_disable_thinking_with_tools": True},
|
| 96 |
-
expected_in=["<think>\n
|
| 97 |
)
|
| 98 |
|
| 99 |
execute_test(
|
|
@@ -120,7 +120,7 @@ execute_test(
|
|
| 120 |
"max_tool_arg_chars truncation",
|
| 121 |
messages=[{"role": "user", "content": "Call tool"}, {"role": "assistant", "content": "", "tool_calls": [{"function": {"name": "test", "arguments": {"param": "1234567890"}}}]}],
|
| 122 |
kwargs={"max_tool_arg_chars": 5},
|
| 123 |
-
expected_in=["
|
| 124 |
)
|
| 125 |
|
| 126 |
execute_test(
|
|
@@ -141,7 +141,7 @@ execute_test(
|
|
| 141 |
execute_test(
|
| 142 |
"parallel tools delimiter",
|
| 143 |
messages=[{"role": "user", "content": "x"}, {"role": "assistant", "content": "", "tool_calls": [{"name": "t1"}, {"name": "t2"}]}],
|
| 144 |
-
expected_in=["</
|
| 145 |
)
|
| 146 |
|
| 147 |
# 6. Deep Agent Fallback
|
|
@@ -160,18 +160,7 @@ execute_test(
|
|
| 160 |
{"role": "assistant", "content": "calling"},
|
| 161 |
{"role": "tool", "content": "error: failed again"}
|
| 162 |
],
|
| 163 |
-
expected_in=["⚠️ SYSTEM WARNING: 2 consecutive tool errors", "<think>\n"]
|
| 164 |
-
)
|
| 165 |
-
|
| 166 |
-
run_test(
|
| 167 |
-
"tool_call_format='json' (override)",
|
| 168 |
-
messages=[{"role": "user", "content": "Call tool"}, {"role": "assistant", "content": "", "tool_calls": [{"function": {"name": "test", "arguments": {"par": "1234567890"}}}]}],
|
| 169 |
-
tools=[{"type": "function", "function": {"name": "test", "description": "test tool"}}],
|
| 170 |
-
kwargs={'tool_call_format': 'json'},
|
| 171 |
-
expected_in=[
|
| 172 |
-
'Function calls MUST follow the specified format: a single JSON object with "name" and "arguments"',
|
| 173 |
-
'{"name": "test", "arguments": {"par": "1234567890"}}'
|
| 174 |
-
]
|
| 175 |
)
|
| 176 |
|
| 177 |
print(f"\n=============================")
|
|
|
|
| 93 |
messages=[{"role": "user", "content": "Hello!"}],
|
| 94 |
tools=[{"name": "test_tool"}],
|
| 95 |
kwargs={"auto_disable_thinking_with_tools": True},
|
| 96 |
+
expected_in=["<think>\n</think>\n"], # Should be stripped
|
| 97 |
)
|
| 98 |
|
| 99 |
execute_test(
|
|
|
|
| 120 |
"max_tool_arg_chars truncation",
|
| 121 |
messages=[{"role": "user", "content": "Call tool"}, {"role": "assistant", "content": "", "tool_calls": [{"function": {"name": "test", "arguments": {"param": "1234567890"}}}]}],
|
| 122 |
kwargs={"max_tool_arg_chars": 5},
|
| 123 |
+
expected_in=["12345\n[TRUNCATED"]
|
| 124 |
)
|
| 125 |
|
| 126 |
execute_test(
|
|
|
|
| 141 |
execute_test(
|
| 142 |
"parallel tools delimiter",
|
| 143 |
messages=[{"role": "user", "content": "x"}, {"role": "assistant", "content": "", "tool_calls": [{"name": "t1"}, {"name": "t2"}]}],
|
| 144 |
+
expected_in=["</tool_call>\n\n<tool_call>"]
|
| 145 |
)
|
| 146 |
|
| 147 |
# 6. Deep Agent Fallback
|
|
|
|
| 160 |
{"role": "assistant", "content": "calling"},
|
| 161 |
{"role": "tool", "content": "error: failed again"}
|
| 162 |
],
|
| 163 |
+
expected_in=["⚠️ SYSTEM WARNING: 2 consecutive tool errors", "<think>\n</think>\n"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 164 |
)
|
| 165 |
|
| 166 |
print(f"\n=============================")
|
scripts/test_v22.py
DELETED
|
@@ -1,1740 +0,0 @@
|
|
| 1 |
-
import os
|
| 2 |
-
import sys
|
| 3 |
-
import re
|
| 4 |
-
import json
|
| 5 |
-
import traceback
|
| 6 |
-
|
| 7 |
-
try:
|
| 8 |
-
from jinja2 import Environment, FileSystemLoader, StrictUndefined
|
| 9 |
-
except ImportError:
|
| 10 |
-
print("Error: jinja2 is required to run tests. Please install it using 'pip install jinja2'")
|
| 11 |
-
sys.exit(1)
|
| 12 |
-
|
| 13 |
-
TEMPLATE_FILE = os.environ.get('QWEN_TEMPLATE_FILE', 'chat_template.jinja')
|
| 14 |
-
TEMPLATE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 15 |
-
|
| 16 |
-
env = Environment(
|
| 17 |
-
loader=FileSystemLoader(TEMPLATE_DIR),
|
| 18 |
-
undefined=StrictUndefined,
|
| 19 |
-
keep_trailing_newline=True,
|
| 20 |
-
lstrip_blocks=True,
|
| 21 |
-
trim_blocks=True
|
| 22 |
-
)
|
| 23 |
-
|
| 24 |
-
def raise_exception(msg):
|
| 25 |
-
raise Exception(msg)
|
| 26 |
-
|
| 27 |
-
env.globals['raise_exception'] = raise_exception
|
| 28 |
-
|
| 29 |
-
try:
|
| 30 |
-
template = env.get_template(TEMPLATE_FILE)
|
| 31 |
-
except Exception as e:
|
| 32 |
-
print(f"Error loading template: {e}")
|
| 33 |
-
sys.exit(1)
|
| 34 |
-
|
| 35 |
-
def run_test(name, messages, tools=None, kwargs=None, expected_in=None, expected_not_in=None, expect_error=False):
|
| 36 |
-
if kwargs is None:
|
| 37 |
-
kwargs = {}
|
| 38 |
-
|
| 39 |
-
print(f"\n--- Running Test: {name} ---")
|
| 40 |
-
|
| 41 |
-
try:
|
| 42 |
-
render_kwargs = {'messages': messages, 'add_generation_prompt': True}
|
| 43 |
-
if tools is not None:
|
| 44 |
-
render_kwargs['tools'] = tools
|
| 45 |
-
render_kwargs.update(kwargs)
|
| 46 |
-
|
| 47 |
-
rendered = template.render(**render_kwargs)
|
| 48 |
-
|
| 49 |
-
if expect_error:
|
| 50 |
-
print("❌ FAILED: Expected an exception but got none.")
|
| 51 |
-
return False
|
| 52 |
-
|
| 53 |
-
success = True
|
| 54 |
-
|
| 55 |
-
if expected_in:
|
| 56 |
-
for ex in expected_in:
|
| 57 |
-
if ex not in rendered:
|
| 58 |
-
print(f"❌ FAILED: Missing expected string:\n'''{ex}'''")
|
| 59 |
-
print(f"Rendered:\n{rendered}")
|
| 60 |
-
success = False
|
| 61 |
-
|
| 62 |
-
if expected_not_in:
|
| 63 |
-
for n_ex in expected_not_in:
|
| 64 |
-
if n_ex in rendered:
|
| 65 |
-
print(f"❌ FAILED: Found string that should NOT be present:\n'''{n_ex}'''")
|
| 66 |
-
print(f"Rendered:\n{rendered}")
|
| 67 |
-
success = False
|
| 68 |
-
|
| 69 |
-
if success:
|
| 70 |
-
print("✅ PASSED")
|
| 71 |
-
return True
|
| 72 |
-
return False
|
| 73 |
-
|
| 74 |
-
except Exception as e:
|
| 75 |
-
if expect_error:
|
| 76 |
-
print(f"✅ PASSED (Caught expected error: {e})")
|
| 77 |
-
return True
|
| 78 |
-
print(f"❌ FAILED with exception:\n{traceback.format_exc()}")
|
| 79 |
-
return False
|
| 80 |
-
|
| 81 |
-
def run_prefix_test(name, messages, kwargs=None):
|
| 82 |
-
"""Asserts render(messages[:k]) is always a strict prefix of render(messages[:k+1]).
|
| 83 |
-
|
| 84 |
-
This is the direct verification of the 100% Prefix KV Cache claim: if any past turn
|
| 85 |
-
is mutated when a new turn arrives, the cached prefix is invalidated from that point.
|
| 86 |
-
"""
|
| 87 |
-
if kwargs is None:
|
| 88 |
-
kwargs = {}
|
| 89 |
-
|
| 90 |
-
print(f"\n--- Running Test: {name} ---")
|
| 91 |
-
|
| 92 |
-
previous = None
|
| 93 |
-
for k in range(1, len(messages) + 1):
|
| 94 |
-
# Checkpoint only at generation boundaries: prefixes splitting a merged
|
| 95 |
-
# system block or a consecutive tool-result batch are never rendered in
|
| 96 |
-
# real serving, so they are not required to be stable.
|
| 97 |
-
if k < len(messages) and messages[k].get('role') == messages[k - 1].get('role') \
|
| 98 |
-
and messages[k].get('role') in ('system', 'tool'):
|
| 99 |
-
continue
|
| 100 |
-
try:
|
| 101 |
-
current = template.render(messages=messages[:k], add_generation_prompt=False, **kwargs)
|
| 102 |
-
except Exception:
|
| 103 |
-
print(f"❌ FAILED with exception:\n{traceback.format_exc()}")
|
| 104 |
-
return False
|
| 105 |
-
|
| 106 |
-
if previous is not None and not current.startswith(previous):
|
| 107 |
-
idx = min(len(previous), len(current))
|
| 108 |
-
for i in range(min(len(previous), len(current))):
|
| 109 |
-
if previous[i] != current[i]:
|
| 110 |
-
idx = i
|
| 111 |
-
break
|
| 112 |
-
print(f"❌ FAILED: turn {k} mutated rendered history at char {idx}.")
|
| 113 |
-
print(f"Before: {previous[max(0, idx - 80):idx + 80]!r}")
|
| 114 |
-
print(f"After: {current[max(0, idx - 80):idx + 80]!r}")
|
| 115 |
-
return False
|
| 116 |
-
previous = current
|
| 117 |
-
|
| 118 |
-
print("✅ PASSED")
|
| 119 |
-
return True
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
def run_oneline_parity_test(name, cases):
|
| 123 |
-
"""Asserts chat_template_oneline.txt renders byte-identically to chat_template.jinja."""
|
| 124 |
-
print(f"\n--- Running Test: {name} ---")
|
| 125 |
-
|
| 126 |
-
oneline_path = os.path.join(TEMPLATE_DIR, 'chat_template_oneline.txt')
|
| 127 |
-
if not os.path.exists(oneline_path):
|
| 128 |
-
print(f"❌ FAILED: {oneline_path} not found.")
|
| 129 |
-
return False
|
| 130 |
-
|
| 131 |
-
with open(oneline_path, 'r', encoding='utf-8') as f:
|
| 132 |
-
oneline_source = f.read()
|
| 133 |
-
with open(os.path.join(TEMPLATE_DIR, TEMPLATE_FILE), 'r', encoding='utf-8') as f:
|
| 134 |
-
jinja_source = f.read()
|
| 135 |
-
|
| 136 |
-
version_pattern = r'template_version\s*=\s*["\']([^"\']+)["\']'
|
| 137 |
-
jinja_version = re.search(version_pattern, jinja_source)
|
| 138 |
-
oneline_version = re.search(version_pattern, oneline_source)
|
| 139 |
-
if not jinja_version or not oneline_version or jinja_version.group(1) != oneline_version.group(1):
|
| 140 |
-
print("❌ FAILED: template_version mismatch between jinja and oneline builds.")
|
| 141 |
-
print("Regenerate with: python3 scripts/minify_jinja.py chat_template.jinja chat_template_oneline.txt")
|
| 142 |
-
return False
|
| 143 |
-
|
| 144 |
-
oneline_template = env.from_string(oneline_source)
|
| 145 |
-
|
| 146 |
-
for label, messages, kwargs in cases:
|
| 147 |
-
try:
|
| 148 |
-
a = template.render(messages=messages, add_generation_prompt=True, **kwargs)
|
| 149 |
-
b = oneline_template.render(messages=messages, add_generation_prompt=True, **kwargs)
|
| 150 |
-
except Exception:
|
| 151 |
-
print(f"❌ FAILED with exception on case '{label}':\n{traceback.format_exc()}")
|
| 152 |
-
return False
|
| 153 |
-
if a != b:
|
| 154 |
-
print(f"❌ FAILED: oneline output diverges from jinja on case '{label}'.")
|
| 155 |
-
print("Regenerate with: python3 scripts/minify_jinja.py chat_template.jinja chat_template_oneline.txt")
|
| 156 |
-
return False
|
| 157 |
-
|
| 158 |
-
print("✅ PASSED")
|
| 159 |
-
return True
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
tests_passed = 0
|
| 163 |
-
tests_total = 0
|
| 164 |
-
|
| 165 |
-
def execute_test(*args, **kwargs):
|
| 166 |
-
global tests_passed, tests_total
|
| 167 |
-
tests_total += 1
|
| 168 |
-
if run_test(*args, **kwargs):
|
| 169 |
-
tests_passed += 1
|
| 170 |
-
|
| 171 |
-
def execute_prefix_test(*args, **kwargs):
|
| 172 |
-
global tests_passed, tests_total
|
| 173 |
-
tests_total += 1
|
| 174 |
-
if run_prefix_test(*args, **kwargs):
|
| 175 |
-
tests_passed += 1
|
| 176 |
-
|
| 177 |
-
def execute_parity_test(*args, **kwargs):
|
| 178 |
-
global tests_passed, tests_total
|
| 179 |
-
tests_total += 1
|
| 180 |
-
if run_oneline_parity_test(*args, **kwargs):
|
| 181 |
-
tests_passed += 1
|
| 182 |
-
|
| 183 |
-
# ==========================================
|
| 184 |
-
# 1. Qwen 3.8 Reasoning Effort Controls (v22.1 Default: medium)
|
| 185 |
-
# ==========================================
|
| 186 |
-
|
| 187 |
-
# 1. Default reasoning_effort="medium" (no system message -> zero system message emitted)
|
| 188 |
-
execute_test(
|
| 189 |
-
"1. reasoning_effort='medium' (v22.1 default, no system message)",
|
| 190 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 191 |
-
expected_in=[
|
| 192 |
-
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n"
|
| 193 |
-
],
|
| 194 |
-
expected_not_in=[
|
| 195 |
-
"<|im_start|>system\n"
|
| 196 |
-
]
|
| 197 |
-
)
|
| 198 |
-
|
| 199 |
-
# 2. Explicit reasoning_effort="xhigh"
|
| 200 |
-
execute_test(
|
| 201 |
-
"2. reasoning_effort='xhigh'",
|
| 202 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 203 |
-
kwargs={"reasoning_effort": "xhigh"},
|
| 204 |
-
expected_in=[
|
| 205 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 206 |
-
"<|im_start|>user\nHello!<|im_end|>\n",
|
| 207 |
-
"<|im_start|>assistant\n<think>\n"
|
| 208 |
-
]
|
| 209 |
-
)
|
| 210 |
-
|
| 211 |
-
# 3. Explicit reasoning_effort="high" (OpenAI alias -> xhigh)
|
| 212 |
-
execute_test(
|
| 213 |
-
"3. reasoning_effort='high' (OpenAI alias)",
|
| 214 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 215 |
-
kwargs={"reasoning_effort": "high"},
|
| 216 |
-
expected_in=[
|
| 217 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 218 |
-
]
|
| 219 |
-
)
|
| 220 |
-
|
| 221 |
-
# 4. Explicit reasoning_effort="max" (API max alias -> xhigh)
|
| 222 |
-
execute_test(
|
| 223 |
-
"4. reasoning_effort='max' (API alias)",
|
| 224 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 225 |
-
kwargs={"reasoning_effort": "max"},
|
| 226 |
-
expected_in=[
|
| 227 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 228 |
-
]
|
| 229 |
-
)
|
| 230 |
-
|
| 231 |
-
# 5. Explicit reasoning_effort="low"
|
| 232 |
-
execute_test(
|
| 233 |
-
"5. reasoning_effort='low'",
|
| 234 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 235 |
-
kwargs={"reasoning_effort": "low"},
|
| 236 |
-
expected_in=[
|
| 237 |
-
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n",
|
| 238 |
-
"<|im_start|>user\nHello!<|im_end|>\n"
|
| 239 |
-
]
|
| 240 |
-
)
|
| 241 |
-
|
| 242 |
-
# 6. Explicit reasoning_effort="minimal" (API minimal alias -> low)
|
| 243 |
-
execute_test(
|
| 244 |
-
"6. reasoning_effort='minimal' (API alias)",
|
| 245 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 246 |
-
kwargs={"reasoning_effort": "minimal"},
|
| 247 |
-
expected_in=[
|
| 248 |
-
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n"
|
| 249 |
-
]
|
| 250 |
-
)
|
| 251 |
-
|
| 252 |
-
# 7. Explicit reasoning_effort="none" (disables thinking)
|
| 253 |
-
execute_test(
|
| 254 |
-
"7. reasoning_effort='none' (disables thinking)",
|
| 255 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 256 |
-
kwargs={"reasoning_effort": "none"},
|
| 257 |
-
expected_in=[
|
| 258 |
-
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 259 |
-
],
|
| 260 |
-
expected_not_in=[
|
| 261 |
-
"Reasoning effort is set to"
|
| 262 |
-
]
|
| 263 |
-
)
|
| 264 |
-
|
| 265 |
-
# 8. Explicit reasoning_effort="unknown_val" (safe fallback to medium)
|
| 266 |
-
execute_test(
|
| 267 |
-
"8. reasoning_effort='unknown_val' (safe fallback to medium)",
|
| 268 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 269 |
-
kwargs={"reasoning_effort": "unrecognized_str"},
|
| 270 |
-
expected_in=[
|
| 271 |
-
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n"
|
| 272 |
-
],
|
| 273 |
-
expected_not_in=[
|
| 274 |
-
"Reasoning effort is set to"
|
| 275 |
-
]
|
| 276 |
-
)
|
| 277 |
-
|
| 278 |
-
# 9. reasoning_effort='xhigh' with user system prompt
|
| 279 |
-
execute_test(
|
| 280 |
-
"9. reasoning_effort='xhigh' with user system prompt",
|
| 281 |
-
messages=[
|
| 282 |
-
{"role": "system", "content": "You are an expert coder."},
|
| 283 |
-
{"role": "user", "content": "Write quicksort in C++"}
|
| 284 |
-
],
|
| 285 |
-
kwargs={"reasoning_effort": "xhigh"},
|
| 286 |
-
expected_in=[
|
| 287 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.\n\nYou are an expert coder.<|im_end|>\n",
|
| 288 |
-
"<|im_start|>user\nWrite quicksort in C++<|im_end|>\n"
|
| 289 |
-
]
|
| 290 |
-
)
|
| 291 |
-
|
| 292 |
-
# 10. reasoning_effort='xhigh' with tools
|
| 293 |
-
tools_sample = [
|
| 294 |
-
{
|
| 295 |
-
"type": "function",
|
| 296 |
-
"function": {
|
| 297 |
-
"name": "get_weather",
|
| 298 |
-
"description": "Get weather for city",
|
| 299 |
-
"parameters": {
|
| 300 |
-
"type": "object",
|
| 301 |
-
"properties": {"city": {"type": "string"}},
|
| 302 |
-
"required": ["city"]
|
| 303 |
-
}
|
| 304 |
-
}
|
| 305 |
-
}
|
| 306 |
-
]
|
| 307 |
-
|
| 308 |
-
execute_test(
|
| 309 |
-
"10. reasoning_effort='xhigh' with tools",
|
| 310 |
-
messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
|
| 311 |
-
tools=tools_sample,
|
| 312 |
-
kwargs={"reasoning_effort": "xhigh"},
|
| 313 |
-
expected_in=[
|
| 314 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.\n\n# Tools\n\nYou have access to the following functions:\n\n<tools>\n"
|
| 315 |
-
]
|
| 316 |
-
)
|
| 317 |
-
|
| 318 |
-
# ==========================================
|
| 319 |
-
# 2. Inline Chat Tags for Reasoning Effort Steering (v22.1)
|
| 320 |
-
# ==========================================
|
| 321 |
-
|
| 322 |
-
# 11. Inline <|think_low|> in user message
|
| 323 |
-
execute_test(
|
| 324 |
-
"11. Inline <|think_low|> in user string",
|
| 325 |
-
messages=[{"role": "user", "content": "What is 2+2? <|think_low|>"}],
|
| 326 |
-
expected_in=[
|
| 327 |
-
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n",
|
| 328 |
-
"<|im_start|>user\nWhat is 2+2?<|im_end|>\n",
|
| 329 |
-
"<|im_start|>assistant\n<think>\n"
|
| 330 |
-
],
|
| 331 |
-
expected_not_in=[
|
| 332 |
-
"<|think_low|>"
|
| 333 |
-
]
|
| 334 |
-
)
|
| 335 |
-
|
| 336 |
-
# 12. Inline <|think_xhigh|> in user message
|
| 337 |
-
execute_test(
|
| 338 |
-
"12. Inline <|think_xhigh|> in user string",
|
| 339 |
-
messages=[{"role": "user", "content": "Prove Fermat's Last Theorem <|think_xhigh|>"}],
|
| 340 |
-
expected_in=[
|
| 341 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 342 |
-
"<|im_start|>user\nProve Fermat's Last Theorem<|im_end|>\n",
|
| 343 |
-
"<|im_start|>assistant\n<think>\n"
|
| 344 |
-
],
|
| 345 |
-
expected_not_in=[
|
| 346 |
-
"<|think_xhigh|>"
|
| 347 |
-
]
|
| 348 |
-
)
|
| 349 |
-
|
| 350 |
-
# 13. Inline <|think_medium|> in user message
|
| 351 |
-
execute_test(
|
| 352 |
-
"13. Inline <|think_medium|> in user string",
|
| 353 |
-
messages=[{"role": "user", "content": "Hello <|think_medium|>"}],
|
| 354 |
-
expected_in=[
|
| 355 |
-
"<|im_start|>user\nHello<|im_end|>\n",
|
| 356 |
-
"<|im_start|>assistant\n<think>\n"
|
| 357 |
-
],
|
| 358 |
-
expected_not_in=[
|
| 359 |
-
"<|think_medium|>",
|
| 360 |
-
"<|im_start|>system\n"
|
| 361 |
-
]
|
| 362 |
-
)
|
| 363 |
-
|
| 364 |
-
# 14. Inline <|think_off|> in user message
|
| 365 |
-
execute_test(
|
| 366 |
-
"14. Inline <|think_off|> in user string",
|
| 367 |
-
messages=[{"role": "user", "content": "Quick answer: what is capital of France? <|think_off|>"}],
|
| 368 |
-
expected_in=[
|
| 369 |
-
"<|im_start|>user\nQuick answer: what is capital of France?<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 370 |
-
],
|
| 371 |
-
expected_not_in=[
|
| 372 |
-
"<|think_off|>",
|
| 373 |
-
"Reasoning effort is set to"
|
| 374 |
-
]
|
| 375 |
-
)
|
| 376 |
-
|
| 377 |
-
# 15. Inline <|think_low|> in multi-part list[dict]
|
| 378 |
-
execute_test(
|
| 379 |
-
"15. Inline <|think_low|> in multi-part list[dict]",
|
| 380 |
-
messages=[
|
| 381 |
-
{"role": "user", "content": [{"type": "text", "text": "Solve this riddle <|think_low|>"}]}
|
| 382 |
-
],
|
| 383 |
-
expected_in=[
|
| 384 |
-
"<|im_start|>system\nReasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.<|im_end|>\n",
|
| 385 |
-
"<|im_start|>user\nSolve this riddle<|im_end|>\n"
|
| 386 |
-
],
|
| 387 |
-
expected_not_in=[
|
| 388 |
-
"<|think_low|>"
|
| 389 |
-
]
|
| 390 |
-
)
|
| 391 |
-
|
| 392 |
-
# 16. Inline <|think_xhigh|> in multi-part list[str]
|
| 393 |
-
execute_test(
|
| 394 |
-
"16. Inline <|think_xhigh|> in multi-part list[str]",
|
| 395 |
-
messages=[
|
| 396 |
-
{"role": "user", "content": ["Solve this deeply", "<|think_xhigh|>"]}
|
| 397 |
-
],
|
| 398 |
-
expected_in=[
|
| 399 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 400 |
-
"<|im_start|>user\nSolve this deeply<|im_end|>\n"
|
| 401 |
-
],
|
| 402 |
-
expected_not_in=[
|
| 403 |
-
"<|think_xhigh|>"
|
| 404 |
-
]
|
| 405 |
-
)
|
| 406 |
-
|
| 407 |
-
# 17. Clean tag stripping across multiple tags in same string
|
| 408 |
-
execute_test(
|
| 409 |
-
"17. Clean tag stripping across multiple tags in same string",
|
| 410 |
-
messages=[
|
| 411 |
-
{"role": "user", "content": "Hello <|think_on|> <|think_minimal|> world"}
|
| 412 |
-
],
|
| 413 |
-
expected_in=[
|
| 414 |
-
"<|im_start|>user\nHello world<|im_end|>\n"
|
| 415 |
-
],
|
| 416 |
-
expected_not_in=[
|
| 417 |
-
"<|think_on|>",
|
| 418 |
-
"<|think_minimal|>"
|
| 419 |
-
]
|
| 420 |
-
)
|
| 421 |
-
|
| 422 |
-
# ==========================================
|
| 423 |
-
# 3. Thinking Toggles & Preserves
|
| 424 |
-
# ==========================================
|
| 425 |
-
|
| 426 |
-
# 18. enable_thinking=false kwarg
|
| 427 |
-
execute_test(
|
| 428 |
-
"18. enable_thinking=false kwarg",
|
| 429 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 430 |
-
kwargs={"enable_thinking": False},
|
| 431 |
-
expected_in=[
|
| 432 |
-
"<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 433 |
-
]
|
| 434 |
-
)
|
| 435 |
-
|
| 436 |
-
# 19. auto_disable_thinking_with_tools=true
|
| 437 |
-
execute_test(
|
| 438 |
-
"19. auto_disable_thinking_with_tools=true",
|
| 439 |
-
messages=[{"role": "user", "content": "What's the weather?"}],
|
| 440 |
-
tools=tools_sample,
|
| 441 |
-
kwargs={"auto_disable_thinking_with_tools": True},
|
| 442 |
-
expected_in=[
|
| 443 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 444 |
-
]
|
| 445 |
-
)
|
| 446 |
-
|
| 447 |
-
# 20. preserve_reasoning=True preserves thinking
|
| 448 |
-
execute_test(
|
| 449 |
-
"20. preserve_reasoning=True preserves thinking",
|
| 450 |
-
messages=[
|
| 451 |
-
{"role": "user", "content": "Question 1"},
|
| 452 |
-
{"role": "assistant", "content": "<think>\nThinking 1\n</think>\n\nAnswer 1"},
|
| 453 |
-
{"role": "user", "content": "Question 2"}
|
| 454 |
-
],
|
| 455 |
-
kwargs={"preserve_reasoning": True},
|
| 456 |
-
expected_in=[
|
| 457 |
-
"<|im_start|>assistant\n<think>\nThinking 1\n</think>\n\nAnswer 1<|im_end|>\n"
|
| 458 |
-
]
|
| 459 |
-
)
|
| 460 |
-
|
| 461 |
-
# 21. preserve_reasoning=False strips past thinking
|
| 462 |
-
execute_test(
|
| 463 |
-
"21. preserve_reasoning=False strips past thinking",
|
| 464 |
-
messages=[
|
| 465 |
-
{"role": "user", "content": "Question 1"},
|
| 466 |
-
{"role": "assistant", "content": "<think>\nThinking 1\n</think>\n\nAnswer 1"},
|
| 467 |
-
{"role": "user", "content": "Question 2"}
|
| 468 |
-
],
|
| 469 |
-
kwargs={"preserve_reasoning": False},
|
| 470 |
-
expected_in=[
|
| 471 |
-
"<|im_start|>assistant\nAnswer 1<|im_end|>\n"
|
| 472 |
-
],
|
| 473 |
-
expected_not_in=[
|
| 474 |
-
"Thinking 1"
|
| 475 |
-
]
|
| 476 |
-
)
|
| 477 |
-
|
| 478 |
-
# 22. In-content <think> parsing (Curing official 3.8 empty think poisoning)
|
| 479 |
-
execute_test(
|
| 480 |
-
"22. In-content <think> parsing (Curing official 3.8 empty think poisoning)",
|
| 481 |
-
messages=[
|
| 482 |
-
{"role": "user", "content": "Solve 1+1"},
|
| 483 |
-
{"role": "assistant", "content": "<think>\n1+1 is 2\n</think>\n\nResult is 2"},
|
| 484 |
-
{"role": "user", "content": "Now 2+2"}
|
| 485 |
-
],
|
| 486 |
-
kwargs={"preserve_thinking": True},
|
| 487 |
-
expected_in=[
|
| 488 |
-
"<|im_start|>assistant\n<think>\n1+1 is 2\n</think>\n\nResult is 2<|im_end|>\n"
|
| 489 |
-
],
|
| 490 |
-
expected_not_in=[
|
| 491 |
-
"<think>\n\n</think>\n\n<think>"
|
| 492 |
-
]
|
| 493 |
-
)
|
| 494 |
-
|
| 495 |
-
# 23. OpenAI reasoning_content field
|
| 496 |
-
execute_test(
|
| 497 |
-
"23. OpenAI reasoning_content field",
|
| 498 |
-
messages=[
|
| 499 |
-
{"role": "user", "content": "Question 1"},
|
| 500 |
-
{"role": "assistant", "content": "Answer 1", "reasoning_content": "Deep thought 1"},
|
| 501 |
-
{"role": "user", "content": "Question 2"}
|
| 502 |
-
],
|
| 503 |
-
kwargs={"preserve_thinking": True},
|
| 504 |
-
expected_in=[
|
| 505 |
-
"<|im_start|>assistant\n<think>\nDeep thought 1\n</think>\n\nAnswer 1<|im_end|>\n"
|
| 506 |
-
]
|
| 507 |
-
)
|
| 508 |
-
|
| 509 |
-
# 24. Anthropic message.thinking field
|
| 510 |
-
execute_test(
|
| 511 |
-
"24. Anthropic message.thinking field",
|
| 512 |
-
messages=[
|
| 513 |
-
{"role": "user", "content": "Question 1"},
|
| 514 |
-
{"role": "assistant", "content": "Answer 1", "thinking": "Anthropic thought 1"},
|
| 515 |
-
{"role": "user", "content": "Question 2"}
|
| 516 |
-
],
|
| 517 |
-
kwargs={"preserve_thinking": True},
|
| 518 |
-
expected_in=[
|
| 519 |
-
"<|im_start|>assistant\n<think>\nAnthropic thought 1\n</think>\n\nAnswer 1<|im_end|>\n"
|
| 520 |
-
]
|
| 521 |
-
)
|
| 522 |
-
|
| 523 |
-
# 24b. OpenAI reasoning field (vLLM / Responses API name)
|
| 524 |
-
execute_test(
|
| 525 |
-
"24b. OpenAI reasoning field (vLLM / Responses API name)",
|
| 526 |
-
messages=[
|
| 527 |
-
{"role": "user", "content": "Q1"},
|
| 528 |
-
{"role": "assistant", "content": "Answer 1", "reasoning": "Thought via reasoning field"},
|
| 529 |
-
{"role": "user", "content": "Q2"}
|
| 530 |
-
],
|
| 531 |
-
expected_in=["<|im_start|>assistant\n<think>\nThought via reasoning field\n</think>\n\nAnswer 1<|im_end|>"]
|
| 532 |
-
)
|
| 533 |
-
|
| 534 |
-
# 24c. _default_reasoning_effort knob steers the no-kwarg default
|
| 535 |
-
def run_default_effort_knob_test():
|
| 536 |
-
global tests_passed, tests_total
|
| 537 |
-
tests_total += 1
|
| 538 |
-
name = "24c. _default_reasoning_effort knob steers the no-kwarg default"
|
| 539 |
-
print(f"\n--- Running Test: {name} ---")
|
| 540 |
-
try:
|
| 541 |
-
src = open(os.path.join(TEMPLATE_DIR, TEMPLATE_FILE), encoding='utf-8').read()
|
| 542 |
-
assert src.count("_default_reasoning_effort = 'medium'") == 1, "knob line not found exactly once"
|
| 543 |
-
msgs = [{"role": "user", "content": "hi"}]
|
| 544 |
-
stock = env.from_string(src).render(messages=msgs, add_generation_prompt=True)
|
| 545 |
-
xhigh = env.from_string(src.replace("_default_reasoning_effort = 'medium'", "_default_reasoning_effort = 'xhigh'")).render(messages=msgs, add_generation_prompt=True)
|
| 546 |
-
off = env.from_string(src.replace("_default_reasoning_effort = 'medium'", "_default_reasoning_effort = 'none'")).render(messages=msgs, add_generation_prompt=True)
|
| 547 |
-
explicit = env.from_string(src.replace("_default_reasoning_effort = 'medium'", "_default_reasoning_effort = 'xhigh'")).render(messages=msgs, add_generation_prompt=True, reasoning_effort="low")
|
| 548 |
-
assert "Reasoning effort is set to" not in stock, "stock default must inject no steering line"
|
| 549 |
-
assert "Reasoning effort is set to xhigh" in xhigh, "xhigh knob must inject the xhigh line"
|
| 550 |
-
assert off.endswith("<think>\n\n</think>\n\n"), "none knob must disable thinking"
|
| 551 |
-
assert "Reasoning effort is set to low" in explicit, "an explicit kwarg must still override the knob"
|
| 552 |
-
print("✅ PASSED")
|
| 553 |
-
tests_passed += 1
|
| 554 |
-
except Exception as e:
|
| 555 |
-
print(f"❌ FAILED: {e}")
|
| 556 |
-
run_default_effort_knob_test()
|
| 557 |
-
|
| 558 |
-
# ==========================================
|
| 559 |
-
# 4. Tool Calling (XML & JSON)
|
| 560 |
-
# ==========================================
|
| 561 |
-
|
| 562 |
-
# 25. Tool calling with dict arguments (XML)
|
| 563 |
-
execute_test(
|
| 564 |
-
"25. Tool calling with dict arguments (XML)",
|
| 565 |
-
messages=[
|
| 566 |
-
{"role": "user", "content": "Weather in Paris?"},
|
| 567 |
-
{
|
| 568 |
-
"role": "assistant",
|
| 569 |
-
"content": "",
|
| 570 |
-
"tool_calls": [
|
| 571 |
-
{
|
| 572 |
-
"type": "function",
|
| 573 |
-
"function": {
|
| 574 |
-
"name": "get_weather",
|
| 575 |
-
"arguments": {"city": "Paris"}
|
| 576 |
-
}
|
| 577 |
-
}
|
| 578 |
-
]
|
| 579 |
-
}
|
| 580 |
-
],
|
| 581 |
-
expected_in=[
|
| 582 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=get_weather>\n<parameter=city>\nParis\n</parameter>\n</function>\n</tool_call><|im_end|>\n"
|
| 583 |
-
]
|
| 584 |
-
)
|
| 585 |
-
|
| 586 |
-
# 26. Tool calling with JSON string arguments (XML)
|
| 587 |
-
execute_test(
|
| 588 |
-
"26. Tool calling with JSON string arguments (XML)",
|
| 589 |
-
messages=[
|
| 590 |
-
{"role": "user", "content": "Weather in Paris?"},
|
| 591 |
-
{
|
| 592 |
-
"role": "assistant",
|
| 593 |
-
"content": "",
|
| 594 |
-
"tool_calls": [
|
| 595 |
-
{
|
| 596 |
-
"type": "function",
|
| 597 |
-
"function": {
|
| 598 |
-
"name": "get_weather",
|
| 599 |
-
"arguments": '{"city": "Paris"}'
|
| 600 |
-
}
|
| 601 |
-
}
|
| 602 |
-
]
|
| 603 |
-
}
|
| 604 |
-
],
|
| 605 |
-
kwargs={"tool_call_format": "xml"},
|
| 606 |
-
expected_in=[
|
| 607 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=get_weather>\n{\"city\": \"Paris\"}</function>\n</tool_call><|im_end|>\n"
|
| 608 |
-
]
|
| 609 |
-
)
|
| 610 |
-
|
| 611 |
-
# 27. Tool calling with dict arguments (JSON format)
|
| 612 |
-
execute_test(
|
| 613 |
-
"27. Tool calling with dict arguments (JSON format)",
|
| 614 |
-
messages=[
|
| 615 |
-
{"role": "user", "content": "Weather in Paris?"},
|
| 616 |
-
{
|
| 617 |
-
"role": "assistant",
|
| 618 |
-
"content": "",
|
| 619 |
-
"tool_calls": [
|
| 620 |
-
{
|
| 621 |
-
"type": "function",
|
| 622 |
-
"function": {
|
| 623 |
-
"name": "get_weather",
|
| 624 |
-
"arguments": {"city": "Paris"}
|
| 625 |
-
}
|
| 626 |
-
}
|
| 627 |
-
]
|
| 628 |
-
}
|
| 629 |
-
],
|
| 630 |
-
kwargs={"tool_call_format": "json"},
|
| 631 |
-
expected_in=[
|
| 632 |
-
'<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n{"name": "get_weather", "arguments": {"city": "Paris"}}\n</tool_call><|im_end|>\n'
|
| 633 |
-
]
|
| 634 |
-
)
|
| 635 |
-
|
| 636 |
-
# 28. Tool calling with JSON string arguments (JSON format)
|
| 637 |
-
execute_test(
|
| 638 |
-
"28. Tool calling with JSON string arguments (JSON format)",
|
| 639 |
-
messages=[
|
| 640 |
-
{"role": "user", "content": "Weather in Paris?"},
|
| 641 |
-
{
|
| 642 |
-
"role": "assistant",
|
| 643 |
-
"content": "",
|
| 644 |
-
"tool_calls": [
|
| 645 |
-
{
|
| 646 |
-
"type": "function",
|
| 647 |
-
"function": {
|
| 648 |
-
"name": "get_weather",
|
| 649 |
-
"arguments": '{"city": "Paris"}'
|
| 650 |
-
}
|
| 651 |
-
}
|
| 652 |
-
]
|
| 653 |
-
}
|
| 654 |
-
],
|
| 655 |
-
kwargs={"tool_call_format": "json"},
|
| 656 |
-
expected_in=[
|
| 657 |
-
'<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n{"name": "get_weather", "arguments": {"city": "Paris"}}\n</tool_call><|im_end|>\n'
|
| 658 |
-
]
|
| 659 |
-
)
|
| 660 |
-
|
| 661 |
-
# 29. Tool calling with empty arguments string
|
| 662 |
-
execute_test(
|
| 663 |
-
"29. Tool calling with empty arguments string",
|
| 664 |
-
messages=[
|
| 665 |
-
{"role": "user", "content": "Call tool without args"},
|
| 666 |
-
{
|
| 667 |
-
"role": "assistant",
|
| 668 |
-
"content": "",
|
| 669 |
-
"tool_calls": [
|
| 670 |
-
{
|
| 671 |
-
"type": "function",
|
| 672 |
-
"function": {
|
| 673 |
-
"name": "no_arg_tool",
|
| 674 |
-
"arguments": ""
|
| 675 |
-
}
|
| 676 |
-
}
|
| 677 |
-
]
|
| 678 |
-
}
|
| 679 |
-
],
|
| 680 |
-
expected_in=[
|
| 681 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=no_arg_tool>\n</function>\n</tool_call><|im_end|>\n"
|
| 682 |
-
]
|
| 683 |
-
)
|
| 684 |
-
|
| 685 |
-
# ==========================================
|
| 686 |
-
# 5. Payload Truncation & Error Escalation
|
| 687 |
-
# ==========================================
|
| 688 |
-
|
| 689 |
-
# 30. Dynamic parameter truncation (max_tool_arg_chars)
|
| 690 |
-
execute_test(
|
| 691 |
-
"30. Dynamic parameter truncation (max_tool_arg_chars)",
|
| 692 |
-
messages=[
|
| 693 |
-
{"role": "user", "content": "Execute SQL"},
|
| 694 |
-
{
|
| 695 |
-
"role": "assistant",
|
| 696 |
-
"content": "",
|
| 697 |
-
"tool_calls": [
|
| 698 |
-
{
|
| 699 |
-
"type": "function",
|
| 700 |
-
"function": {
|
| 701 |
-
"name": "run_sql",
|
| 702 |
-
"arguments": {"query": "SELECT * FROM users WHERE id = 1234567890 AND active = true"}
|
| 703 |
-
}
|
| 704 |
-
}
|
| 705 |
-
]
|
| 706 |
-
}
|
| 707 |
-
],
|
| 708 |
-
kwargs={"max_tool_arg_chars": 20},
|
| 709 |
-
expected_in=[
|
| 710 |
-
"[TRUNCATED - original length"
|
| 711 |
-
]
|
| 712 |
-
)
|
| 713 |
-
|
| 714 |
-
# 31. Dynamic response truncation (max_tool_response_chars)
|
| 715 |
-
execute_test(
|
| 716 |
-
"31. Dynamic response truncation (max_tool_response_chars)",
|
| 717 |
-
messages=[
|
| 718 |
-
{"role": "user", "content": "Search files"},
|
| 719 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "search", "arguments": {}}}]},
|
| 720 |
-
{"role": "tool", "content": "A" * 200}
|
| 721 |
-
],
|
| 722 |
-
kwargs={"max_tool_response_chars": 50},
|
| 723 |
-
expected_in=[
|
| 724 |
-
"[TRUNCATED - original length 200 chars]"
|
| 725 |
-
]
|
| 726 |
-
)
|
| 727 |
-
|
| 728 |
-
# 32. Consecutive tool error warning 1
|
| 729 |
-
execute_test(
|
| 730 |
-
"32. Consecutive tool error warning 1",
|
| 731 |
-
messages=[
|
| 732 |
-
{"role": "user", "content": "Run tool"},
|
| 733 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "run", "arguments": {}}}]},
|
| 734 |
-
{"role": "tool", "content": '{"error": "file not found"}'}
|
| 735 |
-
],
|
| 736 |
-
expected_in=[
|
| 737 |
-
"⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments."
|
| 738 |
-
]
|
| 739 |
-
)
|
| 740 |
-
|
| 741 |
-
# 33. Consecutive tool error warning 2 (retaining reasoning for error correction)
|
| 742 |
-
execute_test(
|
| 743 |
-
"33. Consecutive tool error warning 2 (retaining reasoning for error correction)",
|
| 744 |
-
messages=[
|
| 745 |
-
{"role": "user", "content": "Run tool"},
|
| 746 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "run", "arguments": {}}}]},
|
| 747 |
-
{"role": "tool", "content": '{"error": "file not found"}'},
|
| 748 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "run", "arguments": {}}}]},
|
| 749 |
-
{"role": "tool", "content": '{"error": "permission denied"}'}
|
| 750 |
-
],
|
| 751 |
-
expected_in=[
|
| 752 |
-
"⚠️ SYSTEM WARNING: 2 consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.",
|
| 753 |
-
"<|im_start|>assistant\n<think>\n"
|
| 754 |
-
]
|
| 755 |
-
)
|
| 756 |
-
|
| 757 |
-
# 34. Mid-conversation system & developer messages
|
| 758 |
-
execute_test(
|
| 759 |
-
"34. Mid-conversation system & developer messages",
|
| 760 |
-
messages=[
|
| 761 |
-
{"role": "user", "content": "Hello"},
|
| 762 |
-
{"role": "assistant", "content": "Hi"},
|
| 763 |
-
{"role": "developer", "content": "Mid-conversation update: user changed context."},
|
| 764 |
-
{"role": "user", "content": "Continue"}
|
| 765 |
-
],
|
| 766 |
-
expected_in=[
|
| 767 |
-
"<|im_start|>system\nMid-conversation update: user changed context.<|im_end|>\n",
|
| 768 |
-
"<|im_start|>user\nContinue<|im_end|>\n"
|
| 769 |
-
]
|
| 770 |
-
)
|
| 771 |
-
|
| 772 |
-
# ==========================================
|
| 773 |
-
# 3. v22.2 Enhancements & Community Fixes
|
| 774 |
-
# ==========================================
|
| 775 |
-
|
| 776 |
-
# 35. reasoning_effort='ultracode' (Discussion #78)
|
| 777 |
-
execute_test(
|
| 778 |
-
"35. reasoning_effort='ultracode'",
|
| 779 |
-
messages=[{"role": "user", "content": "Analyze algorithm"}],
|
| 780 |
-
kwargs={"reasoning_effort": "ultracode"},
|
| 781 |
-
expected_in=[
|
| 782 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 783 |
-
]
|
| 784 |
-
)
|
| 785 |
-
|
| 786 |
-
# 36. reasoning_effort='extreme'
|
| 787 |
-
execute_test(
|
| 788 |
-
"36. reasoning_effort='extreme'",
|
| 789 |
-
messages=[{"role": "user", "content": "Analyze algorithm"}],
|
| 790 |
-
kwargs={"reasoning_effort": "extreme"},
|
| 791 |
-
expected_in=[
|
| 792 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n"
|
| 793 |
-
]
|
| 794 |
-
)
|
| 795 |
-
|
| 796 |
-
# 37. Inline <|think_ultracode|> in user message
|
| 797 |
-
execute_test(
|
| 798 |
-
"37. Inline <|think_ultracode|> in user message",
|
| 799 |
-
messages=[{"role": "user", "content": "Optimize this shader <|think_ultracode|>"}],
|
| 800 |
-
expected_in=[
|
| 801 |
-
"<|im_start|>system\nReasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.<|im_end|>\n",
|
| 802 |
-
"<|im_start|>user\nOptimize this shader<|im_end|>\n"
|
| 803 |
-
],
|
| 804 |
-
expected_not_in=[
|
| 805 |
-
"<|think_ultracode|>"
|
| 806 |
-
]
|
| 807 |
-
)
|
| 808 |
-
|
| 809 |
-
# 38. Multiple leading system and developer messages merging
|
| 810 |
-
execute_test(
|
| 811 |
-
"38. Multiple leading system and developer messages merging",
|
| 812 |
-
messages=[
|
| 813 |
-
{"role": "system", "content": "Base instructions."},
|
| 814 |
-
{"role": "developer", "content": "Developer constraints."},
|
| 815 |
-
{"role": "system", "content": "Additional guidelines."},
|
| 816 |
-
{"role": "user", "content": "Hello"}
|
| 817 |
-
],
|
| 818 |
-
expected_in=[
|
| 819 |
-
"<|im_start|>system\nBase instructions.\n\nDeveloper constraints.\n\nAdditional guidelines.<|im_end|>\n",
|
| 820 |
-
"<|im_start|>user\nHello<|im_end|>\n"
|
| 821 |
-
]
|
| 822 |
-
)
|
| 823 |
-
|
| 824 |
-
# 39. Grep code search with throw Error not false-positiving on tool error (Discussion #66)
|
| 825 |
-
execute_test(
|
| 826 |
-
"39. Grep code search with throw Error not false-positiving on tool error (Discussion #66)",
|
| 827 |
-
messages=[
|
| 828 |
-
{"role": "user", "content": "Search for error handlers"},
|
| 829 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "grep", "arguments": {"pattern": "failed to"}}}]},
|
| 830 |
-
{"role": "tool", "content": "src/lib/api.ts:42: throw new Error('failed to fetch');"},
|
| 831 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "grep", "arguments": {"pattern": "failed to"}}}]},
|
| 832 |
-
{"role": "tool", "content": "src/lib/auth.ts:18: throw new Error('failed to authenticate');"}
|
| 833 |
-
],
|
| 834 |
-
expected_not_in=[
|
| 835 |
-
"⚠️ SYSTEM WARNING"
|
| 836 |
-
]
|
| 837 |
-
)
|
| 838 |
-
|
| 839 |
-
# 40. Grep code search with console.error not false-positiving
|
| 840 |
-
execute_test(
|
| 841 |
-
"40. Grep code search with console.error not false-positiving",
|
| 842 |
-
messages=[
|
| 843 |
-
{"role": "user", "content": "Search for error logging"},
|
| 844 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "grep", "arguments": {"pattern": "console.error"}}}]},
|
| 845 |
-
{"role": "tool", "content": "src/utils.js:5: console.error('failed to load config');"}
|
| 846 |
-
],
|
| 847 |
-
expected_not_in=[
|
| 848 |
-
"⚠️ SYSTEM WARNING"
|
| 849 |
-
]
|
| 850 |
-
)
|
| 851 |
-
|
| 852 |
-
# 41. Python Traceback legitimately triggers tool error warning
|
| 853 |
-
execute_test(
|
| 854 |
-
"41. Python Traceback legitimately triggers tool error warning",
|
| 855 |
-
messages=[
|
| 856 |
-
{"role": "user", "content": "Run script"},
|
| 857 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "python", "arguments": {"code": "1/0"}}}]},
|
| 858 |
-
{"role": "tool", "content": "Traceback (most recent call last):\n File 'test.py', line 1\nZeroDivisionError: division by zero"}
|
| 859 |
-
],
|
| 860 |
-
expected_in=[
|
| 861 |
-
"⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments."
|
| 862 |
-
]
|
| 863 |
-
)
|
| 864 |
-
|
| 865 |
-
# 42. Safe XML parameter formatting for booleans, nulls, and numbers
|
| 866 |
-
execute_test(
|
| 867 |
-
"42. Safe XML parameter formatting for booleans, nulls, and numbers",
|
| 868 |
-
messages=[
|
| 869 |
-
{"role": "user", "content": "Call config tool"},
|
| 870 |
-
{"role": "assistant", "content": "", "tool_calls": [{
|
| 871 |
-
"type": "function",
|
| 872 |
-
"function": {
|
| 873 |
-
"name": "set_config",
|
| 874 |
-
"arguments": {
|
| 875 |
-
"is_active": True,
|
| 876 |
-
"is_debug": False,
|
| 877 |
-
"retries": 5,
|
| 878 |
-
"metadata": None
|
| 879 |
-
}
|
| 880 |
-
}
|
| 881 |
-
}]}
|
| 882 |
-
],
|
| 883 |
-
expected_in=[
|
| 884 |
-
"<parameter=is_active>\ntrue\n</parameter>",
|
| 885 |
-
"<parameter=is_debug>\nfalse\n</parameter>",
|
| 886 |
-
"<parameter=retries>\n5\n</parameter>",
|
| 887 |
-
"<parameter=metadata>\nnull\n</parameter>"
|
| 888 |
-
]
|
| 889 |
-
)
|
| 890 |
-
|
| 891 |
-
# 43. KV Cache Stability: History with JSON string arguments preserves canonical XML system prompt
|
| 892 |
-
execute_test(
|
| 893 |
-
"43. KV Cache Stability: History with JSON string arguments preserves canonical XML system prompt",
|
| 894 |
-
messages=[
|
| 895 |
-
{"role": "user", "content": "What's the weather?"},
|
| 896 |
-
{"role": "assistant", "content": "", "tool_calls": [{
|
| 897 |
-
"type": "function",
|
| 898 |
-
"function": {
|
| 899 |
-
"name": "get_weather",
|
| 900 |
-
"arguments": '{"city": "Paris", "unit": "celsius"}'
|
| 901 |
-
}
|
| 902 |
-
}]},
|
| 903 |
-
{"role": "tool", "content": '{"temp": 22}'}
|
| 904 |
-
],
|
| 905 |
-
tools=tools_sample,
|
| 906 |
-
expected_in=[
|
| 907 |
-
"<function=example_function_name>",
|
| 908 |
-
"<tool_call>\n<function=get_weather>\n{\"city\": \"Paris\", \"unit\": \"celsius\"}</function>\n</tool_call>"
|
| 909 |
-
]
|
| 910 |
-
)
|
| 911 |
-
|
| 912 |
-
# ==========================================
|
| 913 |
-
# 6. Control Tag & Alias Completeness (v22.3)
|
| 914 |
-
# ==========================================
|
| 915 |
-
|
| 916 |
-
# 45. Inline <|think_high|>
|
| 917 |
-
execute_test(
|
| 918 |
-
"45. Inline <|think_high|>",
|
| 919 |
-
messages=[{"role": "user", "content": "Analyze this <|think_high|>"}],
|
| 920 |
-
expected_in=[
|
| 921 |
-
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 922 |
-
],
|
| 923 |
-
expected_not_in=[
|
| 924 |
-
"<|think_high|>"
|
| 925 |
-
]
|
| 926 |
-
)
|
| 927 |
-
|
| 928 |
-
# 46. Inline <|think_extreme|>
|
| 929 |
-
execute_test(
|
| 930 |
-
"46. Inline <|think_extreme|>",
|
| 931 |
-
messages=[{"role": "user", "content": "Analyze this <|think_extreme|>"}],
|
| 932 |
-
expected_in=[
|
| 933 |
-
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 934 |
-
],
|
| 935 |
-
expected_not_in=[
|
| 936 |
-
"<|think_extreme|>"
|
| 937 |
-
]
|
| 938 |
-
)
|
| 939 |
-
|
| 940 |
-
# 47. Inline <|think_max|>
|
| 941 |
-
execute_test(
|
| 942 |
-
"47. Inline <|think_max|>",
|
| 943 |
-
messages=[{"role": "user", "content": "Analyze this <|think_max|>"}],
|
| 944 |
-
expected_in=[
|
| 945 |
-
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 946 |
-
],
|
| 947 |
-
expected_not_in=[
|
| 948 |
-
"<|think_max|>"
|
| 949 |
-
]
|
| 950 |
-
)
|
| 951 |
-
|
| 952 |
-
# 48. Inline <|think_minimal|> injects low reasoning instructions
|
| 953 |
-
execute_test(
|
| 954 |
-
"48. Inline <|think_minimal|> injects low reasoning instructions",
|
| 955 |
-
messages=[{"role": "user", "content": "Answer fast <|think_minimal|>"}],
|
| 956 |
-
expected_in=[
|
| 957 |
-
"<|im_start|>system\nReasoning effort is set to low."
|
| 958 |
-
],
|
| 959 |
-
expected_not_in=[
|
| 960 |
-
"<|think_minimal|>"
|
| 961 |
-
]
|
| 962 |
-
)
|
| 963 |
-
|
| 964 |
-
# 49. Inline <|think_on|> overrides enable_thinking=False
|
| 965 |
-
execute_test(
|
| 966 |
-
"49. Inline <|think_on|> overrides enable_thinking=False",
|
| 967 |
-
messages=[{"role": "user", "content": "Think about this <|think_on|>"}],
|
| 968 |
-
kwargs={"enable_thinking": False},
|
| 969 |
-
expected_in=[
|
| 970 |
-
"<|im_start|>assistant\n<think>\n"
|
| 971 |
-
],
|
| 972 |
-
expected_not_in=[
|
| 973 |
-
"<think>\n\n</think>",
|
| 974 |
-
"<|think_on|>"
|
| 975 |
-
]
|
| 976 |
-
)
|
| 977 |
-
|
| 978 |
-
# 50. reasoning_effort is case-insensitive
|
| 979 |
-
execute_test(
|
| 980 |
-
"50. reasoning_effort is case-insensitive",
|
| 981 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 982 |
-
kwargs={"reasoning_effort": "XHigh"},
|
| 983 |
-
expected_in=[
|
| 984 |
-
"<|im_start|>system\nReasoning effort is set to xhigh."
|
| 985 |
-
]
|
| 986 |
-
)
|
| 987 |
-
|
| 988 |
-
# 51. reasoning_effort='off' disables thinking
|
| 989 |
-
execute_test(
|
| 990 |
-
"51. reasoning_effort='off' disables thinking",
|
| 991 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 992 |
-
kwargs={"reasoning_effort": "off"},
|
| 993 |
-
expected_in=[
|
| 994 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 995 |
-
],
|
| 996 |
-
expected_not_in=[
|
| 997 |
-
"Reasoning effort is set to"
|
| 998 |
-
]
|
| 999 |
-
)
|
| 1000 |
-
|
| 1001 |
-
# 52. reasoning_effort=None falls back to medium
|
| 1002 |
-
execute_test(
|
| 1003 |
-
"52. reasoning_effort=None falls back to medium",
|
| 1004 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 1005 |
-
kwargs={"reasoning_effort": None},
|
| 1006 |
-
expected_in=[
|
| 1007 |
-
"<|im_start|>assistant\n<think>\n"
|
| 1008 |
-
],
|
| 1009 |
-
expected_not_in=[
|
| 1010 |
-
"Reasoning effort is set to"
|
| 1011 |
-
]
|
| 1012 |
-
)
|
| 1013 |
-
|
| 1014 |
-
# 53. Non-string reasoning_effort does not crash
|
| 1015 |
-
execute_test(
|
| 1016 |
-
"53. Non-string reasoning_effort does not crash",
|
| 1017 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 1018 |
-
kwargs={"reasoning_effort": 3},
|
| 1019 |
-
expected_in=[
|
| 1020 |
-
"<|im_start|>assistant\n<think>\n"
|
| 1021 |
-
],
|
| 1022 |
-
expected_not_in=[
|
| 1023 |
-
"Reasoning effort is set to"
|
| 1024 |
-
]
|
| 1025 |
-
)
|
| 1026 |
-
|
| 1027 |
-
# 54. Control tag inside a system message is honored and stripped
|
| 1028 |
-
execute_test(
|
| 1029 |
-
"54. Control tag inside a system message is honored and stripped",
|
| 1030 |
-
messages=[
|
| 1031 |
-
{"role": "system", "content": "You are terse. <|think_low|>"},
|
| 1032 |
-
{"role": "user", "content": "Hello"}
|
| 1033 |
-
],
|
| 1034 |
-
expected_in=[
|
| 1035 |
-
"Reasoning effort is set to low.",
|
| 1036 |
-
"You are terse."
|
| 1037 |
-
],
|
| 1038 |
-
expected_not_in=[
|
| 1039 |
-
"<|think_low|>"
|
| 1040 |
-
]
|
| 1041 |
-
)
|
| 1042 |
-
|
| 1043 |
-
# 55. Most recent control tag wins across turns
|
| 1044 |
-
execute_test(
|
| 1045 |
-
"55. Most recent control tag wins across turns",
|
| 1046 |
-
messages=[
|
| 1047 |
-
{"role": "user", "content": "First <|think_xhigh|>"},
|
| 1048 |
-
{"role": "assistant", "content": "Ok"},
|
| 1049 |
-
{"role": "user", "content": "Second <|think_low|>"}
|
| 1050 |
-
],
|
| 1051 |
-
expected_in=[
|
| 1052 |
-
"Reasoning effort is set to low."
|
| 1053 |
-
],
|
| 1054 |
-
expected_not_in=[
|
| 1055 |
-
"Reasoning effort is set to xhigh."
|
| 1056 |
-
]
|
| 1057 |
-
)
|
| 1058 |
-
|
| 1059 |
-
# 56. enable_thinking=False suppresses reasoning effort injection
|
| 1060 |
-
execute_test(
|
| 1061 |
-
"56. enable_thinking=False suppresses reasoning effort injection",
|
| 1062 |
-
messages=[{"role": "user", "content": "Hello!"}],
|
| 1063 |
-
kwargs={"enable_thinking": False, "reasoning_effort": "xhigh"},
|
| 1064 |
-
expected_in=[
|
| 1065 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 1066 |
-
],
|
| 1067 |
-
expected_not_in=[
|
| 1068 |
-
"Reasoning effort is set to"
|
| 1069 |
-
]
|
| 1070 |
-
)
|
| 1071 |
-
|
| 1072 |
-
# 57. <|think_off|> takes precedence over an effort tag in the same message
|
| 1073 |
-
execute_test(
|
| 1074 |
-
"57. <|think_off|> takes precedence over an effort tag in the same message",
|
| 1075 |
-
messages=[{"role": "user", "content": "Quick <|think_xhigh|> <|think_off|>"}],
|
| 1076 |
-
expected_in=[
|
| 1077 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n"
|
| 1078 |
-
],
|
| 1079 |
-
expected_not_in=[
|
| 1080 |
-
"Reasoning effort is set to",
|
| 1081 |
-
"<|think_"
|
| 1082 |
-
]
|
| 1083 |
-
)
|
| 1084 |
-
|
| 1085 |
-
# ==========================================
|
| 1086 |
-
# 7. Malformed Input & Exception Paths (v22.3)
|
| 1087 |
-
# ==========================================
|
| 1088 |
-
|
| 1089 |
-
# 58. Empty message list raises
|
| 1090 |
-
execute_test(
|
| 1091 |
-
"58. Empty message list raises",
|
| 1092 |
-
messages=[],
|
| 1093 |
-
expect_error=True
|
| 1094 |
-
)
|
| 1095 |
-
|
| 1096 |
-
# 59. Image inside a leading system message raises
|
| 1097 |
-
execute_test(
|
| 1098 |
-
"59. Image inside a leading system message raises",
|
| 1099 |
-
messages=[
|
| 1100 |
-
{"role": "system", "content": [{"type": "image", "image": "data"}]},
|
| 1101 |
-
{"role": "user", "content": "Hello"}
|
| 1102 |
-
],
|
| 1103 |
-
expect_error=True
|
| 1104 |
-
)
|
| 1105 |
-
|
| 1106 |
-
# 60. Scalar (non-string, non-list) content raises
|
| 1107 |
-
execute_test(
|
| 1108 |
-
"60. Scalar content raises",
|
| 1109 |
-
messages=[{"role": "user", "content": 12345}],
|
| 1110 |
-
expect_error=True
|
| 1111 |
-
)
|
| 1112 |
-
|
| 1113 |
-
# 61. Unknown multi-part item type raises
|
| 1114 |
-
execute_test(
|
| 1115 |
-
"61. Unknown multi-part item type raises",
|
| 1116 |
-
messages=[{"role": "user", "content": [{"type": "audio", "audio": "data"}]}],
|
| 1117 |
-
expect_error=True
|
| 1118 |
-
)
|
| 1119 |
-
|
| 1120 |
-
# ==========================================
|
| 1121 |
-
# 8. Vision & Multi-Part Content (v22.3)
|
| 1122 |
-
# ==========================================
|
| 1123 |
-
|
| 1124 |
-
# 62. Image part renders vision tokens inline with text
|
| 1125 |
-
execute_test(
|
| 1126 |
-
"62. Image part renders vision tokens inline with text",
|
| 1127 |
-
messages=[{"role": "user", "content": [
|
| 1128 |
-
{"type": "image", "image": "data"},
|
| 1129 |
-
{"type": "text", "text": "What is this?"}
|
| 1130 |
-
]}],
|
| 1131 |
-
expected_in=[
|
| 1132 |
-
"<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>What is this?<|im_end|>\n"
|
| 1133 |
-
]
|
| 1134 |
-
)
|
| 1135 |
-
|
| 1136 |
-
# 63. add_vision_id numbers images across turns
|
| 1137 |
-
execute_test(
|
| 1138 |
-
"63. add_vision_id numbers images across turns",
|
| 1139 |
-
messages=[
|
| 1140 |
-
{"role": "user", "content": [{"type": "image", "image": "a"}]},
|
| 1141 |
-
{"role": "assistant", "content": "Ok"},
|
| 1142 |
-
{"role": "user", "content": [{"type": "image", "image": "b"}]}
|
| 1143 |
-
],
|
| 1144 |
-
kwargs={"add_vision_id": True},
|
| 1145 |
-
expected_in=[
|
| 1146 |
-
"Picture 1: <|vision_start|><|image_pad|><|vision_end|>",
|
| 1147 |
-
"Picture 2: <|vision_start|><|image_pad|><|vision_end|>"
|
| 1148 |
-
]
|
| 1149 |
-
)
|
| 1150 |
-
|
| 1151 |
-
# 64. Video part renders video tokens
|
| 1152 |
-
execute_test(
|
| 1153 |
-
"64. Video part renders video tokens",
|
| 1154 |
-
messages=[{"role": "user", "content": [{"type": "video", "video": "data"}]}],
|
| 1155 |
-
expected_in=[
|
| 1156 |
-
"<|vision_start|><|video_pad|><|vision_end|>"
|
| 1157 |
-
]
|
| 1158 |
-
)
|
| 1159 |
-
|
| 1160 |
-
# ==========================================
|
| 1161 |
-
# 9. Agentic Structure & Wire Format Shapes (v22.3)
|
| 1162 |
-
# ==========================================
|
| 1163 |
-
|
| 1164 |
-
# 65. Assistant text combined with a tool call
|
| 1165 |
-
execute_test(
|
| 1166 |
-
"65. Assistant text combined with a tool call",
|
| 1167 |
-
messages=[
|
| 1168 |
-
{"role": "user", "content": "Weather?"},
|
| 1169 |
-
{"role": "assistant", "content": "Let me check.", "tool_calls": [
|
| 1170 |
-
{"type": "function", "function": {"name": "get_weather", "arguments": {"city": "Paris"}}}
|
| 1171 |
-
]}
|
| 1172 |
-
],
|
| 1173 |
-
expected_in=[
|
| 1174 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\nLet me check.\n\n<tool_call>\n<function=get_weather>\n"
|
| 1175 |
-
]
|
| 1176 |
-
)
|
| 1177 |
-
|
| 1178 |
-
# 66. Parallel tool calls in a single assistant message
|
| 1179 |
-
execute_test(
|
| 1180 |
-
"66. Parallel tool calls in a single assistant message",
|
| 1181 |
-
messages=[
|
| 1182 |
-
{"role": "user", "content": "Compare"},
|
| 1183 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1184 |
-
{"type": "function", "function": {"name": "f1", "arguments": {"a": "1"}}},
|
| 1185 |
-
{"type": "function", "function": {"name": "f2", "arguments": {"b": "2"}}}
|
| 1186 |
-
]}
|
| 1187 |
-
],
|
| 1188 |
-
expected_in=[
|
| 1189 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n<tool_call>\n<function=f1>\n",
|
| 1190 |
-
"</tool_call>\n<tool_call>\n<function=f2>\n"
|
| 1191 |
-
]
|
| 1192 |
-
)
|
| 1193 |
-
|
| 1194 |
-
# 67. Consecutive tool results collapse into a single user turn
|
| 1195 |
-
execute_test(
|
| 1196 |
-
"67. Consecutive tool results collapse into a single user turn",
|
| 1197 |
-
messages=[
|
| 1198 |
-
{"role": "user", "content": "Compare"},
|
| 1199 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1200 |
-
{"type": "function", "function": {"name": "f1", "arguments": {}}},
|
| 1201 |
-
{"type": "function", "function": {"name": "f2", "arguments": {}}}
|
| 1202 |
-
]},
|
| 1203 |
-
{"role": "tool", "content": "result one"},
|
| 1204 |
-
{"role": "tool", "content": "result two"},
|
| 1205 |
-
{"role": "user", "content": "Thanks"}
|
| 1206 |
-
],
|
| 1207 |
-
expected_in=[
|
| 1208 |
-
"<|im_start|>user\n<tool_response>\nresult one\n</tool_response>\n<tool_response>\nresult two\n</tool_response><|im_end|>\n"
|
| 1209 |
-
]
|
| 1210 |
-
)
|
| 1211 |
-
|
| 1212 |
-
# 68. Flat tool_call shape without a 'function' wrapper
|
| 1213 |
-
execute_test(
|
| 1214 |
-
"68. Flat tool_call shape without a 'function' wrapper",
|
| 1215 |
-
messages=[
|
| 1216 |
-
{"role": "user", "content": "Weather?"},
|
| 1217 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1218 |
-
{"name": "get_weather", "arguments": {"city": "Paris"}}
|
| 1219 |
-
]}
|
| 1220 |
-
],
|
| 1221 |
-
expected_in=[
|
| 1222 |
-
"<tool_call>\n<function=get_weather>\n<parameter=city>\nParis\n</parameter>\n</function>\n</tool_call>"
|
| 1223 |
-
]
|
| 1224 |
-
)
|
| 1225 |
-
|
| 1226 |
-
# 69. OpenAI tool_call id/index fields are tolerated
|
| 1227 |
-
execute_test(
|
| 1228 |
-
"69. OpenAI tool_call id/index fields are tolerated",
|
| 1229 |
-
messages=[
|
| 1230 |
-
{"role": "user", "content": "Weather?"},
|
| 1231 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1232 |
-
{"id": "call_abc", "index": 0, "type": "function",
|
| 1233 |
-
"function": {"name": "get_weather", "arguments": {"city": "Paris"}}}
|
| 1234 |
-
]},
|
| 1235 |
-
{"role": "tool", "tool_call_id": "call_abc", "content": "22C"}
|
| 1236 |
-
],
|
| 1237 |
-
expected_in=[
|
| 1238 |
-
"<function=get_weather>",
|
| 1239 |
-
"<tool_response>\n22C\n</tool_response>"
|
| 1240 |
-
]
|
| 1241 |
-
)
|
| 1242 |
-
|
| 1243 |
-
# 70. Unknown roles fall back to a labelled user turn
|
| 1244 |
-
execute_test(
|
| 1245 |
-
"70. Unknown roles fall back to a labelled user turn",
|
| 1246 |
-
messages=[
|
| 1247 |
-
{"role": "user", "content": "Hello"},
|
| 1248 |
-
{"role": "critic", "content": "Needs work"}
|
| 1249 |
-
],
|
| 1250 |
-
expected_in=[
|
| 1251 |
-
"<|im_start|>user\n[critic]: Needs work<|im_end|>\n"
|
| 1252 |
-
]
|
| 1253 |
-
)
|
| 1254 |
-
|
| 1255 |
-
# 71. add_generation_prompt=False emits no assistant header
|
| 1256 |
-
execute_test(
|
| 1257 |
-
"71. add_generation_prompt=False emits no assistant header",
|
| 1258 |
-
messages=[{"role": "user", "content": "Hello"}],
|
| 1259 |
-
kwargs={"add_generation_prompt": False},
|
| 1260 |
-
expected_in=[
|
| 1261 |
-
"<|im_start|>user\nHello<|im_end|>\n"
|
| 1262 |
-
],
|
| 1263 |
-
expected_not_in=[
|
| 1264 |
-
"<|im_start|>assistant"
|
| 1265 |
-
]
|
| 1266 |
-
)
|
| 1267 |
-
|
| 1268 |
-
# 72. Empty tools list emits no tool system block
|
| 1269 |
-
execute_test(
|
| 1270 |
-
"72. Empty tools list emits no tool system block",
|
| 1271 |
-
messages=[{"role": "user", "content": "Hello"}],
|
| 1272 |
-
tools=[],
|
| 1273 |
-
expected_not_in=[
|
| 1274 |
-
"# Tools"
|
| 1275 |
-
]
|
| 1276 |
-
)
|
| 1277 |
-
|
| 1278 |
-
# 73. Assistant content=None does not crash
|
| 1279 |
-
execute_test(
|
| 1280 |
-
"73. Assistant content=None does not crash",
|
| 1281 |
-
messages=[
|
| 1282 |
-
{"role": "user", "content": "Hi"},
|
| 1283 |
-
{"role": "assistant", "content": None},
|
| 1284 |
-
{"role": "user", "content": "Again"}
|
| 1285 |
-
],
|
| 1286 |
-
expected_in=[
|
| 1287 |
-
"<|im_start|>assistant\n<think>\n\n</think>\n\n<|im_end|>\n"
|
| 1288 |
-
]
|
| 1289 |
-
)
|
| 1290 |
-
|
| 1291 |
-
# 74. Scalar tool arguments are serialized, not dropped
|
| 1292 |
-
execute_test(
|
| 1293 |
-
"74. Scalar tool arguments are serialized, not dropped",
|
| 1294 |
-
messages=[
|
| 1295 |
-
{"role": "user", "content": "Call"},
|
| 1296 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1297 |
-
{"type": "function", "function": {"name": "f", "arguments": 42}}
|
| 1298 |
-
]}
|
| 1299 |
-
],
|
| 1300 |
-
expected_in=[
|
| 1301 |
-
"<tool_call>\n<function=f>\n42</function>\n</tool_call>"
|
| 1302 |
-
]
|
| 1303 |
-
)
|
| 1304 |
-
|
| 1305 |
-
# ==========================================
|
| 1306 |
-
# 10. Reasoning Extraction Edge Cases (v22.3)
|
| 1307 |
-
# ==========================================
|
| 1308 |
-
|
| 1309 |
-
# 75. <thinking> variant is extracted and normalized
|
| 1310 |
-
execute_test(
|
| 1311 |
-
"75. <thinking> variant is extracted and normalized",
|
| 1312 |
-
messages=[
|
| 1313 |
-
{"role": "user", "content": "Q"},
|
| 1314 |
-
{"role": "assistant", "content": "<thinking>\nT\n</thinking>\n\nA"},
|
| 1315 |
-
{"role": "user", "content": "Q2"}
|
| 1316 |
-
],
|
| 1317 |
-
expected_in=[
|
| 1318 |
-
"<|im_start|>assistant\n<think>\nT\n</think>\n\nA<|im_end|>\n"
|
| 1319 |
-
]
|
| 1320 |
-
)
|
| 1321 |
-
|
| 1322 |
-
# 76. Spaced </think > variant is extracted
|
| 1323 |
-
execute_test(
|
| 1324 |
-
"76. Spaced </think > variant is extracted",
|
| 1325 |
-
messages=[
|
| 1326 |
-
{"role": "user", "content": "Q"},
|
| 1327 |
-
{"role": "assistant", "content": "<think>\nT\n</think >\n\nA"},
|
| 1328 |
-
{"role": "user", "content": "Q2"}
|
| 1329 |
-
],
|
| 1330 |
-
expected_in=[
|
| 1331 |
-
"<|im_start|>assistant\n<think>\nT\n</think>\n\nA<|im_end|>\n"
|
| 1332 |
-
]
|
| 1333 |
-
)
|
| 1334 |
-
|
| 1335 |
-
# 77. Single-line think block is extracted
|
| 1336 |
-
execute_test(
|
| 1337 |
-
"77. Single-line think block is extracted",
|
| 1338 |
-
messages=[
|
| 1339 |
-
{"role": "user", "content": "Q"},
|
| 1340 |
-
{"role": "assistant", "content": "<think>T</think>A"},
|
| 1341 |
-
{"role": "user", "content": "Q2"}
|
| 1342 |
-
],
|
| 1343 |
-
expected_in=[
|
| 1344 |
-
"<|im_start|>assistant\n<think>\nT\n</think>\n\nA<|im_end|>\n"
|
| 1345 |
-
]
|
| 1346 |
-
)
|
| 1347 |
-
|
| 1348 |
-
# 78. reasoning_content plus in-content tags must not duplicate think blocks
|
| 1349 |
-
execute_test(
|
| 1350 |
-
"78. reasoning_content plus in-content tags must not duplicate think blocks",
|
| 1351 |
-
messages=[
|
| 1352 |
-
{"role": "user", "content": "Q"},
|
| 1353 |
-
{"role": "assistant", "reasoning_content": "R", "content": "<think>\nT\n</think>\n\nA"},
|
| 1354 |
-
{"role": "user", "content": "Q2"}
|
| 1355 |
-
],
|
| 1356 |
-
expected_in=[
|
| 1357 |
-
"<|im_start|>assistant\n<think>\nR\n</think>\n\nA<|im_end|>\n"
|
| 1358 |
-
],
|
| 1359 |
-
expected_not_in=[
|
| 1360 |
-
"</think>\n\n<think>"
|
| 1361 |
-
]
|
| 1362 |
-
)
|
| 1363 |
-
|
| 1364 |
-
# 79. preserve_thinking=False retains reasoning inside the active tool loop
|
| 1365 |
-
execute_test(
|
| 1366 |
-
"79. preserve_thinking=False retains reasoning inside the active tool loop",
|
| 1367 |
-
messages=[
|
| 1368 |
-
{"role": "user", "content": "Fix it"},
|
| 1369 |
-
{"role": "assistant", "content": "<think>\nplan the fix\n</think>\n\n", "tool_calls": [
|
| 1370 |
-
{"type": "function", "function": {"name": "run", "arguments": {}}}
|
| 1371 |
-
]},
|
| 1372 |
-
{"role": "tool", "content": "output"}
|
| 1373 |
-
],
|
| 1374 |
-
kwargs={"preserve_thinking": False},
|
| 1375 |
-
expected_in=[
|
| 1376 |
-
"plan the fix"
|
| 1377 |
-
]
|
| 1378 |
-
)
|
| 1379 |
-
|
| 1380 |
-
# 80. preserve_thinking=False strips reasoning once a new user turn starts
|
| 1381 |
-
execute_test(
|
| 1382 |
-
"80. preserve_thinking=False strips reasoning once a new user turn starts",
|
| 1383 |
-
messages=[
|
| 1384 |
-
{"role": "user", "content": "Fix it"},
|
| 1385 |
-
{"role": "assistant", "content": "<think>\nplan the fix\n</think>\n\n", "tool_calls": [
|
| 1386 |
-
{"type": "function", "function": {"name": "run", "arguments": {}}}
|
| 1387 |
-
]},
|
| 1388 |
-
{"role": "tool", "content": "output"},
|
| 1389 |
-
{"role": "assistant", "content": "<think>\nwrap up\n</think>\n\nDone."},
|
| 1390 |
-
{"role": "user", "content": "Thanks"}
|
| 1391 |
-
],
|
| 1392 |
-
kwargs={"preserve_thinking": False},
|
| 1393 |
-
expected_not_in=[
|
| 1394 |
-
"plan the fix",
|
| 1395 |
-
"wrap up"
|
| 1396 |
-
]
|
| 1397 |
-
)
|
| 1398 |
-
|
| 1399 |
-
# 81. Explicit reasoning preserves literal think tags inside the final answer
|
| 1400 |
-
execute_test(
|
| 1401 |
-
"81. Explicit reasoning preserves literal think tags inside the final answer",
|
| 1402 |
-
messages=[
|
| 1403 |
-
{"role": "user", "content": "How do I close the think block?"},
|
| 1404 |
-
{"role": "assistant", "reasoning_content": "R",
|
| 1405 |
-
"content": "Use this closing tag:\n```\n</think>\n```\nDone."},
|
| 1406 |
-
{"role": "user", "content": "thanks"}
|
| 1407 |
-
],
|
| 1408 |
-
expected_in=[
|
| 1409 |
-
"<|im_start|>assistant\n<think>\nR\n</think>\n\nUse this closing tag:\n```\n</think>\n```\nDone.<|im_end|>\n"
|
| 1410 |
-
]
|
| 1411 |
-
)
|
| 1412 |
-
|
| 1413 |
-
# ==========================================
|
| 1414 |
-
# 11. Tool Error Detection Precision (v22.3)
|
| 1415 |
-
# ==========================================
|
| 1416 |
-
|
| 1417 |
-
# 82. Structural error signal fires regardless of payload length
|
| 1418 |
-
execute_test(
|
| 1419 |
-
"82. Structural error signal fires regardless of payload length",
|
| 1420 |
-
messages=[
|
| 1421 |
-
{"role": "user", "content": "Run"},
|
| 1422 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1423 |
-
{"role": "tool", "content": '{"error": "boom"}\n' + "detail line\n" * 80}
|
| 1424 |
-
],
|
| 1425 |
-
expected_in=[
|
| 1426 |
-
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1427 |
-
]
|
| 1428 |
-
)
|
| 1429 |
-
|
| 1430 |
-
# 83. Long traceback beyond the weak-signal length gate still fires
|
| 1431 |
-
execute_test(
|
| 1432 |
-
"83. Long traceback beyond the weak-signal length gate still fires",
|
| 1433 |
-
messages=[
|
| 1434 |
-
{"role": "user", "content": "Run"},
|
| 1435 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1436 |
-
{"role": "tool", "content": "Traceback (most recent call last):\n" +
|
| 1437 |
-
' File "/app/handler.py", line 118, in process\n result = self.client.fetch(payload)\n' * 6 +
|
| 1438 |
-
"ConnectionResetError: [Errno 104] Connection reset by peer"}
|
| 1439 |
-
],
|
| 1440 |
-
expected_in=[
|
| 1441 |
-
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1442 |
-
]
|
| 1443 |
-
)
|
| 1444 |
-
|
| 1445 |
-
# 84. Shell-echoed command output still reports a real failure
|
| 1446 |
-
execute_test(
|
| 1447 |
-
"84. Shell-echoed command output still reports a real failure",
|
| 1448 |
-
messages=[
|
| 1449 |
-
{"role": "user", "content": "Build"},
|
| 1450 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "bash", "arguments": {}}}]},
|
| 1451 |
-
{"role": "tool", "content": "$ npm run build\nError: command not found"}
|
| 1452 |
-
],
|
| 1453 |
-
expected_in=[
|
| 1454 |
-
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1455 |
-
]
|
| 1456 |
-
)
|
| 1457 |
-
|
| 1458 |
-
# 85. Timing metadata does not suppress a structural error
|
| 1459 |
-
execute_test(
|
| 1460 |
-
"85. Timing metadata does not suppress a structural error",
|
| 1461 |
-
messages=[
|
| 1462 |
-
{"role": "user", "content": "Query"},
|
| 1463 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "db", "arguments": {}}}]},
|
| 1464 |
-
{"role": "tool", "content": '{"error": "db timeout", "note": "took 3ms"}'}
|
| 1465 |
-
],
|
| 1466 |
-
expected_in=[
|
| 1467 |
-
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1468 |
-
]
|
| 1469 |
-
)
|
| 1470 |
-
|
| 1471 |
-
# 86. Successful exit code report does not trigger an error warning
|
| 1472 |
-
execute_test(
|
| 1473 |
-
"86. Successful exit code report does not trigger an error warning",
|
| 1474 |
-
messages=[
|
| 1475 |
-
{"role": "user", "content": "Run"},
|
| 1476 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "bash", "arguments": {}}}]},
|
| 1477 |
-
{"role": "tool", "content": "Command completed successfully.\nExit code: 0\n" + "log line\n" * 40}
|
| 1478 |
-
],
|
| 1479 |
-
expected_not_in=[
|
| 1480 |
-
"⚠️ SYSTEM WARNING"
|
| 1481 |
-
]
|
| 1482 |
-
)
|
| 1483 |
-
|
| 1484 |
-
# 87. Nonzero exit code escalates regardless of payload length
|
| 1485 |
-
execute_test(
|
| 1486 |
-
"87. Nonzero exit code escalates regardless of payload length",
|
| 1487 |
-
messages=[
|
| 1488 |
-
{"role": "user", "content": "Run"},
|
| 1489 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "bash", "arguments": {}}}]},
|
| 1490 |
-
{"role": "tool", "content": "Exit code: 1\n" + "stack frame\n" * 60}
|
| 1491 |
-
],
|
| 1492 |
-
expected_in=[
|
| 1493 |
-
"⚠️ SYSTEM WARNING: The previous tool call returned an error."
|
| 1494 |
-
]
|
| 1495 |
-
)
|
| 1496 |
-
|
| 1497 |
-
# 88. Failure counter resets after a successful call
|
| 1498 |
-
execute_test(
|
| 1499 |
-
"88. Failure counter resets after a successful call",
|
| 1500 |
-
messages=[
|
| 1501 |
-
{"role": "user", "content": "Run"},
|
| 1502 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1503 |
-
{"role": "tool", "content": '{"error": "e1"}'},
|
| 1504 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1505 |
-
{"role": "tool", "content": '{"ok": true}'},
|
| 1506 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1507 |
-
{"role": "tool", "content": '{"error": "e2"}'}
|
| 1508 |
-
],
|
| 1509 |
-
expected_not_in=[
|
| 1510 |
-
"2 consecutive tool errors"
|
| 1511 |
-
]
|
| 1512 |
-
)
|
| 1513 |
-
|
| 1514 |
-
# 89. Failure counter resets on a new user turn
|
| 1515 |
-
execute_test(
|
| 1516 |
-
"89. Failure counter resets on a new user turn",
|
| 1517 |
-
messages=[
|
| 1518 |
-
{"role": "user", "content": "Run"},
|
| 1519 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1520 |
-
{"role": "tool", "content": '{"error": "e1"}'},
|
| 1521 |
-
{"role": "user", "content": "Try again"},
|
| 1522 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1523 |
-
{"role": "tool", "content": '{"error": "e2"}'}
|
| 1524 |
-
],
|
| 1525 |
-
expected_not_in=[
|
| 1526 |
-
"2 consecutive tool errors"
|
| 1527 |
-
]
|
| 1528 |
-
)
|
| 1529 |
-
|
| 1530 |
-
# 90. Tool response truncation is bypassed in JSON format
|
| 1531 |
-
execute_test(
|
| 1532 |
-
"90. Tool response truncation is bypassed in JSON format",
|
| 1533 |
-
messages=[
|
| 1534 |
-
{"role": "user", "content": "Query"},
|
| 1535 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "f", "arguments": {}}}]},
|
| 1536 |
-
{"role": "tool", "content": json.dumps({"rows": ["x"] * 200})}
|
| 1537 |
-
],
|
| 1538 |
-
kwargs={"tool_call_format": "json", "max_tool_response_chars": 50},
|
| 1539 |
-
expected_not_in=[
|
| 1540 |
-
"[TRUNCATED"
|
| 1541 |
-
]
|
| 1542 |
-
)
|
| 1543 |
-
|
| 1544 |
-
# 91. max_tool_arg_chars applies to serialized JSON string arguments
|
| 1545 |
-
execute_test(
|
| 1546 |
-
"91. max_tool_arg_chars applies to serialized JSON string arguments",
|
| 1547 |
-
messages=[
|
| 1548 |
-
{"role": "user", "content": "Query"},
|
| 1549 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1550 |
-
{"type": "function", "function": {"name": "f", "arguments": json.dumps({"q": "S" * 400})}}
|
| 1551 |
-
]}
|
| 1552 |
-
],
|
| 1553 |
-
kwargs={"max_tool_arg_chars": 20},
|
| 1554 |
-
expected_in=[
|
| 1555 |
-
"[TRUNCATED - original length"
|
| 1556 |
-
]
|
| 1557 |
-
)
|
| 1558 |
-
|
| 1559 |
-
# ==========================================
|
| 1560 |
-
# 12. KV Cache Prefix Stability & Build Parity (v22.3)
|
| 1561 |
-
# ==========================================
|
| 1562 |
-
|
| 1563 |
-
agentic_session = [
|
| 1564 |
-
{"role": "system", "content": "You are a coding agent."},
|
| 1565 |
-
{"role": "user", "content": "Fix the build"},
|
| 1566 |
-
{"role": "assistant", "content": "<think>\ninspect the logs\n</think>\n\n", "tool_calls": [
|
| 1567 |
-
{"type": "function", "function": {"name": "run", "arguments": {"cmd": "make"}}}
|
| 1568 |
-
]},
|
| 1569 |
-
{"role": "tool", "content": "Error: missing header"},
|
| 1570 |
-
{"role": "assistant", "content": "<think>\nadd the include\n</think>\n\nAdding it now.", "tool_calls": [
|
| 1571 |
-
{"type": "function", "function": {"name": "edit", "arguments": {"file": "main.c"}}}
|
| 1572 |
-
]},
|
| 1573 |
-
{"role": "tool", "content": "ok"},
|
| 1574 |
-
{"role": "assistant", "content": "<think>\nverify\n</think>\n\nBuild fixed."},
|
| 1575 |
-
{"role": "user", "content": "Thanks"},
|
| 1576 |
-
]
|
| 1577 |
-
|
| 1578 |
-
# 92. Prefix KV cache stability across a full agentic session (default settings)
|
| 1579 |
-
execute_prefix_test(
|
| 1580 |
-
"92. Prefix KV cache stability across a full agentic session (default settings)",
|
| 1581 |
-
messages=agentic_session
|
| 1582 |
-
)
|
| 1583 |
-
|
| 1584 |
-
# 93. Prefix KV cache stability with tools and xhigh reasoning
|
| 1585 |
-
execute_prefix_test(
|
| 1586 |
-
"93. Prefix KV cache stability with tools and xhigh reasoning",
|
| 1587 |
-
messages=agentic_session,
|
| 1588 |
-
kwargs={"tools": tools_sample, "reasoning_effort": "xhigh"}
|
| 1589 |
-
)
|
| 1590 |
-
|
| 1591 |
-
# 94. chat_template_oneline.txt renders identically to chat_template.jinja
|
| 1592 |
-
execute_parity_test(
|
| 1593 |
-
"94. chat_template_oneline.txt renders identically to chat_template.jinja",
|
| 1594 |
-
cases=[
|
| 1595 |
-
("plain", [{"role": "user", "content": "Hello"}], {}),
|
| 1596 |
-
("system + tools", [
|
| 1597 |
-
{"role": "system", "content": "You are helpful."},
|
| 1598 |
-
{"role": "user", "content": "Weather?"}
|
| 1599 |
-
], {"tools": tools_sample}),
|
| 1600 |
-
("agentic session", agentic_session, {}),
|
| 1601 |
-
("json format", [
|
| 1602 |
-
{"role": "user", "content": "Call"},
|
| 1603 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1604 |
-
{"type": "function", "function": {"name": "f", "arguments": {"a": "1"}}}
|
| 1605 |
-
]}
|
| 1606 |
-
], {"tool_call_format": "json"}),
|
| 1607 |
-
]
|
| 1608 |
-
)
|
| 1609 |
-
|
| 1610 |
-
# ==========================================
|
| 1611 |
-
# 13. Success Envelopes & Input Shapes (v22.3)
|
| 1612 |
-
# ==========================================
|
| 1613 |
-
|
| 1614 |
-
# 95. JSON success envelope with error:null does not trigger a warning
|
| 1615 |
-
execute_test(
|
| 1616 |
-
"95. JSON success envelope with error:null does not trigger a warning",
|
| 1617 |
-
messages=[
|
| 1618 |
-
{"role": "user", "content": "Query"},
|
| 1619 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "api", "arguments": {}}}]},
|
| 1620 |
-
{"role": "tool", "content": '{"error": null, "data": {"rows": [1, 2, 3]}}\n' + "x" * 700}
|
| 1621 |
-
],
|
| 1622 |
-
expected_not_in=[
|
| 1623 |
-
"⚠️ SYSTEM WARNING"
|
| 1624 |
-
]
|
| 1625 |
-
)
|
| 1626 |
-
|
| 1627 |
-
# 96. JSON success envelope with error:false does not trigger a warning
|
| 1628 |
-
execute_test(
|
| 1629 |
-
"96. JSON success envelope with error:false does not trigger a warning",
|
| 1630 |
-
messages=[
|
| 1631 |
-
{"role": "user", "content": "Query"},
|
| 1632 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "api", "arguments": {}}}]},
|
| 1633 |
-
{"role": "tool", "content": '{"error": false, "result": "done"}'}
|
| 1634 |
-
],
|
| 1635 |
-
expected_not_in=[
|
| 1636 |
-
"⚠️ SYSTEM WARNING"
|
| 1637 |
-
]
|
| 1638 |
-
)
|
| 1639 |
-
|
| 1640 |
-
# 97. Inline effort tag overrides the reasoning_effort kwarg
|
| 1641 |
-
execute_test(
|
| 1642 |
-
"97. Inline effort tag overrides the reasoning_effort kwarg",
|
| 1643 |
-
messages=[{"role": "user", "content": "Answer fast <|think_low|>"}],
|
| 1644 |
-
kwargs={"reasoning_effort": "xhigh"},
|
| 1645 |
-
expected_in=[
|
| 1646 |
-
"Reasoning effort is set to low."
|
| 1647 |
-
],
|
| 1648 |
-
expected_not_in=[
|
| 1649 |
-
"Reasoning effort is set to xhigh."
|
| 1650 |
-
]
|
| 1651 |
-
)
|
| 1652 |
-
|
| 1653 |
-
# 98. Tool result with multipart text content is flattened
|
| 1654 |
-
execute_test(
|
| 1655 |
-
"98. Tool result with multipart text content is flattened",
|
| 1656 |
-
messages=[
|
| 1657 |
-
{"role": "user", "content": "Query"},
|
| 1658 |
-
{"role": "assistant", "content": "", "tool_calls": [{"type": "function", "function": {"name": "api", "arguments": {}}}]},
|
| 1659 |
-
{"role": "tool", "content": [{"type": "text", "text": "RESULT_42"}]}
|
| 1660 |
-
],
|
| 1661 |
-
expected_in=[
|
| 1662 |
-
"<tool_response>\nRESULT_42\n</tool_response>"
|
| 1663 |
-
]
|
| 1664 |
-
)
|
| 1665 |
-
|
| 1666 |
-
# 99. Argument exactly at max_tool_arg_chars is not truncated
|
| 1667 |
-
execute_test(
|
| 1668 |
-
"99. Argument exactly at max_tool_arg_chars is not truncated",
|
| 1669 |
-
messages=[
|
| 1670 |
-
{"role": "user", "content": "Query"},
|
| 1671 |
-
{"role": "assistant", "content": "", "tool_calls": [
|
| 1672 |
-
{"type": "function", "function": {"name": "f", "arguments": {"k": "A" * 40}}}]}
|
| 1673 |
-
],
|
| 1674 |
-
kwargs={"max_tool_arg_chars": 40},
|
| 1675 |
-
expected_in=[
|
| 1676 |
-
"A" * 40
|
| 1677 |
-
],
|
| 1678 |
-
expected_not_in=[
|
| 1679 |
-
"[TRUNCATED"
|
| 1680 |
-
]
|
| 1681 |
-
)
|
| 1682 |
-
|
| 1683 |
-
# 100. Multiple images in a single message get sequential vision ids
|
| 1684 |
-
execute_test(
|
| 1685 |
-
"100. Multiple images in a single message get sequential vision ids",
|
| 1686 |
-
messages=[{"role": "user", "content": [
|
| 1687 |
-
{"type": "image", "image": "a"},
|
| 1688 |
-
{"type": "image", "image": "b"},
|
| 1689 |
-
{"type": "text", "text": "compare these"}
|
| 1690 |
-
]}],
|
| 1691 |
-
kwargs={"add_vision_id": True},
|
| 1692 |
-
expected_in=[
|
| 1693 |
-
"Picture 1: <|vision_start|><|image_pad|><|vision_end|>",
|
| 1694 |
-
"Picture 2: <|vision_start|><|image_pad|><|vision_end|>"
|
| 1695 |
-
]
|
| 1696 |
-
)
|
| 1697 |
-
|
| 1698 |
-
# ==========================================
|
| 1699 |
-
# 14. Property-Based Fuzzing (v22.3)
|
| 1700 |
-
# ==========================================
|
| 1701 |
-
|
| 1702 |
-
def run_fuzz_property_test(name, cases, seed):
|
| 1703 |
-
print(f"\n--- Running Test: {name} ---")
|
| 1704 |
-
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
| 1705 |
-
try:
|
| 1706 |
-
import fuzz_template
|
| 1707 |
-
failures = fuzz_template.run_fuzz(cases=cases, seed=seed,
|
| 1708 |
-
template_dir=TEMPLATE_DIR,
|
| 1709 |
-
template_file=TEMPLATE_FILE)
|
| 1710 |
-
except Exception:
|
| 1711 |
-
print(f"❌ FAILED with exception:\n{traceback.format_exc()}")
|
| 1712 |
-
return False
|
| 1713 |
-
if failures:
|
| 1714 |
-
first = failures[0]
|
| 1715 |
-
print(f"❌ FAILED: {len(failures)} invariant violation(s); "
|
| 1716 |
-
f"first: [{first['invariant']}] {first['detail']}")
|
| 1717 |
-
print(f"Repro (seed {seed}, case {first['case']}): {first['repro'][:600]}")
|
| 1718 |
-
return False
|
| 1719 |
-
print(f"✅ PASSED ({cases} generated conversations, seed {seed})")
|
| 1720 |
-
return True
|
| 1721 |
-
|
| 1722 |
-
def execute_fuzz_property_test(*args, **kwargs):
|
| 1723 |
-
global tests_passed, tests_total
|
| 1724 |
-
tests_total += 1
|
| 1725 |
-
if run_fuzz_property_test(*args, **kwargs):
|
| 1726 |
-
tests_passed += 1
|
| 1727 |
-
|
| 1728 |
-
# 101. Property fuzz: generated conversations uphold all structural invariants
|
| 1729 |
-
execute_fuzz_property_test(
|
| 1730 |
-
"101. Property fuzz: generated conversations uphold all structural invariants",
|
| 1731 |
-
cases=300,
|
| 1732 |
-
seed=20260820
|
| 1733 |
-
)
|
| 1734 |
-
|
| 1735 |
-
print("\n==========================================")
|
| 1736 |
-
print(f"Results: {tests_passed} / {tests_total} tests passed ({tests_passed/tests_total*100:.1f}%)")
|
| 1737 |
-
print("==========================================")
|
| 1738 |
-
|
| 1739 |
-
if tests_passed != tests_total:
|
| 1740 |
-
sys.exit(1)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|