Instructions to use froggeric/Qwen-Fixed-Chat-Templates with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use froggeric/Qwen-Fixed-Chat-Templates with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Qwen-Fixed-Chat-Templates froggeric/Qwen-Fixed-Chat-Templates
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Bump version to v21 and archive v20
Browse files- README.md +10 -3
- archive/README-v20.md +239 -0
- archive/v20_chat_template.jinja +287 -0
- archive/v20_chat_template_oneline.txt +1 -0
- scripts/{test_v20.py → test_v21.py} +0 -0
README.md
CHANGED
|
@@ -14,9 +14,16 @@ tags:
|
|
| 14 |
- thinking
|
| 15 |
---
|
| 16 |
|
| 17 |
-
# Fixed jinja chat templates for Qwen 3.5 & 3.6 (
|
| 18 |
|
| 19 |
<details open>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
<summary><b>Update History & Changelog (v20)</b></summary>
|
| 21 |
|
| 22 |
> **2026-06-05 Update (v20): The Architect Patch.** A monumental structural overhaul targeting deep agentic loops and C++ inference engine compatibility. (1) **Minja AST Flattening:** Dramatically optimized Jinja nesting depths to resolve severe parsing bottlenecks that were dropping inference throughput by 80% on `llama.cpp`. (2) **Minja Replace Bug Fix (Hotfix):** Bypassed a severe C++ parsing bug in `llama.cpp` where using the `replace` filter at index 0 of a user prompt silently dropped the entire text payload. Inline thinking toggles now use `split` and `join` for robust stripping. (3) **Auto-disable Thinking:** Introduced `auto_disable_thinking_with_tools` kwarg (default `false`) that allows users to instantly shut off reasoning blocks during tool use. (4) **Deep Agent Fallbacks:** Resolved exceptions triggered by mid-conversation system prompts or loops lacking human `user` messages. (5) **Payload Truncation:** Implemented `max_tool_arg_chars` and `max_tool_response_chars` configurations to definitively stop context-window explosions from massive data returns. *(Huge thanks to `barubary` / `spiritbuun` for their contributions to these C++ architecture optimizations!)*
|
|
@@ -181,7 +188,7 @@ When a tool call fails validation repeatedly, the model can enter a degenerate r
|
|
| 181 |
### 5. Smart False-Positive Detection (v18)
|
| 182 |
Instead of broad substring matching that triggers false retry-loops on successful database returns containing words like "error", v18 utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell-echo exclusions (`$ `).
|
| 183 |
|
| 184 |
-
### 6. minijinja Compatibility Constraints (
|
| 185 |
Python-only Jinja2 features crash or misbehave on `minijinja`/`minja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 186 |
- `content | replace('<|think_on|>', '')` -> `content.split('<|think_on|>') | join('')` (Fixes a severe bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 187 |
- `\| items` -> `for key in mapping`
|
|
@@ -219,7 +226,7 @@ Python-only Jinja2 features crash or misbehave on `minijinja`/`minja` (the C++ r
|
|
| 219 |
## Running the test suite
|
| 220 |
|
| 221 |
```bash
|
| 222 |
-
python3 scripts/
|
| 223 |
```
|
| 224 |
|
| 225 |
Tests cover: `auto_disable_thinking_with_tools`, payload truncation logic, parallel tool spacing, mid-conversation system rendering, deep agent loop fallback, XML tool format, `<|think_off|>` / `<|think_on|>` inline overrides, and all legacy v19 regression tests.
|
|
|
|
| 14 |
- thinking
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# Fixed jinja chat templates for Qwen 3.5 & 3.6 (v21)
|
| 18 |
|
| 19 |
<details open>
|
| 20 |
+
<summary><b>Update History & Changelog (v21)</b></summary>
|
| 21 |
+
|
| 22 |
+
> **2026-07-02 Update (v21): Reliability & Tooling Overhaul.** Addressed several critical bugs and compatibility issues, particularly around tool calling and prefix cache efficiency. (1) **Tool Call XML -> Hermes JSON:** Switched tool-call output to the native JSON format to restore compatibility with `llama.cpp` and `ik_llama`. (2) **Prefix Cache Fixes:** Restored the `preserve_thinking` default to `true` and removed extraneous newlines that broke caching. (3) **Prompt Injection Guard:** `<|think_off|>` tags in untrusted tool responses are now correctly ignored. (4) **Quoted Tag Bug Fix:** The template no longer corrupts history when the assistant quotes `</think>`. (5) **Anthropic `message.thinking`:** Added support for Anthropic-style reasoning content. (6) **False-Positive Tool Errors:** Reduced error detection scope to the first 80 characters of a response to prevent false positives. *(Huge thanks to `Moore2877` and `choongng` for their excellent contributions!)*
|
| 23 |
+
|
| 24 |
+
</details>
|
| 25 |
+
|
| 26 |
+
<details>
|
| 27 |
<summary><b>Update History & Changelog (v20)</b></summary>
|
| 28 |
|
| 29 |
> **2026-06-05 Update (v20): The Architect Patch.** A monumental structural overhaul targeting deep agentic loops and C++ inference engine compatibility. (1) **Minja AST Flattening:** Dramatically optimized Jinja nesting depths to resolve severe parsing bottlenecks that were dropping inference throughput by 80% on `llama.cpp`. (2) **Minja Replace Bug Fix (Hotfix):** Bypassed a severe C++ parsing bug in `llama.cpp` where using the `replace` filter at index 0 of a user prompt silently dropped the entire text payload. Inline thinking toggles now use `split` and `join` for robust stripping. (3) **Auto-disable Thinking:** Introduced `auto_disable_thinking_with_tools` kwarg (default `false`) that allows users to instantly shut off reasoning blocks during tool use. (4) **Deep Agent Fallbacks:** Resolved exceptions triggered by mid-conversation system prompts or loops lacking human `user` messages. (5) **Payload Truncation:** Implemented `max_tool_arg_chars` and `max_tool_response_chars` configurations to definitively stop context-window explosions from massive data returns. *(Huge thanks to `barubary` / `spiritbuun` for their contributions to these C++ architecture optimizations!)*
|
|
|
|
| 188 |
### 5. Smart False-Positive Detection (v18)
|
| 189 |
Instead of broad substring matching that triggers false retry-loops on successful database returns containing words like "error", v18 utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell-echo exclusions (`$ `).
|
| 190 |
|
| 191 |
+
### 6. minijinja Compatibility Constraints (v20/v21)
|
| 192 |
Python-only Jinja2 features crash or misbehave on `minijinja`/`minja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 193 |
- `content | replace('<|think_on|>', '')` -> `content.split('<|think_on|>') | join('')` (Fixes a severe bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 194 |
- `\| items` -> `for key in mapping`
|
|
|
|
| 226 |
## Running the test suite
|
| 227 |
|
| 228 |
```bash
|
| 229 |
+
python3 scripts/test_v21.py
|
| 230 |
```
|
| 231 |
|
| 232 |
Tests cover: `auto_disable_thinking_with_tools`, payload truncation logic, parallel tool spacing, mid-conversation system rendering, deep agent loop fallback, XML tool format, `<|think_off|>` / `<|think_on|>` inline overrides, and all legacy v19 regression tests.
|
archive/README-v20.md
ADDED
|
@@ -0,0 +1,239 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- jinja
|
| 5 |
+
- chat-template
|
| 6 |
+
- qwen
|
| 7 |
+
- qwen3.5
|
| 8 |
+
- qwen3.6
|
| 9 |
+
- lm-studio
|
| 10 |
+
- mlx
|
| 11 |
+
- llama.cpp
|
| 12 |
+
- vllm
|
| 13 |
+
- tool-calling
|
| 14 |
+
- thinking
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# Fixed jinja chat templates for Qwen 3.5 & 3.6 (v20)
|
| 18 |
+
|
| 19 |
+
<details open>
|
| 20 |
+
<summary><b>Update History & Changelog (v20)</b></summary>
|
| 21 |
+
|
| 22 |
+
> **2026-06-05 Update (v20): The Architect Patch.** A monumental structural overhaul targeting deep agentic loops and C++ inference engine compatibility. (1) **Minja AST Flattening:** Dramatically optimized Jinja nesting depths to resolve severe parsing bottlenecks that were dropping inference throughput by 80% on `llama.cpp`. (2) **Minja Replace Bug Fix (Hotfix):** Bypassed a severe C++ parsing bug in `llama.cpp` where using the `replace` filter at index 0 of a user prompt silently dropped the entire text payload. Inline thinking toggles now use `split` and `join` for robust stripping. (3) **Auto-disable Thinking:** Introduced `auto_disable_thinking_with_tools` kwarg (default `false`) that allows users to instantly shut off reasoning blocks during tool use. (4) **Deep Agent Fallbacks:** Resolved exceptions triggered by mid-conversation system prompts or loops lacking human `user` messages. (5) **Payload Truncation:** Implemented `max_tool_arg_chars` and `max_tool_response_chars` configurations to definitively stop context-window explosions from massive data returns. *(Huge thanks to `barubary` / `spiritbuun` for their contributions to these C++ architecture optimizations!)*
|
| 23 |
+
|
| 24 |
+
</details>
|
| 25 |
+
|
| 26 |
+
<details>
|
| 27 |
+
<summary><b>Update History & Changelog (v19)</b></summary>
|
| 28 |
+
|
| 29 |
+
> **2026-05-18 Update (v19): The Agentic Loop Cure.** (1) **Abolished "Empty Think" Poisoning:** Rewrote the AST history rendering to completely remove the injection of empty `<think>\n</think>` blocks. This cures a severe in-context learning bias where the model assumed tools could only be called if it didn't think first, which was causing 80%+ of premature `<|im_end|>` turn aborts. (2) **System Prompt Logic Trap Removed:** Softened the absolute tool mandate in the `<IMPORTANT>` block and restored Universal Synthesis instructions. The model is now explicitly permitted to transition from `</think>` to a conversational answer without panicking. (3) **True 100% KV Cache & Amnesia Fix:** `preserve_thinking` now defaults to `true`. Past thoughts are retained chronologically, permanently curing "amnesia stalls" during multi-step tool loops while mathematically guaranteeing 100% KV Cache prefix matching out-of-the-box.
|
| 30 |
+
|
| 31 |
+
</details>
|
| 32 |
+
|
| 33 |
+
<details>
|
| 34 |
+
<summary><b>Update History & Changelog (v11-v18)</b></summary>
|
| 35 |
+
|
| 36 |
+
> **2026-05-16 Update (v18): Stability & Precision Patch.** (1) **Bulletproof False-Positive Detection:** Shifted agentic error detection from broad substring matching to strict structural formats (e.g., `"error":`, `Exception:`, `Traceback`), completely curing false-positive retry loops when successful JSON returns simply contain the word "error" or "fail". (2) **Legacy Engine Compatibility:** Replaced `loop.previtem` with explicit array indexing, fixing AST crashes on older `llama.cpp` and `minijinja` builds that do not track loop state items. (3) **True Whitespace Normalization:** Fixed a bug where reasoning bypasses and hallucinated tag recovery stacked hidden multi-newlines (`\n\n\n`), strictly fulfilling the 100% KV Cache hit rate claim for all edge cases. (4) **Code Cleanup:** Removed dead conditional branches during XML tool parsing.
|
| 37 |
+
>
|
| 38 |
+
> **2026-05-15 Update (v17):** Major architecture overhaul resolving edge cases in agentic tooling and KV Cache. (1) **Unified Template:** Consolidated Qwen 3.5 and Qwen 3.6 into a single `chat_template.jinja` file that handles all variants seamlessly. (2) **Fixed "Mutually Exclusive" Stopping Bug:** Changed the history-pruning logic from wiping the entire turn to safely array-slicing out just the raw tool tags (`content.split('<tool_call>')[0]`). This preserves the conversational text in the history, which cures the bug where the model would artificially abort its turn (output `<|im_end|>`) when it wanted to talk and use a tool simultaneously. (3) **100% KV Cache Hit Rate Restoration:** Fully normalized internal whitespace logic (`\n\n` -> `\n`) around think blocks and tool calls to exactly match the model's native autoregressive generation spacing. This perfectly synchronizes the template's rendered history with the cached generated tokens, completely eliminating the severe cache invalidation and full-prompt re-processing issues present in v16.
|
| 39 |
+
>
|
| 40 |
+
> **2026-05-14 Update (v16):** Four-part fix addressing community-reported regressions. (1) **Native XML tool format:** reverted from JSON back to the native `<function=name>` / `<parameter=x>` format the model was trained on, restoring full compatibility with vLLM's `qwen3_coder` parser and all inference engines that implement the Qwen tool protocol. (2) **`--reasoning off` respected in error paths:** when thinking is disabled (`enable_thinking=false` / `--reasoning off`), the error escalation directives are now injected as plain text without opening any `<think>` block, preventing degenerate prompts in no-reasoning sessions. (3) **Smarter false-positive detection:** short shell command results (starting with `$ `) and search results with timing footers (`Took X.Xs`) are now correctly excluded from error detection, preventing tool-retry loops when commands succeed but their output happens to contain the word `error`. (4) **`consecutive_failures` counter no longer resets on assistant messages**, allowing Tier 2 escalation to actually fire across multi-turn tool retry chains.
|
| 41 |
+
>
|
| 42 |
+
> **2026-05-13 Update (v15):** Three-part fix for agentic tool-loop failures. (1) **Two-tier error escalation:** replaced the brittle backwards-lookahead error detection with a fully forward-tracking `last_tool_failed` + `consecutive_failures` counter. On the first error the generation prompt is pre-seeded with a correction directive inside `<think>`; on the 2nd+ consecutive error the think block is bypassed and an out-of-band directive forces an immediate corrected action. (2) **Length-gated detection:** error signals are only read from short tool responses (< 500 chars), preventing false positives when reading code files containing `error`, `exception`, etc. in legitimate content. (3) **Static system prompt:** tool instructions are now fully unconditional, permanently eliminating the KV cache invalidation vector introduced in v14.
|
| 43 |
+
>
|
| 44 |
+
> **2026-05-12 Update (v14):** Cured tool amnesia loops and post-tool overthinking friction! Implemented **Smart Loop Preservation** to dynamically scan subsequent tool returns for error markers and conditionally preserve historical reasoning context during active tool failures. Broadened the system instruction scope to define `<think>` as a dual-purpose planning **or synthesis** space, completely eliminating indecisiveness post-tool retrieval.
|
| 45 |
+
>
|
| 46 |
+
> **2026-05-11 Update (v13):** Radical simplification and compatibility overhaul! Reverted tool schemas and assistant output formatting to standard JSON to natively fix downstream MCP parser crashes and C++ implicit enum coercion bugs. Removed the `ns_scan` history loop to permanently fix KV cache invalidation mid-conversation. Replaced global string replacement for hallucinated tags with a C++ safe, localized array-slicing method to prevent data-corruption on user code blocks.
|
| 47 |
+
>
|
| 48 |
+
> **2026-05-10 Update (v12):** Fixed agent stalls, parameter data-loss, and hallucination bugs! Restored dynamic tool instructions and the `<IMPORTANT>` formatting reminder block to stop grammar parser crashes.
|
| 49 |
+
>
|
| 50 |
+
> **2026-05-10 Update (v11):** Fixed agent looping and overthinking! Re-implemented `preserve_thinking` kwarg to properly strip reasoning blocks from history by default, and restored the reasoning bypass (`<think>\n\n</think>\n\n`).
|
| 51 |
+
|
| 52 |
+
</details>
|
| 53 |
+
|
| 54 |
+
This is a drop-in Jinja template that fixes rendering errors, KV cache invalidation, token waste, and fatal agentic stalling in the official Qwen chat templates.
|
| 55 |
+
|
| 56 |
+
It is tested to work across LM Studio, llama.cpp, vLLM, MLX, oMLX, and any engine that supports HuggingFace Jinja templates.
|
| 57 |
+
|
| 58 |
+
---
|
| 59 |
+
|
| 60 |
+
## Why you need this
|
| 61 |
+
The official Qwen templates contain restrictions and Python-specific Jinja logic that break usage on many inference engines and agent frameworks.
|
| 62 |
+
|
| 63 |
+
Here are the critical issues this template fixes:
|
| 64 |
+
|
| 65 |
+
| Category | Problem | Impact | Fix |
|
| 66 |
+
|---|---|---|---|
|
| 67 |
+
| **Agentic Loop** | **Premature Stalls (Stopping Bug)** | Model aborts its turn (`<\|im_end\|>`) when trying to combine conversation and a tool call. | Resolved the System Prompt logic trap and cured "Empty Think" poisoning (v19). |
|
| 68 |
+
| **Agentic Loop** | **Retry Stall & Reasoning Spiral** | Model correctly diagnoses a tool error but repeatedly emits the identical failing `<tool_call>`. | Two-tier escalation: seeds `<think>` with correction directive; injects urgent out-of-band directive. |
|
| 69 |
+
| **Agentic Loop** | **Post-Tool Overthinking** | Forced `<think>` block prefilling causes model to panic and debate internal rules after fetching data. | Broadened instructions to define `<think>` as a dual-purpose space for planning *or synthesis*. |
|
| 70 |
+
| **Agentic Loop** | **False-Positive Error Detection** | Short successful API/JSON returns containing the word `error` trigger false retry loops. | Strict structural guards look for exact system failures (`"error":`, `Traceback`, etc.) instead of broad words (v18). |
|
| 71 |
+
| **Performance** | **KV Cache Invalidation** | History pruning dynamically mutates past turns, causing full prompt re-processing every turn. | `preserve_thinking` defaults to `true`, maintaining strict chronological rendering for a 100% KV cache hit rate (v19). |
|
| 72 |
+
| **Performance** | **Empty Think Poisoning** | Stripped past turns leave behind empty `<think></think>` tags, tricking the model into a severe in-context learning bias. | Template completely abolishes the injection of empty think blocks (v19). |
|
| 73 |
+
| **Compatibility** | **Legacy Engine Crashes** | Older C++ parsing engines crash when evaluating `loop.previtem`. | Uses strict chronological array indexing universally supported by all Jinja iterations (v18). |
|
| 74 |
+
| **Compatibility** | **Wrong Tool Call Format** | Qwen-native parsers (like vLLM's `qwen3_coder`) expect XML `<function=name>`. JSON format breaks them. | Restored native XML format while keeping C++ safety. |
|
| 75 |
+
| **Compatibility** | **Jinja C++ Crashes** | Python-specific filters (`map`, `first` on strings) crash on `minijinja`. | All filters replaced with universally compatible equivalents. |
|
| 76 |
+
| **Stability** | **Mid-Conversation System Crash** | Frameworks injecting mid-conversation steering instructions trigger a hard crash. | Native, chronological rendering for system messages anywhere in the history. |
|
| 77 |
+
| **Stability** | **No-User-Query Crash** | `raise_exception` crashes agentic loops or system-only contexts. | Graceful fallback implemented. |
|
| 78 |
+
| **Stability** | **Unclosed Thinking Before Tool** | Model calls a tool without closing its reasoning, bleeding XML tags into tool parsers. | Auto-injects closing tags before tool boundaries securely. |
|
| 79 |
+
| **Edge Cases** | **`developer` Role Rejected** | Modern APIs send the developer role; the official template rejects it. | Added full support for `"developer"`. |
|
| 80 |
+
| **Edge Cases** | **`--reasoning off` Ignored** | When thinking is disabled, tool error escalation still opened a `<think>` block, corrupting the prompt. | Error escalation branches now fully respect `enable_thinking=false`. |
|
| 81 |
+
| **Edge Cases** | **Reasoning Bypass Hallucinations** | When thinking is disabled, Qwen models inherently hallucinate reasoning tags anyway. | Injects a safe boundary to successfully force reasoning bypass without stacking newlines (v18). |
|
| 82 |
+
|
| 83 |
+
---
|
| 84 |
+
|
| 85 |
+
## Quick install
|
| 86 |
+
|
| 87 |
+
Choose your environment and update the template:
|
| 88 |
+
|
| 89 |
+
### LM Studio
|
| 90 |
+
1. Open your Qwen model in the right-side panel.
|
| 91 |
+
2. Scroll down to **Prompt Template**.
|
| 92 |
+
3. Replace the template with the contents of `chat_template.jinja`.
|
| 93 |
+
4. Click **Save**.
|
| 94 |
+
|
| 95 |
+
### llama.cpp / koboldcpp
|
| 96 |
+
```bash
|
| 97 |
+
--jinja --chat-template-file chat_template.jinja
|
| 98 |
+
```
|
| 99 |
+
|
| 100 |
+
### vLLM
|
| 101 |
+
Replace the `"chat_template"` string in your `tokenizer_config.json` with the raw file contents. Use the `qwen3_coder` tool parser:
|
| 102 |
+
```bash
|
| 103 |
+
--tool-call-parser qwen3_coder
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
### oMLX
|
| 107 |
+
Overwrite `chat_template.jinja` in your local model directory. Load with `--jinja`. Remove any `chat_template_kwargs` overrides because the template handles everything internally.
|
| 108 |
+
|
| 109 |
+
---
|
| 110 |
+
|
| 111 |
+
## Which file do I use?
|
| 112 |
+
|
| 113 |
+
Both Qwen 3.5 and Qwen 3.6 variants (including 35B, 32B, 27B, and 14B parameters) have been consolidated. You only need the single `chat_template.jinja` file at the root of the repository.
|
| 114 |
+
|
| 115 |
+
One-line versions (`chat_template_oneline.txt`) are pre-minified for engines that require a single-line template string.
|
| 116 |
+
|
| 117 |
+
---
|
| 118 |
+
|
| 119 |
+
## The thinking toggle
|
| 120 |
+
You can control the model reasoning behavior. Insert `<|think_on|>` or `<|think_off|>` anywhere in your system or user prompt.
|
| 121 |
+
|
| 122 |
+
The template natively intercepts the tag, removes it from the final context so the model never sees it, and flips the reasoning mode instantly.
|
| 123 |
+
|
| 124 |
+
**Fast answer, no reasoning:**
|
| 125 |
+
```text
|
| 126 |
+
System: You are a coding assistant. <|think_off|>
|
| 127 |
+
User: What's 2+2?
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
**Deep reasoning:**
|
| 131 |
+
```text
|
| 132 |
+
System: You are a coding assistant. <|think_on|>
|
| 133 |
+
User: Implement a red-black tree in Rust.
|
| 134 |
+
```
|
| 135 |
+
*(The tag syntax uses Qwen's control-token delimiters to guarantee it will never collide with legitimate text or file paths, unlike earlier community templates that used `/think`)*
|
| 136 |
+
|
| 137 |
+
---
|
| 138 |
+
|
| 139 |
+
## Token Saving: Stripping past thoughts
|
| 140 |
+
|
| 141 |
+
By default in v19, this template **preserves** all past `<think>` blocks in the chat history. This is intentional: it prevents the model from suffering "amnesia stalls" during complex, multi-step agentic loops, and it mathematically guarantees a 100% Prefix KV Cache hit rate on local inference engines.
|
| 142 |
+
|
| 143 |
+
However, if you are running constrained hardware and need to save context tokens, you can explicitly disable this feature in your engine's template kwargs to automatically strip past thoughts:
|
| 144 |
+
|
| 145 |
+
```json
|
| 146 |
+
{
|
| 147 |
+
"preserve_thinking": false
|
| 148 |
+
}
|
| 149 |
+
```
|
| 150 |
+
*(Note: Setting this to false will naturally reduce your KV Cache hit rate during multi-turn chats, as the prompt string will dynamically mutate).*
|
| 151 |
+
|
| 152 |
+
---
|
| 153 |
+
|
| 154 |
+
<details>
|
| 155 |
+
<summary>Technical Details of the Critical Fixes</summary>
|
| 156 |
+
|
| 157 |
+
### 1. The "Empty Think" Poisoning & Logic Trap Cure (v19)
|
| 158 |
+
Previous versions attempted to save tokens by replacing past thoughts with empty `<think>\n</think>` blocks, combined with an absolute system prompt demanding a tool be called immediately after `</think>`. This created a toxic in-context learning pattern: the model associated empty thoughts with tools, and full thoughts with forbidden conversational text, causing an 80%+ premature `<|im_end|>` stalling rate. v19 abolishes empty think injection and rewrites the `<IMPORTANT>` directives to explicitly authorize conversational synthesis after a thought block.
|
| 159 |
+
|
| 160 |
+
### 2. KV Cache Safety & Autoregressive Normalization (v18/v19)
|
| 161 |
+
Llama.cpp and vLLM utilize prefix KV caching to speed up generation. Because v19 now preserves historical thoughts chronologically by default, the rendered history perfectly synchronizes with the cached generated tokens. Combined with strict single `\n` normalization at autoregressive boundaries, this achieves a 100% KV Cache hit rate in multi-turn loops.
|
| 162 |
+
|
| 163 |
+
### 3. Native XML Tool Call Format (v16)
|
| 164 |
+
The model was trained with the XML-based tool call format used by Qwen3-Coder:
|
| 165 |
+
```xml
|
| 166 |
+
<tool_call>
|
| 167 |
+
<function=tool_name>
|
| 168 |
+
<parameter=param_name>
|
| 169 |
+
value
|
| 170 |
+
</parameter>
|
| 171 |
+
</function>
|
| 172 |
+
</tool_call>
|
| 173 |
+
```
|
| 174 |
+
v16 restored this format natively, making it compatible with all parsers while bypassing the `|items` crash by using C++ safe key iteration (`for args_name in tool_call.arguments`).
|
| 175 |
+
|
| 176 |
+
### 4. Two-Tier Agentic Error Escalation (v15)
|
| 177 |
+
When a tool call fails validation repeatedly, the model can enter a degenerate reasoning spiral. This template leverages a two-tier escalation system driven by a forward-tracked `consecutive_failures` counter:
|
| 178 |
+
- **Tier 1 (1st error):** Generation prompt prefix changes to seed reasoning at a different token position, breaking the cached attractor state.
|
| 179 |
+
- **Tier 2 (2nd+ consecutive errors):** Think block bypassed entirely. An urgent out-of-band directive forces an immediate corrected action wrapped safely within the user `tool_response` block.
|
| 180 |
+
|
| 181 |
+
### 5. Smart False-Positive Detection (v18)
|
| 182 |
+
Instead of broad substring matching that triggers false retry-loops on successful database returns containing words like "error", v18 utilizes strict structural guards looking for `Exception:`, `"error":`, `Traceback`, and `command not found`, combined with length gates and shell-echo exclusions (`$ `).
|
| 183 |
+
|
| 184 |
+
### 6. minijinja Compatibility Constraints (v18/v20)
|
| 185 |
+
Python-only Jinja2 features crash or misbehave on `minijinja`/`minja` (the C++ runtime used by llama.cpp, LM Studio, and MLX). All instances have been refactored for universal support:
|
| 186 |
+
- `content | replace('<|think_on|>', '')` -> `content.split('<|think_on|>') | join('')` (Fixes a severe bug where `minja` silently drops the entire text payload if the replaced string is found at index 0).
|
| 187 |
+
- `\| items` -> `for key in mapping`
|
| 188 |
+
- `loop.previtem` -> `messages[loop.index0 - 1]`
|
| 189 |
+
- `map('string')` -> `join('|')`
|
| 190 |
+
- `\| first` -> `'$ ' in content`
|
| 191 |
+
|
| 192 |
+
</details>
|
| 193 |
+
|
| 194 |
+
<details>
|
| 195 |
+
<summary>Comparison Matrix: Official vs Fixed vs Community</summary>
|
| 196 |
+
|
| 197 |
+
| Feature | Official Qwen Templates | LuffyTheFox | mod-ellary | Pneuny | **This Fixed Template (v19)** |
|
| 198 |
+
|---------|----------|-------------|------------|--------|----------------|
|
| 199 |
+
| Tool call format | XML (native) | JSON | JSON | JSON | **XML (native, qwen3_coder compatible)** |
|
| 200 |
+
| Tool arguments | Fails (`\|items`) | Fixed | Missing | Fixed | **Fixed (C++ safe XML)** |
|
| 201 |
+
| Premature Stalls (Stopping Bug) | Stalls | Stalls | Stalls | Stalls | **Fixed via Logic Trap / Poisoning removal (v19)** |
|
| 202 |
+
| Agentic Retry Stall & Reasoning Spiral | Stalls | Stalls | Stalls | Stalls | **Two-tier escalation system** |
|
| 203 |
+
| False-positive tool errors | N/A | N/A | N/A | N/A | **Guarded (Strict structural matching)** |
|
| 204 |
+
| Post-Tool Overthinking | Spams/Stalls | Broken | Broken | Broken | **Universal Synthesis** |
|
| 205 |
+
| `--reasoning off` on tool errors | N/A | N/A | N/A | N/A | **Fully respected** |
|
| 206 |
+
| `developer` role | Missing | Missing | Missing | Missing | **Added** |
|
| 207 |
+
| Thinking toggle | None | None | `/think` (system only) | None | **`<\|think_off\|>` anywhere** |
|
| 208 |
+
| Empty think in history | Spams empty blocks | Broken | Tags omitted | Broken | **Abolished completely (v19)** |
|
| 209 |
+
| KV prefix caching | Breaks on dynamic history | Breaks | Breaks | Breaks | **100% stable out-of-the-box (v19)** |
|
| 210 |
+
| Mid-conversation system | Crashes | Crashes | Crashes | Crashes | **Fixed** |
|
| 211 |
+
| No-user-query crash | Crashes | Crashes | Crashes | Crashes | **Graceful fallback** |
|
| 212 |
+
| Legacy AST support | Fails (`previtem`) | Fails | Fails | Fails | **Fixed (`index0`)** |
|
| 213 |
+
| `</thinking>` hallucination | Fails | N/A | N/A | N/A | **Detected and safely trimmed** |
|
| 214 |
+
|
| 215 |
+
</details>
|
| 216 |
+
|
| 217 |
+
---
|
| 218 |
+
|
| 219 |
+
## Running the test suite
|
| 220 |
+
|
| 221 |
+
```bash
|
| 222 |
+
python3 scripts/test_v20.py
|
| 223 |
+
```
|
| 224 |
+
|
| 225 |
+
Tests cover: `auto_disable_thinking_with_tools`, payload truncation logic, parallel tool spacing, mid-conversation system rendering, deep agent loop fallback, XML tool format, `<|think_off|>` / `<|think_on|>` inline overrides, and all legacy v19 regression tests.
|
| 226 |
+
|
| 227 |
+
---
|
| 228 |
+
|
| 229 |
+
## Authorship
|
| 230 |
+
|
| 231 |
+
| Role | Author |
|
| 232 |
+
|------|--------|
|
| 233 |
+
| Original models | Alibaba Cloud (Qwen team) |
|
| 234 |
+
| Template fixes | [froggeric](https://huggingface.co/froggeric) |
|
| 235 |
+
| C++ AST optimizations | [barubary](https://github.com/spiritbuun/buun-llama-cpp) / `spiritbuun` |
|
| 236 |
+
|
| 237 |
+
## License
|
| 238 |
+
|
| 239 |
+
Apache-2.0, inherited from Qwen.
|
archive/v20_chat_template.jinja
ADDED
|
@@ -0,0 +1,287 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.6-froggeric-v20" %}
|
| 2 |
+
{%- set image_count = namespace(value=0) %}
|
| 3 |
+
{%- set video_count = namespace(value=0) %}
|
| 4 |
+
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
|
| 5 |
+
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
|
| 6 |
+
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
|
| 7 |
+
{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else false %}
|
| 8 |
+
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
|
| 9 |
+
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
|
| 10 |
+
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
|
| 11 |
+
{%- set ns_state = namespace(thinking=enable_thinking) %}
|
| 12 |
+
{%- if auto_disable_thinking_with_tools and _has_tools %}
|
| 13 |
+
{%- set ns_state.thinking = false %}
|
| 14 |
+
{%- endif %}
|
| 15 |
+
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
|
| 16 |
+
{%- if content is string %}
|
| 17 |
+
{{- content }}
|
| 18 |
+
{%- elif content is iterable and content is not mapping %}
|
| 19 |
+
{%- for item in content %}
|
| 20 |
+
{%- if item is mapping %}
|
| 21 |
+
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
|
| 22 |
+
{%- if is_system_content %}
|
| 23 |
+
{{- raise_exception('System message cannot contain images.') }}
|
| 24 |
+
{%- endif %}
|
| 25 |
+
{%- if do_vision_count %}
|
| 26 |
+
{%- set image_count.value = image_count.value + 1 %}
|
| 27 |
+
{%- endif %}
|
| 28 |
+
{%- if add_vision_id %}
|
| 29 |
+
{{- 'Picture ' ~ image_count.value ~ ': ' }}
|
| 30 |
+
{%- endif %}
|
| 31 |
+
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
|
| 32 |
+
{%- elif item.type == 'video' or 'video' in item %}
|
| 33 |
+
{%- if is_system_content %}
|
| 34 |
+
{{- raise_exception('System message cannot contain videos.') }}
|
| 35 |
+
{%- endif %}
|
| 36 |
+
{%- if do_vision_count %}
|
| 37 |
+
{%- set video_count.value = video_count.value + 1 %}
|
| 38 |
+
{%- endif %}
|
| 39 |
+
{%- if add_vision_id %}
|
| 40 |
+
{{- 'Video ' ~ video_count.value ~ ': ' }}
|
| 41 |
+
{%- endif %}
|
| 42 |
+
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
|
| 43 |
+
{%- elif 'text' in item %}
|
| 44 |
+
{{- item.text }}
|
| 45 |
+
{%- else %}
|
| 46 |
+
{{- raise_exception('Unexpected item type in content.') }}
|
| 47 |
+
{%- endif %}
|
| 48 |
+
{%- else %}
|
| 49 |
+
{{- item | string }}
|
| 50 |
+
{%- endif %}
|
| 51 |
+
{%- endfor %}
|
| 52 |
+
{%- elif content is none or content is undefined %}
|
| 53 |
+
{{- '' }}
|
| 54 |
+
{%- else %}
|
| 55 |
+
{{- raise_exception('Unexpected content type.') }}
|
| 56 |
+
{%- endif %}
|
| 57 |
+
{%- endmacro %}
|
| 58 |
+
{%- if not messages %}
|
| 59 |
+
{{- raise_exception('No messages provided.') }}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{%- set _first_role = messages[0].role %}
|
| 62 |
+
{%- if _first_role == 'system' or _first_role == 'developer' %}
|
| 63 |
+
{%- set _sys_msg = messages[0] %}
|
| 64 |
+
{%- set _msgs = messages[1:] %}
|
| 65 |
+
{%- else %}
|
| 66 |
+
{%- set _sys_msg = none %}
|
| 67 |
+
{%- set _msgs = messages %}
|
| 68 |
+
{%- endif %}
|
| 69 |
+
{%- set _sc = '' %}
|
| 70 |
+
{%- if _sys_msg is not none %}
|
| 71 |
+
{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}
|
| 72 |
+
{%- if '<|think_off|>' in _sc %}
|
| 73 |
+
{%- set ns_state.thinking = false %}
|
| 74 |
+
{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}
|
| 75 |
+
{%- elif '<|think_on|>' in _sc %}
|
| 76 |
+
{%- set ns_state.thinking = true %}
|
| 77 |
+
{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}
|
| 78 |
+
{%- endif %}
|
| 79 |
+
{%- endif %}
|
| 80 |
+
{%- if _has_tools %}
|
| 81 |
+
{{- '<|im_start|>system\n' }}
|
| 82 |
+
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
|
| 83 |
+
{%- for tool in tools %}
|
| 84 |
+
{{- '\n' }}
|
| 85 |
+
{{- tool | tojson }}
|
| 86 |
+
{%- endfor %}
|
| 87 |
+
{{- '\n</tools>' }}
|
| 88 |
+
{%- set tool_instructions %}
|
| 89 |
+
If you choose to call a function ONLY reply in the following format with NO suffix:
|
| 90 |
+
|
| 91 |
+
<think>
|
| 92 |
+
Brief explanation of tool call
|
| 93 |
+
</think>
|
| 94 |
+
<tool_call>
|
| 95 |
+
<function=example_function_name>
|
| 96 |
+
<parameter=example_parameter_1>
|
| 97 |
+
value_1
|
| 98 |
+
</parameter>
|
| 99 |
+
<parameter=example_parameter_2>
|
| 100 |
+
This is the value for the second parameter
|
| 101 |
+
that can span
|
| 102 |
+
multiple lines
|
| 103 |
+
</parameter>
|
| 104 |
+
</function>
|
| 105 |
+
</tool_call>
|
| 106 |
+
|
| 107 |
+
<IMPORTANT>
|
| 108 |
+
Reminder:
|
| 109 |
+
- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.
|
| 110 |
+
- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.
|
| 111 |
+
- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.
|
| 112 |
+
- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after closing </think>. Do NOT output any conversational text before the tool call.
|
| 113 |
+
- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.
|
| 114 |
+
- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.
|
| 115 |
+
- If you have gathered all necessary data and do not need to call a tool, answer the question like normal and provide your final response to the user IMMEDIATELY after closing </think>.
|
| 116 |
+
</IMPORTANT>
|
| 117 |
+
{%- endset %}
|
| 118 |
+
{{- '\n\n' ~ tool_instructions | trim }}
|
| 119 |
+
{%- if _sc %}
|
| 120 |
+
{{- '\n\n' + _sc }}
|
| 121 |
+
{%- endif %}
|
| 122 |
+
{{- '<|im_end|>\n' }}
|
| 123 |
+
{%- else %}
|
| 124 |
+
{%- if _sc %}
|
| 125 |
+
{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}
|
| 126 |
+
{%- endif %}
|
| 127 |
+
{%- endif %}
|
| 128 |
+
{%- set _last_idx = _msgs | length - 1 %}
|
| 129 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
|
| 130 |
+
{%- for message in _msgs[::-1] %}
|
| 131 |
+
{%- set index = (_msgs | length - 1) - loop.index0 %}
|
| 132 |
+
{%- if ns.multi_step_tool and message.role == 'user' %}
|
| 133 |
+
{%- set _rc = render_content(message.content, false) | trim %}
|
| 134 |
+
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
|
| 135 |
+
{%- set ns.multi_step_tool = false %}
|
| 136 |
+
{%- set ns.last_query_index = index %}
|
| 137 |
+
{%- endif %}
|
| 138 |
+
{%- endif %}
|
| 139 |
+
{%- endfor %}
|
| 140 |
+
{%- if ns.multi_step_tool %}
|
| 141 |
+
{%- if _last_idx > 50 %}
|
| 142 |
+
{%- set ns.last_query_index = _last_idx %}
|
| 143 |
+
{%- else %}
|
| 144 |
+
{%- set ns.last_query_index = 0 %}
|
| 145 |
+
{%- endif %}
|
| 146 |
+
{%- endif %}
|
| 147 |
+
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
|
| 148 |
+
{%- for message in _msgs %}
|
| 149 |
+
{%- set is_system = (message.role == "system" or message.role == "developer") %}
|
| 150 |
+
{%- set content = render_content(message.content, true, is_system) | trim %}
|
| 151 |
+
{%- if '<|think_off|>' in content %}
|
| 152 |
+
{%- set ns_state.thinking = false %}
|
| 153 |
+
{%- set content = content.split('<|think_off|>') | join('') | trim %}
|
| 154 |
+
{%- elif '<|think_on|>' in content %}
|
| 155 |
+
{%- set ns_state.thinking = true %}
|
| 156 |
+
{%- set content = content.split('<|think_on|>') | join('') | trim %}
|
| 157 |
+
{%- endif %}
|
| 158 |
+
{%- if is_system %}
|
| 159 |
+
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
|
| 160 |
+
{%- elif message.role == 'user' %}
|
| 161 |
+
{%- set ns2.consecutive_failures = 0 %}
|
| 162 |
+
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
|
| 163 |
+
{%- elif message.role == 'assistant' %}
|
| 164 |
+
{%- set reasoning_content = '' %}
|
| 165 |
+
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
| 166 |
+
{%- if message.reasoning_content is string %}
|
| 167 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 168 |
+
{%- else %}
|
| 169 |
+
{%- set reasoning_content = message.reasoning_content | string %}
|
| 170 |
+
{%- endif %}
|
| 171 |
+
{%- else %}
|
| 172 |
+
{%- set _think_end = '' %}
|
| 173 |
+
{%- if '</think>' in content %}
|
| 174 |
+
{%- set _think_end = '</think>' %}
|
| 175 |
+
{%- elif '</thinking>' in content %}
|
| 176 |
+
{%- set _think_end = '</thinking>' %}
|
| 177 |
+
{%- elif '</ think>' in content %}
|
| 178 |
+
{%- set _think_end = '</ think>' %}
|
| 179 |
+
{%- elif '</think >' in content %}
|
| 180 |
+
{%- set _think_end = '</think >' %}
|
| 181 |
+
{%- endif %}
|
| 182 |
+
{%- if _think_end %}
|
| 183 |
+
{%- if _think_end == '</thinking>' %}
|
| 184 |
+
{%- set _think_start = '<thinking>' %}
|
| 185 |
+
{%- else %}
|
| 186 |
+
{%- set _think_start = '<think>' %}
|
| 187 |
+
{%- endif %}
|
| 188 |
+
{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
|
| 189 |
+
{%- if _think_start in reasoning_content %}
|
| 190 |
+
{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
|
| 191 |
+
{%- endif %}
|
| 192 |
+
{%- set content = content.split(_think_end)[-1].lstrip('\n') %}
|
| 193 |
+
{%- endif %}
|
| 194 |
+
{%- endif %}
|
| 195 |
+
{%- set reasoning_content = reasoning_content | trim %}
|
| 196 |
+
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
|
| 197 |
+
{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
|
| 198 |
+
{%- else %}
|
| 199 |
+
{{- '<|im_start|>assistant\n' + content }}
|
| 200 |
+
{%- endif %}
|
| 201 |
+
{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
|
| 202 |
+
{%- for tool_call in message.tool_calls %}
|
| 203 |
+
{%- if tool_call.function is defined and tool_call.function is not none %}
|
| 204 |
+
{%- set tc = tool_call.function %}
|
| 205 |
+
{%- else %}
|
| 206 |
+
{%- set tc = tool_call %}
|
| 207 |
+
{%- endif %}
|
| 208 |
+
{%- if loop.first %}
|
| 209 |
+
{%- if content | trim %}
|
| 210 |
+
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 211 |
+
{%- else %}
|
| 212 |
+
{{- '<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 213 |
+
{%- endif %}
|
| 214 |
+
{%- else %}
|
| 215 |
+
{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}
|
| 216 |
+
{%- endif %}
|
| 217 |
+
{%- if tc.arguments is defined and tc.arguments is not none %}
|
| 218 |
+
{%- if tc.arguments is mapping %}
|
| 219 |
+
{%- for args_name, args_value in tc.arguments.items() %}
|
| 220 |
+
{{- '<parameter=' + args_name + '>\n' }}
|
| 221 |
+
{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
|
| 222 |
+
{%- set _av = args_value | tojson %}
|
| 223 |
+
{%- else %}
|
| 224 |
+
{%- set _av = args_value | string %}
|
| 225 |
+
{%- endif %}
|
| 226 |
+
{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
|
| 227 |
+
{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}
|
| 228 |
+
{%- else %}
|
| 229 |
+
{{- _av }}
|
| 230 |
+
{%- endif %}
|
| 231 |
+
{{- '\n</parameter>\n' }}
|
| 232 |
+
{%- endfor %}
|
| 233 |
+
{%- elif tc.arguments is string and tc.arguments %}
|
| 234 |
+
{{- tc.arguments }}
|
| 235 |
+
{%- endif %}
|
| 236 |
+
{%- endif %}
|
| 237 |
+
{%- if loop.last %}
|
| 238 |
+
{{- '</function>\n</tool_call>\n' }}
|
| 239 |
+
{%- else %}
|
| 240 |
+
{{- '</function>\n</tool_call>' }}
|
| 241 |
+
{%- endif %}
|
| 242 |
+
{%- endfor %}
|
| 243 |
+
{%- endif %}
|
| 244 |
+
{{- '<|im_end|>\n' }}
|
| 245 |
+
{%- elif message.role == 'tool' %}
|
| 246 |
+
{%- set _content_lower = content | lower %}
|
| 247 |
+
{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_lower or 'error:' in _content_lower or 'exception:' in _content_lower or 'traceback' in _content_lower or 'command not found' in _content_lower or 'invalid syntax' in _content_lower or 'failed to' in _content_lower) %}
|
| 248 |
+
{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
|
| 249 |
+
{%- else %}
|
| 250 |
+
{%- set ns2.consecutive_failures = 0 %}
|
| 251 |
+
{%- endif %}
|
| 252 |
+
{%- if ns2.prev_role != 'tool' %}
|
| 253 |
+
{{- '<|im_start|>user' }}
|
| 254 |
+
{%- endif %}
|
| 255 |
+
{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
|
| 256 |
+
{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}
|
| 257 |
+
{%- endif %}
|
| 258 |
+
{{- '\n<tool_response>\n' + content }}
|
| 259 |
+
{%- if ns2.consecutive_failures >= 2 %}
|
| 260 |
+
{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
|
| 261 |
+
{%- elif ns2.consecutive_failures == 1 %}
|
| 262 |
+
{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
|
| 263 |
+
{%- endif %}
|
| 264 |
+
{{- '\n</tool_response>' }}
|
| 265 |
+
{%- if loop.last %}
|
| 266 |
+
{{- '<|im_end|>\n' }}
|
| 267 |
+
{%- else %}
|
| 268 |
+
{%- set _next_role = _msgs[loop.index0 + 1].role %}
|
| 269 |
+
{%- if _next_role != 'tool' %}
|
| 270 |
+
{{- '<|im_end|>\n' }}
|
| 271 |
+
{%- endif %}
|
| 272 |
+
{%- endif %}
|
| 273 |
+
{%- else %}
|
| 274 |
+
{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
|
| 275 |
+
{%- endif %}
|
| 276 |
+
{%- set ns2.prev_role = message.role %}
|
| 277 |
+
{%- endfor %}
|
| 278 |
+
{%- if add_generation_prompt %}
|
| 279 |
+
{{- '<|im_start|>assistant\n' }}
|
| 280 |
+
{%- if not ns_state.thinking %}
|
| 281 |
+
{{- '<think>\n</think>\n' }}
|
| 282 |
+
{%- elif ns2.consecutive_failures >= 2 %}
|
| 283 |
+
{{- '<think>\n</think>\n' }}
|
| 284 |
+
{%- else %}
|
| 285 |
+
{{- '<think>\n' }}
|
| 286 |
+
{%- endif %}
|
| 287 |
+
{%- endif %}
|
archive/v20_chat_template_oneline.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{%- set template_version = "qwen3.6-froggeric-v20" %}{%- set image_count = namespace(value=0) %}{%- set video_count = namespace(value=0) %}{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else false %}{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}{%- set ns_state = namespace(thinking=enable_thinking) %}{%- if auto_disable_thinking_with_tools and _has_tools %}{%- set ns_state.thinking = false %}{%- endif %}{%- macro render_content(content, do_vision_count, is_system_content=false) %}{%- if content is string %}{{- content }}{%- elif content is iterable and content is not mapping %}{%- for item in content %}{%- if item is mapping %}{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain images.') }}{%- endif %}{%- if do_vision_count %}{%- set image_count.value = image_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Picture ' ~ image_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|image_pad|><|vision_end|>' }}{%- elif item.type == 'video' or 'video' in item %}{%- if is_system_content %}{{- raise_exception('System message cannot contain videos.') }}{%- endif %}{%- if do_vision_count %}{%- set video_count.value = video_count.value + 1 %}{%- endif %}{%- if add_vision_id %}{{- 'Video ' ~ video_count.value ~ ': ' }}{%- endif %}{{- '<|vision_start|><|video_pad|><|vision_end|>' }}{%- elif 'text' in item %}{{- item.text }}{%- else %}{{- raise_exception('Unexpected item type in content.') }}{%- endif %}{%- else %}{{- item | string }}{%- endif %}{%- endfor %}{%- elif content is none or content is undefined %}{{- '' }}{%- else %}{{- raise_exception('Unexpected content type.') }}{%- endif %}{%- endmacro %}{%- if not messages %}{{- raise_exception('No messages provided.') }}{%- endif %}{%- set _first_role = messages[0].role %}{%- if _first_role == 'system' or _first_role == 'developer' %}{%- set _sys_msg = messages[0] %}{%- set _msgs = messages[1:] %}{%- else %}{%- set _sys_msg = none %}{%- set _msgs = messages %}{%- endif %}{%- set _sc = '' %}{%- if _sys_msg is not none %}{%- set _sc = render_content(_sys_msg.content, false, true) | trim %}{%- if '<|think_off|>' in _sc %}{%- set ns_state.thinking = false %}{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in _sc %}{%- set ns_state.thinking = true %}{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- endif %}{%- if _has_tools %}{{- '<|im_start|>system\n' }}{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}{%- for tool in tools %}{{- '\n' }}{{- tool | tojson }}{%- endfor %}{{- '\n</tools>' }}{%- set tool_instructions %} If you choose to call a function ONLY reply in the following format with NO suffix: <think> Brief explanation of tool call </think> <tool_call> <function=example_function_name> <parameter=example_parameter_1> value_1 </parameter> <parameter=example_parameter_2> This is the value for the second parameter that can span multiple lines </parameter> </function> </tool_call> <IMPORTANT> Reminder: - You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user. - ALL explanation and reasoning MUST be placed strictly inside the <think></think> block. - Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags. - If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after closing </think>. Do NOT output any conversational text before the tool call. - The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them. - To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks. - If you have gathered all necessary data and do not need to call a tool, answer the question like normal and provide your final response to the user IMMEDIATELY after closing </think>. </IMPORTANT> {%- endset %}{{- '\n\n' ~ tool_instructions | trim }}{%- if _sc %}{{- '\n\n' + _sc }}{%- endif %}{{- '<|im_end|>\n' }}{%- else %}{%- if _sc %}{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- set _last_idx = _msgs | length - 1 %}{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}{%- for message in _msgs[::-1] %}{%- set index = (_msgs | length - 1) - loop.index0 %}{%- if ns.multi_step_tool and message.role == 'user' %}{%- set _rc = render_content(message.content, false) | trim %}{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}{%- set ns.multi_step_tool = false %}{%- set ns.last_query_index = index %}{%- endif %}{%- endif %}{%- endfor %}{%- if ns.multi_step_tool %}{%- if _last_idx > 50 %}{%- set ns.last_query_index = _last_idx %}{%- else %}{%- set ns.last_query_index = 0 %}{%- endif %}{%- endif %}{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}{%- for message in _msgs %}{%- set is_system = (message.role == "system" or message.role == "developer") %}{%- set content = render_content(message.content, true, is_system) | trim %}{%- if '<|think_off|>' in content %}{%- set ns_state.thinking = false %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- elif '<|think_on|>' in content %}{%- set ns_state.thinking = true %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}{%- if is_system %}{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'user' %}{%- set ns2.consecutive_failures = 0 %}{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}{%- elif message.role == 'assistant' %}{%- set reasoning_content = '' %}{%- if message.reasoning_content is defined and message.reasoning_content is not none %}{%- if message.reasoning_content is string %}{%- set reasoning_content = message.reasoning_content %}{%- else %}{%- set reasoning_content = message.reasoning_content | string %}{%- endif %}{%- else %}{%- set _think_end = '' %}{%- if '</think>' in content %}{%- set _think_end = '</think>' %}{%- elif '</thinking>' in content %}{%- set _think_end = '</thinking>' %}{%- elif '</ think>' in content %}{%- set _think_end = '</ think>' %}{%- elif '</think >' in content %}{%- set _think_end = '</think >' %}{%- endif %}{%- if _think_end %}{%- if _think_end == '</thinking>' %}{%- set _think_start = '<thinking>' %}{%- else %}{%- set _think_start = '<think>' %}{%- endif %}{%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}{%- if _think_start in reasoning_content %}{%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}{%- endif %}{%- set content = content.split(_think_end)[-1].lstrip('\n') %}{%- endif %}{%- endif %}{%- set reasoning_content = reasoning_content | trim %}{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}{{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}{%- else %}{{- '<|im_start|>assistant\n' + content }}{%- endif %}{%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}{%- for tool_call in message.tool_calls %}{%- if tool_call.function is defined and tool_call.function is not none %}{%- set tc = tool_call.function %}{%- else %}{%- set tc = tool_call %}{%- endif %}{%- if loop.first %}{%- if content | trim %}{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}{%- else %}{{- '<tool_call>\n<function=' + tc.name + '>\n' }}{%- endif %}{%- else %}{{- '\n\n<tool_call>\n<function=' + tc.name + '>\n' }}{%- endif %}{%- if tc.arguments is defined and tc.arguments is not none %}{%- if tc.arguments is mapping %}{%- for args_name, args_value in tc.arguments.items() %}{{- '<parameter=' + args_name + '>\n' }}{%- if args_value is mapping or (args_value is sequence and args_value is not string) %}{%- set _av = args_value | tojson %}{%- else %}{%- set _av = args_value | string %}{%- endif %}{%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}{{- _av[:max_tool_arg_chars] + '\n[TRUNCATED — original length ' ~ (_av | length | string) ~ ' chars]' }}{%- else %}{{- _av }}{%- endif %}{{- '\n</parameter>\n' }}{%- endfor %}{%- elif tc.arguments is string and tc.arguments %}{{- tc.arguments }}{%- endif %}{%- endif %}{%- if loop.last %}{{- '</function>\n</tool_call>\n' }}{%- else %}{{- '</function>\n</tool_call>' }}{%- endif %}{%- endfor %}{%- endif %}{{- '<|im_end|>\n' }}{%- elif message.role == 'tool' %}{%- set _content_lower = content | lower %}{%- if content | length < 500 and '$ ' not in content and 'took ' not in _content_lower and ('"error":' in _content_lower or 'error:' in _content_lower or 'exception:' in _content_lower or 'traceback' in _content_lower or 'command not found' in _content_lower or 'invalid syntax' in _content_lower or 'failed to' in _content_lower) %}{%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}{%- else %}{%- set ns2.consecutive_failures = 0 %}{%- endif %}{%- if ns2.prev_role != 'tool' %}{{- '<|im_start|>user' }}{%- endif %}{%- if max_tool_response_chars > 0 and content | length > max_tool_response_chars %}{%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED — original length ' ~ (content | length | string) ~ ' chars]' %}{%- endif %}{{- '\n<tool_response>\n' + content }}{%- if ns2.consecutive_failures >= 2 %}{{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}{%- elif ns2.consecutive_failures == 1 %}{{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}{%- endif %}{{- '\n</tool_response>' }}{%- if loop.last %}{{- '<|im_end|>\n' }}{%- else %}{%- set _next_role = _msgs[loop.index0 + 1].role %}{%- if _next_role != 'tool' %}{{- '<|im_end|>\n' }}{%- endif %}{%- endif %}{%- else %}{{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}{%- endif %}{%- set ns2.prev_role = message.role %}{%- endfor %}{%- if add_generation_prompt %}{{- '<|im_start|>assistant\n' }}{%- if not ns_state.thinking %}{{- '<think>\n</think>\n' }}{%- elif ns2.consecutive_failures >= 2 %}{{- '<think>\n</think>\n' }}{%- else %}{{- '<think>\n' }}{%- endif %}{%- endif %}
|
scripts/{test_v20.py → test_v21.py}
RENAMED
|
File without changes
|