Moore2877/Qwen-Fixed-Chat-Templates-llamacpp

Qwen 3.6 Agentic Chat Template for llama.cpp

A Jinja chat template for Qwen 3.6 tuned for llama.cpp / ik_llama driving agentic coding harnesses (OpenCode, Claude Code, Hermes, aider, and others).

Built on froggeric/Qwen-Fixed-Chat-Templates v21.3 with an agentic hardening layer on top: structural loop detection, tool-failure classification, vanished-tool detection, and argument-grounding rules. Every change is grounded in captured harness traffic and live-tested on llama.cpp and ik_llama before release.

Install

llama-server ... --jinja --chat-template-file chat_template.jinja

Modern builds auto-derive the tool-call parser, and the default XML format works out of the box (large multiline writes included — the server escapes, not the model). Older parsers that only read Hermes JSON need --chat-template-kwargs '{"tool_call_format":"json"}' (symptom without it: tool calls appear as plain <function=...> text).

Set kwargs on the llama-server command line, not the client config — some harnesses silently drop client-side chat_template_kwargs (OpenCode #26233).

What it adds over upstream

Structural loop detection (template-level; samplers only catch verbatim token loops, not action loops). Three detectors, each covering what the others cannot:

• No-progress — a tool returns identical results repeat_nudge_after times (default 6). Skips mutating tools (mutating_tools kwarg): writing six files returns six identical "success" strings that are productive, not stuck. Measured on 381 real sessions, this fires ~90% less than naive counting while keeping every genuine catch (scripts/measure_loop_nudges.py). Includes a polling escape hatch: deliberate wait-and-check patterns can declare themselves.
• Action loop / ping-pong — the model repeats the same call (tool + arguments) while the results differ every time (screenshots, timestamps, or harness message IDs make results unique and blind the no-progress detector). Call signatures are compared against the previous two turns, catching A→A→A and A→B→A→B patterns. pingpong_nudge_after (default 4). Deliberately does not skip mutating tools — re-issuing the same write with the same arguments is a loop. The warning includes a polling escape hatch so deliberate wait-and-check patterns can declare themselves.
• Consecutive errors — failure loops, with escalating warnings.

Vanished-tools note. If a tool was called earlier in the session but is missing from the current tools[], its server likely died mid-session (observed in the field: a model killed its own MCP servers with a broad process kill, then improvised around the gap for 40 turns without noticing). The template detects the mismatch structurally and injects a system note naming the missing tools. The note bans pretending the tools exist and bans repairing servers, but allows safe continuation with the remaining tools. warn_vanished_tools (default true) — set it to false if your harness legitimately changes the tool list per request (just-in-time tool loading; meta-tool patterns such as mcp_search are fine because the meta-tool stays listed).

Tool-failure classification that tells the truth. Output starting with Traceback/Error:/fatal: marks a failed tool — unless the tool is a read/query tool (read_tools kwarg): a read returning a log file that begins with Error: is a successful read of error-shaped content, not a tool failure. Harness-generated envelopes (invalid input for tool, json parsing failed, unknown tool, ...) count as failures for every tool class. Deliberately never matches user permission rejections — scolding the model into retrying after a human said no is harmful. Warnings are worded to let the model self-classify: an expected failure mid-fix (a failing test being worked on) is told to continue its normal cycle, and consecutive-error warnings distinguish an identical repeated error (change the approach) from changing errors (normal burn-down — continue).

Per-result tool attribution. When an assistant turn issues several tool calls at once, each tool result resolves its own function via its tool_call_id before classification. Without this, a batch that mixes tool names loses the classification for every result in it — a parallel read returning an error log is reported as a failed tool invocation, and parallel writes lose their no-progress exemption. Results that carry no usable tool_call_id keep the conservative single-tool fallback.

Argument grounding rules. The tool instructions require every required argument value to come from the user request, visible repository state, the tool schema, or a prior tool result — and require edit/write paths and replacement strings to have been observed before editing. This targets hallucinated file paths and invented replacement strings, the most expensive local-agent failure.

Two-sided turn management. The tool instructions balance continuation against stopping: the model is told to keep working until the request is complete (verify each change, then move on) and to prefer small concrete actions over long deliberation, while a bounded set of exit conditions (task done, blocked, needs user input, or a meaningful checkpoint worth showing) tells it when to stop. This replaced an earlier one-sided instruction whose trigger — "if you have all necessary data" — was satisfied after every successful tool call, and it is deliberately tuned to avoid both premature mid-task stops and reasoning spirals.

Harness-aware error markers — adds json parsing failed, unknown tool, tool execution failed, err:, and similar to upstream's CLI-oriented list.

Configuration

Everything is a kwarg:

kwarg	default	purpose
tool_call_format	xml	json for parsers that only read Hermes JSON
repeat_nudge_after	6	identical-result turns before the no-progress warning; 0 disables
pingpong_nudge_after	4	identical call-signature (tool+args) turns before the action-loop warning; 0 disables
warn_vanished_tools	true	system note when a tool used earlier disappears from tools[]; set false under just-in-time tool loading
mutating_tools	~55-name cross-harness union	tool names excluded from no-progress detection (case-insensitive). Covers 16+ harnesses; override for a custom one. MCP-prefixed variants are not matched by design
read_tools	~30-name cross-harness union	read/query tools exempt from content-based failure markers (case-insensitive)
think_on_tool_failure	false	true keeps reasoning open during error escalation
enable_thinking · preserve_thinking · max_tool_arg_chars · max_tool_response_chars	upstream	inherited from froggeric v21.3; max_tool_arg_chars also truncates historical string arguments

Put all kwargs in a single JSON object — a second --chat-template-kwargs flag overrides, it does not merge. A fully loaded example:

--chat-template-kwargs '{"tool_call_format":"json","repeat_nudge_after":4,"think_on_tool_failure":true,"max_tool_response_chars":4000,"mutating_tools":"write_file patch terminal execute_code todo memory skill_manage read_file"}'

On Windows PowerShell the inner quotes need escaping: --chat-template-kwargs '{\"tool_call_format\":\"json\"}'.

Files

• chat_template.jinja / chat_template_oneline.txt — the template (multi-line and single-line)
• tests/test_template.py — render-test suite (run after any edit)
• scripts/measure_loop_nudges.py — reproduce the loop-nudge metric on your own opencode.db
• archive/ — every prior version plus a changelog table

llama.cpp notes

• Grammar enforcement is active on auto-parser builds (b8227+) — malformed tool calls are token-level impossible (verified to 163 tool schemas). It cannot stop max_tokens truncation, so keep your output limit above reasoning budget + largest tool call.
• String arguments are normalized by mainline llama.cpp. Harnesses like OpenCode send historical tool-call arguments as JSON strings, but current mainline parses them into objects in C++ (func_args_not_string, common/chat.cpp) before the template renders — so they arrive as mappings and render as clean <parameter=> blocks. On renderers without that normalization (older forks, direct Jinja), string arguments render inside a labeled <arguments_json> compatibility block instead of masquerading as XML syntax, and the instructions tell the model never to emit that block in new calls.
• JSON mode drops grammar enforcement. The auto-parser only grammar-constrains the XML format (verified: tool_choice: required is not enforced in JSON mode). On a measured A/B with identical planning prompts, JSON mode also caused a 26–73× reasoning-length blowup on this model class — XML is the default for a reason.
• Media marker: the template strips <__media__> only from system/assistant/reasoning content (roles that cannot hold a real image); user/tool markers are left intact so vision and tool screenshots work. A literal marker in user/tool text (for example, pasting this template into chat) 400s the server and cannot be fixed in-template — on mainline llama.cpp set LLAMA_MEDIA_MARKER; ik_llama does not support it, so avoid pasting the raw token.

Design notes

Rationale that used to live in template comments (the template itself is now comment-free; the single-line variant always was):

• Media-marker guards. llama.cpp flattens images into string content as bitmap-backed marker tokens. The guards test for the marker's presence first and never touch normal content. They strip only from roles that cannot legitimately hold an image; stripping user/tool markers would desync markers vs. bitmaps and 400 the server (a v3/v7 unconditional strip silently broke real vision and was reverted in v8). Automated reviewers: HTML-interpreting fetchers eat the marker token and make the guards look like if '' in content — review the raw file.
• Why the no-progress detector skips mutating tools: identical "success" strings from different writes are productive work; identical results from read/query tools are the genuine stuck signal. Empty/unknown tool names count conservatively.
• Why the ping-pong detector does not skip mutating tools: the same write with the same arguments is a loop even though its result says "success."
• Why call signatures are never truncated: two writes can differ only deep in their content; a truncated signature would false-match them. Signatures are compare-only and never rendered, so they cost zero prompt tokens.
• Why tool output is re-rendered untrimmed: leading/trailing whitespace in diffs, Makefile tabs, and exact compiler output are significant. Head+tail truncation (when max_tool_response_chars is set) keeps both ends because coding output puts the actionable summary at the end.
• Why failure detection distinguishes harness envelopes from generic markers: the harness saying invalid input for tool is a real failure for any tool; content starting with Error: is only a failure signal for tools that execute things. Tool failure is undecidable from content alone, so classification is by tool class.
• Why the vanished-tools note exists: models do not notice their own tool list shrinking; they improvise around missing capability instead of reporting it. Structural detection converts "hope the model notices" into "tell it."

Changelog

Latest first; full detail per version lives in archive/.

• v13 — Per-result tool attribution. Mixed parallel tool-call batches now resolve each result's function from its tool_call_id before applying the read/mutating classifications, removing false failure warnings on parallel reads. Falls back to the previous behaviour when no usable id is present.
• v12 — Two-sided turn management. Added a continuation rule (keep working until complete; verify then move on; prefer small concrete actions over long deliberation) and replaced the every-turn stop-invitation with bounded exit conditions. Tuned across field sessions to avoid both premature stops and reasoning spirals — the single highest-gain wording lever in the template.
• v11 — Language release. Research-driven rewording of every model-visible phrase (negation backfires — "pink elephant" effect; neutral tone; absolutes only where literally true): warnings now let the model self-classify (expected failure mid-fix vs. real failure), consecutive-error warnings split on evidence (identical repeated error vs. changing errors/burn-down), the no-progress warning gained a polling escape hatch, and all instructions are positively framed. A permanent negation-audit test keeps prohibitive phrasings from returning.
• v10 — Truthfulness release. Read-tool exemption for content-based failure markers (read_tools kwarg); <arguments_json> compatibility wrapper for renderers without C++ argument normalization, with max_tool_arg_chars truncation; argument-grounding rules in the tool instructions; vanished-tools note reworded to allow safe continuation with remaining tools; ping-pong warning reworded to remove a false absolute and add a polling escape hatch; template comments removed (design rationale moved to this README).
• v9 — Two new structural detectors, both field-validated: ping-pong/action-loop (call-signature repeats catch loops whose results differ every turn) and vanished-tools (a tool called in history but missing from tools[] gets a system note naming it). New kwargs pingpong_nudge_after, warn_vanished_tools.
• v8 — Media-marker correctness: strip only from system/assistant/reasoning; preserve user/tool markers (a v3/v7 unconditional strip silently broke real vision and tool screenshots).
• v7 — Runtime hardening: repeat_nudge_after=0 fix, tool-output fidelity, long-error detection, head+tail truncation. Added the test cases and repro script.
• v6 — Cross-harness mutating_tools union (16+ harnesses), case-insensitive. Zero prompt-token cost (never rendered).
• v5 — Loop-detector rework on 381 real sessions: removed the noisy same-tool-streak detector; no-progress detector now skips mutating tools. ~90% fewer nudges, every real catch kept.
• v4 — Tool-envelope unwrapping (~6.5k tokens saved at 163 tools); self-healing spilled tool calls; think_on_tool_failure kwarg; emoji-free warnings.
• v3 — Media-marker injection guard (fixes ik_llama tokenize 400 from hallucinated/fetched markers).
• v2 — Date awareness via strftime_now (guarded; no-op on runtimes without it).
• v1 — Initial release: froggeric v21.3 base + loop detectors + error markers + kwargs.

Credits and license

• Base template: froggeric/Qwen-Fixed-Chat-Templates v21.3. All upstream fixes are inherited; please report template-core issues upstream.
• Agentic layer: Moore2877, developed with AI assistance. Report agentic-layer issues in this repository.
• License: Apache-2.0.