Odd issues with v21.3 with tool calling that didn't happen with previous version
I had been using the previous chat template (from May maybe?) with the minor edit in that important block to fix the stray think that would sometimes show up:
<IMPORTANT>
Reminder:
- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.
- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.
- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.
- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY after thinking, with NO conversational text before it.
- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.
- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.
- If you have all necessary data, provide your final answer directly to the user without any tool call.
</IMPORTANT>
With this template, my tool calls had been >95% success rate. I use a Q5_K_S quantization of a qwen3.6-27b finetune at 200k context with kv at q8_0. This is on the latest llama.cpp beta build (main branch/main llama.cpp) on CUDA with a 5090. I would not get looping, opencode was my main usage and have coded probably thousands of hours with this template and quantization/model over these last several months.
In addition, i run the Hermes Agent, and have a handful of crons that give me briefings by doing lots of internet research using MCPs that i self-host. SearXNG, Camofox, Firecrawl, HuggingFace, etc. On the earlier chat template, i'd sometimes get failures because the crons are basically one-shot instructions for the model to do it and if it messes up then it basically blows up due to harness rules. So probably 5-10% failure rates here, but generally would nail it if we'd potentially adjust the cron/skill to clarify/simplify things.
After swapping to the v21.3 template and using that as the daily driver, things have degraded a lot. i had tested setting the tool_call_format to json, but that was worse. Now i've got loops happening all the time in hermes. weird things where hermes will want to add memories about everything and loop there even though memories are not part of any request. my briefing crons will fail >75% of the time when it's more complicated tool calls because the tool calls will loop.
I haven't spent as much time in opencode with this new chat template over the last week so i cannot comment on that side of it, but as soon as i swap back over to the previous template build, my hermes agent has no problem with tool calls and getting all the data i need. No llama-server parameter changes, no hermes config changes - just the template swap.
I also use litellm so i see logs of when things go off the rails, but it's difficult for me to make sense of it because basically it will either loop until it hits the loop guard of tooling, or in some cases if it's not an actual tool call, it'll loop until it burns through 65k output tokens and hits max output before failing.
If there's anything i can do to help debug, i just need a bit of guidance since this is all a little over my head.