OrionLLM

/

GRM-2.6-Plus-0628

like 35

Follow

Orion LLM Labs 218

First of all: this is the best I ever ran locally!

But I can't get it to call tools right.
Very often it calls tools inside reasoning_content.
I run with llama.cpp. Tried 3 different quants, the same thing.
I use chat template from your repo.
Relevant parameters:
--chat-template-file .../chat_template.jinja
--reasoning-format deepseek (tried without it - no difference)
--chat-template-kwargs "{"preserve_thinking": true}"
--temp 0.7
--top-p 0.95
--top-k 20
--min-p 0.0
--presence-penalty 0.0
--repeat-penalty 1.0

Any thoughts?

Thanks for the pointer! Assuming you mean froggeric/Qwen-Fixed-Chat-Templates (831 likes, the main community one for 3.5/3.6) — I pulled it and compared line by line.

Didn't do a full swap, for a couple reasons: it's a generic Qwen3.5/3.6 template and GRM-2.6-Plus is a finetune with its own behavior in the agentic loop (plans ahead, doesn't re-reason on every call), so a one-size-fits-all template would've cost some of that control. Also their auto_disable_thinking_with_tools kills reasoning session-wide the moment tools are present — I wanted it scoped to just the turns right after a tool_response, so steps that genuinely need longer reasoning still get it.

That said, two things from their template were straight-up better than my first patch, so I pulled them in:

• they guard against injecting an empty into history when there's no real reasoning content — I wasn't doing that, and it was quietly teaching the model that "empty think → tool call" is the expected pattern
• they detect closing-tag variants (, malformed spacing, etc.), not just the exact

My fallback for when is missing entirely and shows up raw in content — that one they don't have, so I kept mine.

Updated chat_template.jinja is up now. Appreciate you flagging it, this ecosystem-comparison stuff is genuinely useful.

Why dont you apply the qwen 3.6 fixed chat templates, dont have the link handy right now.

Why dont you apply the qwen 3.6 fixed chat templates, dont have the link handy right now.

I just pulled it rn, thanks!