Very specific json formatting issue in tool calls
With opencode (at least), this model exhibits a very frequent and specific issue. It outputs:
"filePath"/home/username
rather than:
"filePath": "/home/username
thereby breaking the json syntax and the tool call. I've seen this in two different Q6 GGUF variants with llama.cpp, and also in a AWQ 4-bit variant with vLLM, so I think it's in the model itself, though it could be very repeatable quantization artifact.
If @JoeSmith245 is using the provided jinja chat template and the recommended tool calling parser the model wouldn't directly generate json output.
From the jinja chat template:
If you choose to call a function ONLY reply in the following format with NO suffix:
<tool_call>
<function=example_function_name>
<parameter=example_parameter_1>
value_1
</parameter>
<parameter=example_parameter_2>
This is the value for the second parameter
that can span
multiple lines
</parameter>
</function>
</tool_call>
"filePath"/home/username has to come imo from the tool parser. llama.cpp bug? I briefly checked the relevant source and it seems that llama.cpp already does grammar-based sampling of tool calls (ofc requires --jinjain the llama-server launch command). The model imo has no other choice than to produce a syntactically correct tool call, so I am really not sure how the posted output is possible.