Tool call stops in middle of the conversation, doesn't finish the complete task
Hi,
I am hosting the model on vllm, using this docker image - vllm/vllm-openai:cu130-nightly, everything works great, but I noticed when I give a task which has multiple tool calls, it stop in middle doesnt complete the task, my agent gets empty response from the model although the task is not completed. I get finish reason stop-
INFO:codingagent:[Agent] LLM response: finish_reason=stop, tool_calls=0, content_preview=''
although the task is not completed, then if I send a message like continue, it continues with next steps, could you let me know how do I fix this issue, the normal messaging works really great, no issues even with huge conversation history.
Thanks
y same Issue on Ollama install, but still the best local model to run^^ on normal GPU
I noticed, giving a nudge to the agent, makes it continue, check for empty response and add a message like continue automatically, that fixes this issue, no need to manually enter command/prompt to continue. I have built a totally sick agent eco system with this model, https://youtu.be/5IMHFsERlGg , worth it, just host it on cheap 4090 for $0.2/hr , and keep it running, after a long time I saw one such model. Thanks to qwen team.
Using a custom Jinja template worked for me. See:
https://github.com/ggml-org/llama.cpp/issues/20837#issuecomment-4218130554