Used quants, but model is not recognized to support tools, though it does
#1
by JLouisBiz - opened
I don't know details. I have used iquants from mrradermacher
Used quants, but model is not recognized to support tools, though it does. Maybe you need to do some settings.
Hello JLouisBiz, thank you for contacting us!
It's likely that the quantized versions of mrradermacher aren't activating the tools for this model.
Which platform are you using? Ollama, LM Studios, if you tell me, we can try to help you!
Anyway, if you're going to use quantization, I recommend using LM Studios, as it's efficient, supports Hugging Face GGUFs, and already has integrated tools!
GRM2 has been optimized for tool use, being one of the most efficient models for tools up to 3b params.