Is it possible to quantise this model using GGUF?
#8
by shrikrishnaholla - opened
I am interested in using this using Ollama. However, GGUF my Repo doesn't seem to be able to detect / convert it. Has anyone tried to quantise this model? How would one approach it?