GGUF version ?
#2
by notzero - opened
Hi, is there gguf version for this ?
Thanks
Yes, there is a GGUF version available for this. You can find it at the following link: LDCC-SOLAR-10.7B-GGUF.
@krevas Thanks
Edit : Sorry, do you have smaller quants ? I only have M2 16gb ram, or can you tell me how to quants it to gguf ? I think I tried it before and not working
Edit2 : I can just quantize it myself then using your fp16.gguf, sorry :)
notzero changed discussion status to closed
I've also uploaded the quantization model.