Q6 & Q8
#1
by
RuneXX - opened
Any chance for higher quants?
And thanks for those already there, works great ;-)
thanks a ton ;-) I know Q5_K is usually plenty ... But nice to have the Q6 and Q8 as options
Any chance for higher quants?
And thanks for those already there, works great ;-)
? the full model is only 20gb anyways so why?
not sure why, but my end the GGUF models seems to work better memory wise.
Often prefer the gguf ones