Q6 & Q8

#1
by RuneXX - opened

Any chance for higher quants?

And thanks for those already there, works great ;-)

@RuneXX good to hear it works.
uploaded both q6_k and q8_0 now.

thanks a ton ;-) I know Q5_K is usually plenty ... But nice to have the Q6 and Q8 as options

Any chance for higher quants?

And thanks for those already there, works great ;-)

? the full model is only 20gb anyways so why?

not sure why, but my end the GGUF models seems to work better memory wise.
Often prefer the gguf ones

Sign up or log in to comment