larger file size for same quant
#4
by CHNtentes - opened
q4_k_xl for m2.5 is 131gb, while m2.7 is 141gb
Oh the old Q4_K_XL is different :) Our new method guarantees always _XL is always bigger than _M.
I would use Q4_K_M which is also dynamic!
@shimmyshimmer thank you for all the quick work you are always doing.
It would be great if you could write-up at unsloth.ai what all the suffixes mean, and how to pick a quant, especially since you've updated your methods. Also, apologies if that is written somewhere and I've missed it.
I've always just picked the XL, assuming bigger is better, and hoping the accuracy is better.
How come you are recommending Q4_K_M in this case? 😀