The K_G_3.50 quant seems to be outperforming the UD IQ4_NL quant. Whats the difference between v1 and v2?
I see you uploaded a v2 whats the difference?
I see you uploaded a v2 whats the difference?
I fixed the one tensor mentioned in this post to always be Q6_K instead of Q4_K or Q5_K on the v2 quants - https://old.reddit.com/r/LocalLLaMA/comments/1slk4di/minimax_m27_gguf_investigation_fixes_benchmarks/
It might not be necessary as I didn't see NaN when I tested them, but it's just in case. I will be removing the old ones and renaming v2 for them.
will redownload
I see you uploaded a v2 whats the difference?
I fixed the one tensor mentioned in this post to always be Q6_K instead of Q4_K or Q5_K on the v2 quants - https://old.reddit.com/r/LocalLLaMA/comments/1slk4di/minimax_m27_gguf_investigation_fixes_benchmarks/
It might not be necessary as I didn't see NaN when I tested them, but it's just in case. I will be removing the old ones and renaming v2 for them.
v2 passes the vibe check it does feel smarter. Nice work this is both smaller than the UD IQ4_NL quant and smarter.
It gives alot more nuance in its answers compared to UD IQ4_NL. How did you quant it so good this shits good lol.
It gives alot more nuance in its answers compared to UD IQ4_NL. How did you quant it so good this shits good lol.
I am actually not a big fan of the IQ4_NL or IQ4_XS tensors since they gave me less interesting outputs in inference time
The quant types I consider useful are: IQ1_S, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, Q3_K, Q4_K, Q5_K, Q6_K
My quants usually use at least 3 different quant levels distributed across the tensors depending on how important/unimportant they are. More important tensors are getting higher than average quant level compared to traditional quants while the less sensitive tensors are actually being downgraded more than usual to fit the bpw budget.
That said, Minimax did not seem to quant nearly as well as my Qwen 3.5 tunes if you compare the KLD levels. And the comparisons with Unsloth's UD quants on this one is not that big of a disparity in KLD.
This quant has felt much better than the UD IQ4_NL and UD Q3_K_XL quants I tested. IQ4_NL will spew more confident sounding BS and likes to format responses weirdly where as the 3.50 bpw quant is very solid.
yet again this quant still does much better than the iq4_nl. The iq4_nl kept searching the same typo it made over and over while the 3.5 bpw caught the typo and correctly completed the search.