ox-ox

/

MiniMax-M2.5-GGUF

like 14

Q4_K_M

Sir I would love a 5-bit KL or KM quant if you have the spare compute to make that happen. Just downloaded your 4-bit quant for M2.5 and am enjoying be able to run this awesome model locally. Thanks for making it!

Q4_K_M

He's got you covered! :-)

I'm running this one now btw. So far it's excellent.

https://huggingface.co/ox-ox/MiniMax-M2.5-GGUF/blob/main/minimax-m2.5-Q4_K_M.gguf

4bit AWQ

GPU poor quant. Q2 please for 96GB M3 Ultra :)