Model

This is a 3.35bpw / IQ3_M quantization of moonshotai/Kimi-K2.5. The other quantization in the following table are in this repo: https://huggingface.co/AesSedai/Kimi-K2.5-GGUF

Quant	Size	Mixture	PPL	1-(Mean PPL(Q)/PPL(base))	KLD
Q4_X	543.62 GiB (4.55 BPW)	Q8_0 / Q4_0	1.8248 +/- 0.00699	0	0
IQ3_M	399.94 GiB (3.35 BPW)	Q8_0 / varies	2.036651 ± 0.008187	+11.6890%	0.119262 ± 0.000856
IQ3_S	377.50 GiB (3.16 BPW)	Q8_0 / varies	2.116713 ± 0.008620	+16.0796%	0.158551 ± 0.001084
IQ2_S	311.71 GiB (2.61 BPW)	Q8_0 / varies	2.433594 ± 0.010455	+33.4572%	0.294937 ± 0.001721
IQ2_XXS	262.74 GiB (2.20 BPW)	Q8_0 / varies	3.119876 ± 0.014508	+71.0926%	0.540149 ± 0.002570

GGUF

Model size

1T params

Architecture

deepseek2

Hardware compatibility

3-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Panchovix/Kimi-K2.5-GGUF

Base model

Quantized

(38)

this model