---
quantized_by: gghfez
pipeline_tag: text-generation
base_model:
- moonshotai/Kimi-K2.7-Code
license: other
license_name: modified-mit
license_link: https://huggingface.co/moonshotai/Kimi-K2.7/blob/main/LICENSE
base_model_relation: quantized
tags:
- mla
- imatrix
- conversational
- ik_llama.cpp
---



## imatrix Quantization of moonshotai/Kimi-K2.7

ik_llama.cpp quants of moonshotai/Kimi-K2.7 using Unsloth's imatrix and Ubergarm's quant recipes*.
*embedding and output tensors left at q8_0

The other quants in this collection **REQUIRE** [ik_llama.cpp](https://github.com/ikawrakow/ik_llama.cpp/) fork to support the ik's latest SOTA quants and optimizations! Do **not** download these big files and expect them to run on mainline vanilla llama.cpp, ollama, LM Studio, KoboldCpp, etc!
*NOTE* `ik_llama.cpp` can also run your existing GGUFs from AesSedai, unsloth, bartowski, mradermacher, etc

Some of ik's new quants are supported with [Nexesenex/croco.cpp](https://github.com/Nexesenex/croco.cpp) fork of KoboldCPP with Windows builds for CUDA 12.9. Also check for [Windows builds by Thireus here.](https://github.com/Thireus/ik_llama.cpp/releases) which have been CUDA 12.8.

These quants provide best in class perplexity for the given memory footprint.

The IQ2_KT is the most accurate 2-bit Kimi-K2.7-Code quant I've found on huggingface but it's slower to run.

### Available quants

**IQ2_KT - 264.5 GiB**

Final estimate: PPL over 568 chunks for n_ctx=512 = 2.8960 +/- 0.01474 (+44.14% vs baseline)

**IQ2_KS - 270.9 GiB**

Final estimate: PPL over 568 chunks for n_ctx=512 = 2.9740 +/- 0.01518 (+48.02% vs baseline)

**IQ2_KL - 329.7 GiB**

Final estimate: PPL over 568 chunks for n_ctx=512 = 2.4417 +/- 0.01166 (+21.52% vs baseline)

**IQ3_KT - 381.8 GiB**

PPL Untested / don't have the hardware. Responds coherently to a few prompts.

## References
* [ik_llama.cpp](https://github.com/ikawrakow/ik_llama.cpp)
* [Great mainline MoE optimizd quants AesSedai/Kimi-K2.7-GGUF](https://huggingface.co/AesSedai/Kimi-K2.7-GGUF)

## ACK

* Original Imatrix from [Unsloth/Kimi-K2.7-Code-GGUF](https://huggingface.co/unsloth/Kimi-K2.7-Code-GGUF/blob/main/imatrix_unsloth.gguf_file) converted via https://gghfez-ik-llama-imatrix-converter.hf.space/
* Quant Recipes and parts of the README.md based off [ubergarm/Kimi-K2.6-GGUF](https://huggingface.co/ubergarm/Kimi-K2.6-GGUF)