---
tags:
- qwen
- qwen3_5_moe
- gguf
- mtp
- text-to-text
- conversational
- text-generation-inference
base_model:
- Kwaipilot/KAT-Coder-V2.5-Dev
- unsloth/Qwen3.6-35B-A3B-MTP-GGUF
notes:
- based on GGUF format quantizations by bartowski and unsloth
---

## About

What you probably want instead is one of the [APEX][apex] `gguf` builds by [gbuzhf][gbuzhf]: 
 * [gbuzhf/KAT-Coder-V2.5-Dev-APEX-MTP-GGUF][gbuzhf]

However, sometimes you want to see if a more classic quantization will be more suitable for your specific task, or simply want to fit (or use) a specific vram target.
In this case, "one can't go wrong with [bartowski][bartowski] builds (c) reddit" -- except for these are sadly missing any draft models, 
and a 20-30 %% gain in inference speed is not something to easily give up on.

Hence, I'll be adding here some "franken-merges" of original [bartowski builds][bartowski] with 20 MTP layers added to it from [unsloth/Qwen3.6-35B-A3B-MTP-GGUF][unsloth] model 
using a [conversion script] made by [buzz][buzz].

All uploaded models were tested and add a ~20-30 % gain in inference speed compared to original ("ungrafted") models on AMD hardware.

(It is strongly recommended to use at most `q8_0` quantizations for KV cache if your hardware supoorts it, especially if it has a chance to be more performant on it.)

// TODO: add invocation examples //

[apex]: https://github.com/localai-org/apex-quant
[gbuzhf]: https://huggingface.co/gbuzhf/KAT-Coder-V2.5-Dev-APEX-MTP-GGUF
[bartowski]: https://huggingface.co/bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF
[unsloth]: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/blob/main/Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf
[buzz]: https://gist.github.com/buzz/1c439684d5e3f36492ae9f64ef7e3f67