nerun/PentaCoder-9B-i1-MTP-GGUF

PentaCoder-9B i1 GGUF with embedded MTP draft

This repository provides GGUF files of pragmaticcs/PentaCoder-9B-i1-GGUF with an embedded MTP (Multi-Token Prediction) block, ready for speculative decoding in llama.cpp with --spec-type draft-mtp.

PentaCoder-9B is a Qwen3.5-9B-based merge that ships without MTP tensors. The MTP block was transplanted (copied tensor by tensor, with no training) from a file of a4lg/Qwen3.5-9B-MTP-ONLY-GGUF, itself extracted from the original Qwen3.5-9B.

Benchmark it on your own machine before trusting the numbers below.

What this is (and is not)

• Main model: the imatrix quants PentaCoder-9B.i1-*.gguf by mradermacher, copied tensor by tensor, unchanged.
• MTP block (blk.32.*): taken from a file of a4lg/Qwen3.5-9B-MTP-ONLY-GGUF, an MTP-only subset of Qwen/Qwen3.5-9B. The quant of the donor file used for each file is listed in the table under "Files".
• The MTP head was trained on Qwen3.5-9B, not on PentaCoder-9B. It still drafts well for this model (acceptance ~0.6–0.7 in my tests), but it is not a PentaCoder-specific draft head.
• In the grafted files the MTP block shares token_embd and output with PentaCoder, so each file is only slightly larger than the original quant and uses less VRAM than loading a separate draft model.

Files

File	Quant main	Quant MTP	Size
PentaCoder-9B.i1-IQ4_XS-MTP.gguf	i1-IQ4_XS	Q4_K_M	5.0 GiB (~5.4 GB)

The MTP block adds about 0.2 GiB (4.8 GiB without MTP -> 5.0 GiB for this file).

No vision projector (mmproj) is provided; use the model as text-only.

Usage

Requires a llama.cpp build with MTP draft support (--spec-type draft-mtp).

# Replace <QUANT> with the quant of the file you downloaded
llama-server \
    -m PentaCoder-9B.i1-<QUANT>-MTP.gguf \
    -ngl auto -c 0 -fit on -fitc 32768 -fa on -np 1 \
    -ctk q8_0 -ctv q8_0 \
    --temp 0.6 --top-k 20 --min-p 0.0 \
    --spec-type draft-mtp --spec-draft-n-max 2

Use the grafted file as the main model. Do not pass it again with --model-draft.

Notes:

• In llama.cpp --spec-draft-n-max defaults to 3. In my tests 2 was consistently a little better than 3; values of 1 or 4 and above were slower (see below).
• On 8 GB of VRAM, keep the default micro-batch (-ub 512). With -ub 1024 or -ub 2048, generation with MTP dropped to ~22 t/s, probably because the larger compute buffer left less VRAM for the model layers under -fit on.

Benchmarks

Same prompt and settings for all rows: a short prose question (Explain why the sky is blue). --spec-draft-n-max 2 and 3: mean ± sample standard deviation of 10 runs (server restarted before each run). Other rows, including the baseline: a single run. Measured with the IQ4_XS; other quants may behave differently.

• GPU: AMD Radeon RX 6600M (Navi 23, 8 GB) on a desktop board
• CPU: AMD Ryzen 5 5000 series, 32 GB DDR4-3200
• OS: Debian GNU/Linux Forky/testing
• llama.cpp build: b12230

Setting	Generation (t/s)	Draft acceptance
No MTP (baseline)	37.05	—
--spec-draft-n-max 1	49.95	0.81
--spec-draft-n-max 2	54.07 ± 0.70	0.6781 ± 0.0161
--spec-draft-n-max 3 (default)	51.57 ± 1.51	0.5566 ± 0.0266
--spec-draft-n-max 4	49.32	0.51
--spec-draft-n-max 6	36.63	0.33

About +46% generation speed over the same file without MTP, with --spec-draft-n-max 2 (mean of 10 runs against a single baseline run). With the default --spec-draft-n-max 3 the gain is about +39%. Results depend heavily on hardware, backend, prompt type and settings.

How it was made

The MTP block was transplanted with convert.py by @buzz, following "Method 2" of the a4lg README:

# ./convert.py TARGET DONOR OUTPUT
python3 convert.py \
    PentaCoder-9B.i1-<QUANT>.gguf \
    Qwen3.5-9B-MTP-ONLY-<DONOR_QUANT>.gguf \
    PentaCoder-9B.i1-<QUANT>-MTP.gguf

The script as published did not work for the qwen35 architecture. It raises block_count from 32 to 33 but copies per-layer array keys unchanged, so llama.cpp refused the output with:

key qwen35.attention.recurrent_layers has wrong array length; expected 33, got 32

The donor file does not carry that key, so I extended per-layer arrays in the loop that writes the target's KVs, appending False for the MTP block (it is a full-attention layer: attn_q/k/v/output, no ssm_*):

contents = field.contents()
# Per-layer arrays have one entry per block: add an entry for the MTP block
if len(contents) == target_block_count:
    src_field = source_reader.get_field(key)
    if src_field is not None and len(src_field.contents()) == source_block_count:
        extra = list(src_field.contents())[target_block_count:]
    elif key == f"{arch}.attention.recurrent_layers":
        # Donor lacks the key; the MTP block uses full attention
        extra = [False]
    else:
        extra = []
    contents = list(contents) + extra
if sub_type == GGUFValueType.BOOL:
    contents = [int(x) for x in contents]
write_array_value(fout, sub_type, contents)

Resulting metadata:

qwen35.block_count = 33
qwen35.nextn_predict_layers = 1
qwen35.attention.recurrent_layers = [... 33 entries, last one False]

Updated script convert.py.

Limitations

• Only the quants listed under "Files" were grafted and tested.
• Only tested with the llama.cpp Vulkan backend on one AMD GPU.
• Speculative decoding with MTP does not apply to image input.

License and Copyright

All GGUF files in this repository are distributed under the Apache License, Version 2.0, the license of both Qwen/Qwen3.5-9B and pragmaticcs/PentaCoder-9B-i1-GGUF. A copy of the license is included in the LICENSE file.

Copyright 2026 Alibaba Cloud (Qwen3.5-9B, including the MTP head)

Modification notice: the files were produced by mechanically combining tensors from the two GGUF files listed above and adjusting the metadata described in "How it was made". No weights were changed or trained.

This README file is licensed under the terms of CC-BY-4.0. It is partly based on the README of a4lg/Qwen3.5-9B-MTP-ONLY-GGUF (CC-BY-4.0, Copyright 2026 a4lg).

Copyright 2026 Daniel Dias Rodrigues (Nerun).

Credits

• Qwen team / Alibaba Cloud — Qwen3.5-9B and its MTP head
• pragmaticcs — PentaCoder-9B
• mradermacher — imatrix GGUF quants of PentaCoder-9B
• a4lg — MTP-only GGUF subset of Qwen3.5-9B
• @buzz — convert.py MTP transplant script

Links: Sources

• Original model: pragmaticcs/PentaCoder-9B-i1-GGUF
• Main quants: mradermacher/PentaCoder-9B-i1-GGUF
• MTP donor: a4lg/Qwen3.5-9B-MTP-ONLY-GGUF
• Base of the MTP head: Qwen/Qwen3.5-9B
• Graft script: buzz/convert.py