---
license: other
license_name: tencent-hunyuan-community
base_model: tencent/Hy3-preview
tags:
- mlx
- speculative-decoding
- mtp
- multi-token-prediction
- hunyuan
library_name: mlx
---

# Hy3-preview-MTP-4bit

Native **Multi-Token-Prediction (MTP)** sidecar for
[`mlx-community/Hy3-preview-4bit`](https://huggingface.co/mlx-community/Hy3-preview-4bit),
for use as a self-speculative draft head with
[rapid-mlx](https://github.com/machinefi/rapid-mlx).

## What this is

Tencent's Hunyuan 3 (`model_type=hy_v3`) ships a **DeepSeek-V3-style native MTP
head** as the final decoder layer (`model.layers.80.*`) of the full-precision
[`tencent/Hy3-preview`](https://huggingface.co/tencent/Hy3-preview) checkpoint.
The 4-bit MLX conversion `mlx-community/Hy3-preview-4bit` keeps only layers
`0..79` (the backbone) and **strips** the MTP head.

This repo re-supplies just that head, quantized to match the base checkpoint,
as a single-file sidecar (`model-mtp.safetensors`, 44 tensors). rapid-mlx loads
the base 4-bit backbone and grafts this head at boot to run **self-speculative
decoding** (one draft token per verify step, K=1 chain MTP).

## Provenance

Extracted from `tencent/Hy3-preview` shards `model-00111-of-00112` +
`model-00112-of-00112` (the two shards holding layer 80). The 593 layer-80
tensors are remapped to the rapid-mlx MTP param tree:

* `enorm` / `hnorm` — RMSNorms on the next-token embedding and previous hidden
  state (DeepSeek-V3 convention).
* `eh_proj` — the `2H -> H` fused projection, applied as
  `eh_proj(concat([enorm(embed_next), hnorm(prev_hidden)], -1))` (embedding
  first, confirmed against vLLM `deepseek_mtp.py` and SGLang `hunyuan` nextn).
* one HY3 `DecoderLayer` on the MoE branch (QK-norm attention + sigmoid-router
  SwitchGLU MoE over 192 experts + a shared expert).
* `norm` — the head's final RMSNorm (upstream `final_layernorm`).

## Quantization

Matches the base checkpoint: **4-bit `group_size=64` affine** for every Linear
(`eh_proj`, attention projections, `switch_mlp.*`, `shared_mlp.*`); **8-bit
`group_size=64`** for `mlp.router.gate`; all RMSNorms and `router.expert_bias`
kept in full precision.

## Usage

```bash
rapid-mlx serve hy3-preview-4bit --speculative-config '{"method":"mtp"}'
```

rapid-mlx auto-resolves and downloads this sidecar. The base 4-bit backbone
loads normally; the MTP head is grafted at boot for self-speculative decoding.

## Measured

* Draft accept rate ~58% (K=1) across code / chinese / reasoning / list prompts.
* Greedy output is batched-consistent lossless vs the MTP-off reference.

The projection tree is byte-identical to a quantized backbone MoE layer, so the
sidecar param names line up 1:1 with the runtime module.