README.md

---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-9B
- nightmedia/Qwen3.5-9B-Holodeck-Lounge
- Jackrong/Qwopus3.5-9B-Coder
tags:
- multi-token-prediction
- speculative-decoding
- qwen3.5
- mtp
- merge
---

# Qwen3.5-9B-Holodeck-Lounge (MTP-restored)

> **TL;DR** A working **native Multi-Token Prediction (MTP) head** on top of the
> Holodeck-Lounge merge. The upstream merge lineage shipped **without a
> functional MTP head** (the `mtp.fc.weight` tensor is absent after merge), so
> we restored it by sourcing only that **single missing tensor** —
> `mtp.fc.weight` — from
> [Jackrong/Qwopus3.5-9B-Coder](https://huggingface.co/Jackrong/Qwopus3.5-9B-Coder).
> The other **14 of the MTP head's 15 layers** (the `mtp.layers.*` blocks) were
> already present in the merge. We also patched in the
> [froggeric/Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates)
> fixed jinja template (**v22.1**), which improves reasoning
> structure and MTP acceptance in our evals. This model is the reference donor for our
> [head fine-tune experiments](https://huggingface.co/Wiself/Holodeck-Lounge-Regressed-MTP).

## Why this model exists

- [nightmedia/Qwen3.5-9B-Holodeck-Lounge](https://huggingface.co/nightmedia/Qwen3.5-9B-Holodeck-Lounge)
  is a fine creative-fiction merge. Excellent writing model — but its merge
  lineage (13 base models, see the upstream card for the full recipe) does not
  carry a working MTP head: `mtp.fc.weight` is missing, so native speculative
  decoding is impossible from the source.
- **What we did:** the merge already carried **14 of the MTP head's 15 layers**
  (the `mtp.layers.*` transformer blocks); only the input-fusion projection
  `mtp.fc.weight` was absent. We sourced that **single tensor** from
  **Jackrong/Qwopus3.5-9B-Coder**, transplanted it into the Holodeck-Lounge
  backbone, and verified it loads and runs end-to-end in llama.cpp
  (`draft-mtp`) and vLLM (`qwen3_5` MTP method). Backbone weights untouched.
- **Chat template:** baked in the fixed jinja template v22.1 from
  [froggeric/Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates)
  (Qwen 3.5/3.6/3.8), improving reasoning
  structure, organisation, coherence, and MTP acceptance rate in our evals.

## Native MTP head

- Single-layer, DeepSeek-style MTP head (as in the
  [DeepSeek-V3 Technical Report](https://arxiv.org/abs/2412.19437)),
  **3 draft tokens**, fixed adjacent context.
- The 14 `mtp.layers.*` blocks come from the Holodeck-Lounge merge; only the
  `mtp.fc.weight` fusion projection was sourced from
  Jackrong/Qwopus3.5-9B-Coder (a model whose checkpoint carries the same
  native Qwen3.5 MTP architecture).
- The donor projection plus the 14 host layers reach the acceptance range
  below — higher than any of our head-only fine-tunes
  ([related experiment →](https://huggingface.co/Wiself/Holodeck-Lounge-Regressed-MTP)).

## Evaluation

| Metric | Value |
|---|---|
| llama.cpp greedy / target-only acceptance | **59.0%** |
| vLLM rejection-sampling acceptance | **58.55%** |
| RS pos0 / pos1 / pos2 | 77.2% / 57.0% / 41.4% |
| Mean accepted length | 2.756 |

Eval: 30 prompts, temperature 0.7, max 256 tokens, concurrency 8. llama.cpp
numbers are greedy target-only acceptance (reasoning off); vLLM RS acceptance
from `vllm:spec_decode_num_accepted_tokens / _num_draft_tokens`. With
reasoning enabled per the Usage section, measured acceptance reaches the
**60–69%** range (n=30, same eval set).

> **Eval caveat:** n=30 is a small sample — treat the 3-decimal precision as
> indicative; confidence intervals are wide.

## Real-world usage

With the recommended settings — `draft-mtp` in llama.cpp, `qwen3_5` MTP in
vLLM — the restored head has consistently delivered a **60–69% draft-token
acceptance rate** (65+% typical) in our testing. Real-world mileage will vary with workload
and hardware; treat 65% as a strong baseline, not a guarantee. The point of
this model isn't raw speed, but **option**: MTP when it pays off, plain greedy
decoding when it doesn't.

Where the win matters most:

- Bandwidth-constrained serving, where accepted draft tokens cut
  tokens-per-request at the wire.
- High-concurrency, latency-sensitive inference, where saved decode steps
  shrink queue and p95.
- Settings where every accepted draft token compounds — long generations,
  reasoning-heavy prompts, tight SLOs.

The clean part: the host checkpoint already carried **14 of the MTP head's 15
layers**, so this model ships at essentially the same size as its host — it was
always that big. The single-tensor transplant just completes the missing
`mtp.fc.weight` fusion layer, so you get the choice in one model, at no real
size cost.

Thanks to nightmedia for the Holodeck-Lounge backbone (which already carried 14
of the MTP head's 15 layers), to Jackrong/Qwopus3.5-9B-Coder for the
`mtp.fc.weight` tensor that completed the restore, and to froggeric for the
Qwen-Fixed-Chat-Templates v22.1 baked in here.

## Usage (recommended)

For best performance, use the baked-in froggeric template with reasoning
enabled and preserved:

```bash
llama-server -m model.gguf \
    --jinja --chat-template-file chat_template.jinja \
    --reasoning-format deepseek --reasoning on --reasoning-preserve \
    --spec-type draft-mtp --spec-draft-n-max 3
```

- **llama.cpp MTP:** `--spec-type draft-mtp --spec-draft-n-max 3`
- **vLLM MTP:** `--speculative-config '{"method":"mtp","num_speculative_tokens":3}'`

## Provenance & credits

| Component | Source |
|---|---|
| Base architecture | [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) (Qwen team) |
| Backbone merge | [nightmedia/Qwen3.5-9B-Holodeck-Lounge](https://huggingface.co/nightmedia/Qwen3.5-9B-Holodeck-Lounge) — 13-model creative-writing merge (DavidAU heretic family, armand0e, microsoft/Fara1.5-9B, Jackrong, et al.; see upstream card) |
| MTP head — 14 `mtp.layers.*` blocks + `mtp.norm`/`mtp.pre_fc_norm_*` | Present in [nightmedia/Qwen3.5-9B-Holodeck-Lounge](https://huggingface.co/nightmedia/Qwen3.5-9B-Holodeck-Lounge) merge |
| MTP head — `mtp.fc.weight` (input-fusion projection, the missing 15th layer) | [Jackrong/Qwopus3.5-9B-Coder](https://huggingface.co/Jackrong/Qwopus3.5-9B-Coder) |
| Chat template (v22.1) | [froggeric/Qwen-Fixed-Chat-Templates](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates) |

License: **apache-2.0**. License chain (all components apache-2.0, verified):
Qwen/Qwen3.5-9B → nightmedia merge → Jackrong MTP head → froggeric template.

## Limitations

- Acceptance trails dedicated trained draft heads such as
  [EAGLE-3](https://arxiv.org/abs/2503.01840) (≈61% on Qwen3.5-9B).
- Deepest draft positions (pos2) accept at lower rates than pos0/pos1.
- The transplanted head is inference-accelerator only; it does not affect
  backbone text quality.
- The head is only directly usable as a built-in MTP head for this
  architecture (same hidden size + vocab); as a standalone GGUF it can also be
  used as a sidecar draft (`--model-draft mtp-*.gguf`).

**AI was used to draft this report.**