Add MTP acceleration recipe with measured speedup

README.md

@@ -122,7 +122,57 @@ This confirms the architectural insight from prior research: safety alignment do
 - **Text-only.** No vision input. No audio input. The encoders are gone. Passing image or audio tokens will produce undefined behavior.
 - **Same context window as base** (128k tokens).
 - **Same tokenizer.** The vocab includes vision/audio special tokens (`<image>`, `<audio>`, etc.) for compatibility with the official tokenizer; these tokens won't activate any modality processing in this variant.
-- **No MTP drafter support on Ollama yet.** The official `google/gemma-4-E4B-it-assistant` MTP drafter works with Transformers and vLLM but not with Ollama on Linux/CUDA as of May 2026 (upstream llama.cpp doesn't recognize the `Gemma4AssistantForCausalLM` architecture). For MTP-accelerated inference, use Transformers or vLLM directly with this model as the target.
+- **No MTP drafter support on Ollama yet.** Upstream llama.cpp doesn't recognize the `Gemma4AssistantForCausalLM` architecture as of May 2026, so Ollama on Linux/CUDA can't pair this target with the official MTP drafter. For MTP-accelerated inference, use Transformers or vLLM directly — see the [MTP acceleration](#mtp-acceleration) section below.
+
+## MTP acceleration
+
+The official MTP drafter [`google/gemma-4-E4B-it-assistant`](https://huggingface.co/google/gemma-4-E4B-it-assistant) (78M params, activation-aware) pairs cleanly with this stripped target. Output is lossless (byte-identical at deterministic decode). Measured on RTX 3090 via HF Transformers:
+
+| Prompt shape | Tokens generated | Baseline | + MTP drafter | Speedup |
+|---|---:|---:|---:|---:|
+| MCQ single letter | 5 | 394 ms | 363 ms | 1.09x |
+| Open Q one-word | 5 | 395 ms | 249 ms | 1.59x |
+| Slug classification | 5 | 462 ms | 224 ms | 2.07x |
+| JSON entity list (128 tok) | 128 | 12291 ms | 6712 ms | 1.83x |
+| JSON memories (114 tok) | 114 | 8425 ms | **2771 ms** | **3.04x** |
+
+Speedup tracks output predictability — structured JSON outputs (the most common MemPalace surface) land at the high end (3x).
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+target = AutoModelForCausalLM.from_pretrained(
+    "igorls/gemma4-e4b-classifier",
+    dtype=torch.bfloat16,
+    device_map="cuda",
+)
+drafter = AutoModelForCausalLM.from_pretrained(
+    "google/gemma-4-E4B-it-assistant",
+    dtype=torch.bfloat16,
+    device_map="cuda",
+)
+tok = AutoTokenizer.from_pretrained("igorls/gemma4-e4b-classifier")
+
+messages = [{"role": "user", "content": "Classify into one word (indoor, outdoor): The kids are playing in the backyard."}]
+chat = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+ids = tok(chat, return_tensors="pt").input_ids.to("cuda")
+
+out = target.generate(
+    ids,
+    assistant_model=drafter,
+    max_new_tokens=20,
+    do_sample=False,
+)
+print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
+```
+
+For production throughput, use vLLM — it implements the drafter's centroid-masking optimization (sparse lm_head over ~4K candidates instead of ~262K vocab, ~45x reduction):
+
+```bash
+vllm serve igorls/gemma4-e4b-classifier \
+  --speculative-config '{"model": "google/gemma-4-E4B-it-assistant", "num_speculative_tokens": 4}'
+```
 
 ## License