Ilya626/MoonGem-MTP

llama.cpp startup

Use a llama.cpp build supporting Gemma 4 Assistant draft-mtp. Download the Assistant GGUF from this repository and your MoonGem-31B target GGUF. Replace the three paths below with your own paths, then run:

LLAMA_SERVER="/path/to/llama-server"
TARGET_GGUF="/path/to/MoonGem-31B.gguf"
ASSISTANT_GGUF="/path/to/MoonGem-MTP-Assistant-Q4_K_M.gguf"

"$LLAMA_SERVER" \
  --model "$TARGET_GGUF" \
  --model-draft "$ASSISTANT_GGUF" \
  --spec-type draft-mtp \
  --spec-draft-n-max 4 \
  --spec-draft-p-min 0 \
  --n-gpu-layers 99 \
  --n-gpu-layers-draft 99 \
  --ctx-size 8192 \
  --jinja \
  --temp 0.9 --top-p 0.95 --top-k 64 --min-p 0.02 \
  --host 127.0.0.1 --port 8080

The Assistant GGUF already includes the LoRA; no additional --lora flag is needed.