Instructions to use Youssofal/Qwen3.5-4B-Optimized-MTPLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Youssofal/Qwen3.5-4B-Optimized-MTPLX with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Youssofal/Qwen3.5-4B-Optimized-MTPLX")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps
LM Studio

Pi new

How to use Youssofal/Qwen3.5-4B-Optimized-MTPLX with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Youssofal/Qwen3.5-4B-Optimized-MTPLX"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Youssofal/Qwen3.5-4B-Optimized-MTPLX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Youssofal/Qwen3.5-4B-Optimized-MTPLX with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Youssofal/Qwen3.5-4B-Optimized-MTPLX"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Youssofal/Qwen3.5-4B-Optimized-MTPLX

Run Hermes

hermes

MLX LM

How to use Youssofal/Qwen3.5-4B-Optimized-MTPLX with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Youssofal/Qwen3.5-4B-Optimized-MTPLX"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Youssofal/Qwen3.5-4B-Optimized-MTPLX"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Youssofal/Qwen3.5-4B-Optimized-MTPLX",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Qwen3.5-4B Optimized MTPLX (Q8 trunk)

Run this with MTPLX

MTPLX is an MLX-native runtime for native Multi-Token-Prediction speculative decoding on Apple Silicon. Up to 2.24× faster decode at real coding temperatures (temp=0.6 / top_p=0.95 / top_k=20) using the model's own built-in MTP heads — no external drafter, no greedy hack.

pip install mtplx
mtplx start

Project: github.com/youssofal/MTPLX

Other MTPLX checkpoints:

Qwen3.6-27B-MTPLX-Optimized-Speed — 4-bit flagship speed (63 TPS on M5 Max)
Qwen3.6-27B-MTPLX-Optimized — verified default (GDN8-Speed4 trunk + CyanKiwi INT4 MTP)
Qwen3.5-4B-MTPLX-Optimized-Speed — small 4-bit speed-test

Q8-trunk MTPLX speed-test artifact for Apple Silicon.

This model uses the public mlx-community/Qwen3.5-4B-MLX-8bit MLX affine 8-bit trunk and grafts back the official native MTP head from Qwen/Qwen3.5-4B. The MTP head is stored as mtp.safetensors; layer-0 attention/MLP linears are quantized to 4-bit affine group-64, while mtp.fc and the MTP norms stay BF16.

Intended Use

A quick MTPLX download / load / speed-path test artifact at 4B scale, with the larger 8-bit trunk for higher-fidelity target verification. Once the runtime ships:

mtplx start

Choose Custom Hugging Face repo, then enter:

Youssofal/Qwen3.5-4B-Optimized-MTPLX

Artifact Layout

Trunk: MLX affine 8-bit, group size 64
MTP sidecar: official Qwen3.5-4B MTP tensors
MTP sidecar quantization: body-int4
Runtime contract: mtplx_runtime.json
MTPLX default: depth 2, target temperature 0.6, draft temperature 0.7

Local Smoke Result

On the local Apple Silicon MTPLX workstation, the depth-2 speed path measured 105.21 tok/s versus 75.63 tok/s AR on the warm-code prompt (max_tokens=48, temperature=0.6, top_p=0.95, top_k=20).

This Q8 artifact had the best multiplier in the local one-prompt matrix. The Q4 sibling (Youssofal/Qwen3.5-4B-MTPLX-Optimized-Speed) remains faster in absolute tok/s because its 4-bit trunk has a faster AR baseline.

Build Stats

{
  "bits": 4,
  "group_size": 64,
  "mode": "affine",
  "output_size_bytes": 86701040,
  "output_tensor_count": 29,
  "policy": "cyankiwi",
  "quantization": "body-int4",
  "quantized_linears": {
    "mtp.layers.0.mlp.down_proj":   {"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.mlp.gate_proj":   {"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.mlp.up_proj":     {"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.k_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.o_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.q_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
    "mtp.layers.0.self_attn.v_proj":{"bits": 4, "group_size": 64, "mode": "affine"}
  },
  "source_tensor_count": 15
}

Model tree for Youssofal/Qwen3.5-4B-Optimized-MTPLX

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Quantized

(188)

this model

Youssofal
/

Qwen3.5-4B-Optimized-MTPLX