Instructions to use PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-27B — MTPLX Optimized Speed

Qwen3.8 was created by the Qwen team. This is a derivative conversion, not a PocketAI-created base model. PocketAiHub reproduced, validated, and packaged the published MTPLX Optimized Speed recipe.

Built with MTPLX 2.7.1 from Qwen/Qwen3.8-27B at revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. The mixed-precision layout follows Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed: bulk 4-bit/group 32, embeddings + LM head + GDN output projections + final eight MLP blocks at 8-bit/group 64, sensitive state/norm tensors and the native MTP head in BF16.

Use with MTPLX

pip install "mtplx==2.7.1"
mtplx serve --model PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed --profile turbo

The OpenAI-compatible API is then available at http://127.0.0.1:8000/v1. The shipped runtime contract selects native MTP depth 3. Official Qwen3.8 target sampling is temperature 1.0, top-p 0.95, top-k 20.

Measured performance

Apple M5 Max (40-core GPU, 128 GB unified memory), macOS 26.4, single stream, Apple automatic fan control. These are measurements on this exact artifact, not universal hardware guarantees.

Mode	Decode tok/s	End-to-end tok/s	Speedup vs AR	Acceptance by depth
AR	24.62	24.11	1.00×	—
D1	41.69	40.51	1.69×	96.8%
D2	52.85	51.18	2.15×	95.4%, 87.3%
D3	58.02	56.01	2.36×	93.3%, 87.4%, 79.9%

The controlled tune used official Qwen3.8 sampling, thinking disabled, up to 1,024 generated tokens, and selected D3. Download payload: 21.31 GB.

4K context

The prompt contained 4,099 formatted tokens with COBALT-7319 at 59.9% depth. AR and D3 both returned exactly COBALT-7319; greedy outputs were identical.

Mode	Prefill tok/s	Decode tok/s	Prompt eval	Peak process RSS	Peak footprint
AR	907.8	26.4	4.52s	20.9 GB	24.9 GB
D3	810.5	70.5	5.06s	20.9 GB	29.0 GB

The 4K response contained only nine generated tokens, so decode tok/s is more informative than its end-to-end generation rate.

Feature validation

12/12 capability checks
8/8 structured tool-selection checks
reasoning on and reasoning off
OpenAI image_url input: solid-red image identified as red
verified-native MTP contract with all 15 BF16 MTP tensors
262,144-token configured context window; exact 4K retrieval tested

The 8/8 tool-selection suite follows the repository's established scorer. In the no-tool control, MTPLX 2.7.1 correctly emitted no tool call but returned an empty body while the tool schema was attached; ordinary text exactness passed separately. PNG/JPEG/WebP image input is live-tested. Video processor metadata is included for checkpoint completeness, but live video-server input was not claimed or tested.

KL divergence

PocketAiHub measured mean forward KL 0.05850 nats for BF16 → regular MTPLX on the disclosed token-level suite below (median 0.00203, p95 0.25609, top-1 agreement 94.27%). This repository's six weight payloads are also byte-for-byte identical to the official MTPLX reference, whose card reports 0.0220 on a different coding battery. The two values are not directly comparable because their prompt suites and token positions differ.

Method: 24 prompts (12 capability + 12 pinned JailbreakBench harmful), 16 BF16-greedy teacher-forced assistant positions per prompt, 384 positions total, all 248,320 vocabulary logits, float32 capture and float64 probability math. Direction is D_KL(P_reference || P_candidate) in nats. Full per-suite and per-case aggregates are in evaluation/kl-summary.json.

Provenance and license

Base model: Qwen/Qwen3.8-27B by Qwen
MTPLX recipe/runtime: MTPLX by Youssofal
Conversion, validation, packaging: PocketAiHub
License: Apache-2.0; see LICENSE and NOTICE

Exact build and test summaries are in build-recipe.json and evaluation/results.json.

Downloads last month: -

Safetensors

Model size

6B params

Tensor type

BF16

U32

MLX

Hardware compatibility

4-bit

Model tree for PocketAiHub/Qwen3.8-27B-MTPLX-Optimized-Speed

Base model

Qwen/Qwen3.8-27B

Quantized

(480)

this model