OsaurusAI/Qwen3.8-27B-MXFP8

The reference tier — true MXFP8 (e8m0 scales, group size 32), the bundle every KL number on this card is measured against.

A calibrated JANG bundle of Qwen/Qwen3.8-27B — Qwen's 27B dense hybrid (GatedDeltaNet + gated attention) vision-language model with flexible thinking control — quantized for Apple Silicon / MLX and runnable today with stock mlx_vlm. Text, image, and video understanding all verified working in this exact bundle, with the model's native multi-token-prediction head preserved.

Why this quant

This is not a uniform one-bit-number-for-everything conversion:

• Measured allocation, not name rules — every one of the 590 quantized modules got its bit width from a Hessian-trace sensitivity capture (tr(H)·‖W‖²_F per module, from activation statistics over a calibration corpus), so the bits go where this model actually needs them. Attention stays high-precision; insensitive FFN blocks give the budget back.
• No affine refit applies here — MXFP8 is an e8m0 shared-exponent format, not an affine fit; this bundle is the untouched reference tier.
• fp16 where quantization would lie — the 27 vision-block linear_fc2 projections (in_features 4304, indivisible by any MLX quant group) pass through in fp16 instead of being force-fit. No silently broken vision tower.
• The full serving contract is stamped, not documented-elsewhere — sampling presets, reasoning-effort tiers, thinking defaults, EOS pair, context guidance: all machine-readable in the bundle (see below).
• Honest method record — imatrix + Hessian allocation applied; AWQ and GPTQ deliberately not applied (AWQ's norm-fold is unsafe on this family's zero-centered-norm convention; GPTQ needs off-diagonal Hessians — logged as follow-up), and the bundle metadata says exactly that.

The lineup

Four bundles of the same calibrated build, by size/quality trade: JANG_2D (10.9 GiB) · JANG_4D (17.0 GiB) · JANG_6D (24.1 GiB) · MXFP8 (26.8 GiB, reference tier).

Model + bundle facts

Field	Value
Base model	Qwen/Qwen3.8-27B (dense 27B VLM)
Layout	64 layers — 48 GatedDeltaNet + 16 gated full-attention (partial RoPE dim 64)
Vision	native image + video tower (verified in this bundle)
MTP	native multi-token-prediction head preserved (31 tensors, own shard)
Context	262,144 native, extensible to 1M
Quantization	mxfp8 gs32 (e8m0)
On disk	26.8 GiB
Runs on	Apple Silicon Macs with ≥ 36 GB unified memory

Serving contract (stamped in the bundle)

Everything below is stamped in generation_config.json + jang_config.json — runtimes should read it, not re-derive it:

• Thinking mode ON by default — sampling temperature=1.0, top_p=0.95, top_k=20 (the agentic preset; also the correct preset for coding agents). Instruct / non-thinking preset: temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5.
• reasoning_effort: low / medium / xhigh (default xhigh), carried as a chat-template kwarg.
• preserve_thinking ON by default (Qwen3.8 retains reasoning context across turns — new in this generation, and prefix-cache friendly).
• Reasoning OFF = prefilled closed <think>\n\n</think>\n\n — never plain omission. Reasoning parser qwen3; tool-call parser qwen3_coder.
• Stop on both EOS ids 248046 and 248044.
• Recommended output budget: up to 262,144 reasoning + 131,072 final tokens.

Use it

pip install -U mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.utils import load_config

model, processor = load("OsaurusAI/Qwen3.8-27B-MXFP8")

# text + image
out = generate(model, processor, "Describe this image.", image=["photo.png"],
               max_tokens=512, temperature=1.0, top_p=0.95)

Video note: render video prompts through the bundle's own chat template ({"type": "video"} content item → <|vision_start|><|video_pad|><|vision_end|>); mlx_vlm.prompt_utils.apply_chat_template silently drops video items.

MTP head

The bundle ships the model's trained multi-token-prediction head (Qwen trained it with multiple steps) plus a vmlx_mtp_tuning.json sidecar recommending 1 draft token/step as the starting depth. Speculative decoding is opt-in for runtimes that support it; the head is skipped harmlessly everywhere else.

---

Quantized and calibrated by Jinho Jang — eric@osaurus.ai — for Osaurus, the native local-AI runtime for macOS.