Qwen3.6-27B-4bit-test

This repository holds Qwen/Qwen3.6-27B converted to MLX and quantized to 4 bits. It is a TEST artifact that validates an internal conversion pipeline, so prefer mlx-community/Qwen3.6-27B-4bit for general use.

Pair this model with Qwen3.6-27B-MTP-4bit-test to get speculative decoding, because Qwen3.6 carries a Multi-Token Prediction head that the MLX conversion moves into a separate repository.

Use with mlx-vlm

pip install -U mlx-vlm

# plain decoding
mlx_vlm generate \
  --model vvsotnikov/Qwen3.6-27B-4bit-test \
  --prompt "Explain speculative decoding in two sentences." \
  --max-tokens 256

# with the MTP drafter
mlx_vlm generate \
  --model vvsotnikov/Qwen3.6-27B-4bit-test \
  --draft-model vvsotnikov/Qwen3.6-27B-MTP-4bit-test \
  --prompt "Write a quicksort in Python." \
  --max-tokens 256 --temperature 0.6

How this was produced

mlx_vlm convert --hf-path Qwen/Qwen3.6-27B \
  --mlx-path Qwen3.6-27B-4bit -q --q-bits 4 --q-group-size 64

The quantization is uniform affine at 4 bits with group size 64, and it covers 498 modules of the language model including embed_tokens, lm_head and the linear_attn projections. The vision tower stays dense in bfloat16, because the converter skips multimodal modules by default.

Verification

The build reproduces mlx-community/Qwen3.6-27B-4bit on all 1190 dense tensors, while 990 quantization payloads differ, since mx.quantize changed its rounding between the mlx version that built the published repository and the version used here. The two builds are equal in quality rather than merely similar: six modules sampled across the full depth show IDENTICAL maximum error against the original bfloat16 weights, and their RMSE agrees to six significant figures.

Check	Result
Quantization	{bits: 4, group_size: 64, mode: affine}
Quantized modules	498 of 2180 tensors
Vision tower	dense, 333 tensors
MTP tensors	0, moved to the drafter repository
Acceptance with the 4-bit drafter	93.5 percent, 2.86 accepted tokens per round

Measured on an M1 Max: 15.5 tokens per second without the drafter, and 16.0 with it. The drafter raises acceptance a long way, yet it did not raise throughput on that machine, so measure your own hardware before you depend on the speedup.

License and attribution

The weights derive from Qwen/Qwen3.6-27B under Apache 2.0, so the original license and its terms carry over. Refer to the upstream model card for the model's capabilities, its training and its usage limits.