Qwen3.8-27B-OptiQ-4bit
Weights are uploading. The sensitivity sweep is running; this card will be updated with the per-layer bit allocation, the size on disk and the full six-metric Capability Score when the quant lands. Nothing is published here yet that has been measured, so nothing is claimed here yet.
An OptiQ mixed-precision quant of mlx-community/Qwen3.8-27B-bf16, for running the model on Apple Silicon with MLX.
What OptiQ does differently
Uniform quantization gives every layer the same bit-width. OptiQ measures how much each layer actually costs when it is quantized — a KL divergence against the full-precision reference, per layer and per candidate bit-width — and then spends a fixed bit budget where it buys the most accuracy. Layers that tolerate 4 bits get 4; the ones that do not keep more.
The result is a standard MLX checkpoint. It loads with mlx_lm.load(...), no
special runtime.
Running it
pip install mlx-optiq
import optiq # registers the vendored architectures
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Qwen3.8-27B-OptiQ-4bit")
print(generate(model, tokenizer, prompt="Hello", verbose=True))
Qwen3.8 is a vision-language model. Image input goes through OptiQ's server,
which loads the bf16 vision tower from the optiq/ sidecar:
optiq serve --model mlx-community/Qwen3.8-27B-OptiQ-4bit
That exposes an OpenAI-compatible endpoint on 127.0.0.1:8080 accepting
image_url content parts.