Qwen3.8-27B-Vision-CoreML-fp16

The vision encoder of Qwen/Qwen3.8-27B, compiled for Core ML in fp16. It lets a Qwen3.8-27B GGUF answer questions about images in tempo9: a GGUF carries the language model, and tempo9 runs the vision encoder through Core ML, on the GPU or the Neural Engine, instead of from an mmproj file.

Use

With tempo9 1.2.2 or later, --tower auto reads the GGUF's header and fetches this encoder:

hf download lmstudio-community/Qwen3.8-27B-GGUF Qwen3.8-27B-Q4_K_M.gguf \
  --local-dir ~/models/Qwen3.8-27B-GGUF
tempo9 --gguf ~/models/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf --tower auto

To keep the encoder in a directory of your own, download it and pass the directory:

hf download thinkspread/Qwen3.8-27B-Vision-CoreML-fp16 \
  --local-dir ~/models/Qwen3.8-27B-Vision-CoreML-fp16
hf download lmstudio-community/Qwen3.8-27B-GGUF Qwen3.8-27B-Q4_K_M.gguf \
  --local-dir ~/models/Qwen3.8-27B-GGUF
tempo9 --gguf ~/models/Qwen3.8-27B-GGUF/Qwen3.8-27B-Q4_K_M.gguf \
  --tower ~/models/Qwen3.8-27B-Vision-CoreML-fp16

The Q4_K_M file (16.8 GB) and this encoder fit a 24 GB Mac together at the default settings. On tempo9 1.2.1, start it with AS_METAL_DENSE_RESIDENCY=0: 1.2.1 runs out of GPU memory loading this model on a 24 GB Mac, and 1.2.2 fixes that.

Then send the image as an image_url data URI to /v1/chat/completions. Any OpenAI client works; with nothing but the standard library:

python3 - photo.png <<'EOF'
import base64, json, sys, urllib.request
img = base64.b64encode(open(sys.argv[1], "rb").read()).decode()
body = {"model": "local", "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "data:image/png;base64," + img}},
    {"type": "text", "text": "What is in this picture?"}]}]}
req = urllib.request.Request("http://127.0.0.1:11435/v1/chat/completions",
    json.dumps(body).encode(), {"content-type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"]["content"])
EOF

• Pair it with Qwen3.8-27B only. It emits 5120-wide embeddings, the width of the 27B language model. Any quantization of the GGUF that tempo9 loads works with it. tempo9 does not load i-quants, and unsloth's UD-Q4_K_S of this model mixes iq2_xs into its FFN weights, so use a K-quant such as lmstudio-community's Q4_K_M.
• One image per request, as a data: URI.
• An image costs about one prompt token per 32×32 pixels, plus two. Images larger than about one megapixel (4096 patches of 16×16) are scaled down, aspect ratio kept.
• Apple Silicon, macOS 26 or later.

Files

file	what it is
qwen35_vit_b1024.mlmodelc	encoder for images up to 1024 patches (≈ 512×512 px)
qwen35_vit_b4096.mlmodelc	encoder for images up to 4096 patches (≈ 1024×1024 px)
tower_meta.json	shapes, buckets, and the accuracy measured at export
tower_pos_embed.f32, tower_aux.npz	position-embedding table

The two encoders share their weights and differ only in input size; a small image runs on the smaller one.

Accuracy

Measured at export: Core ML fp16 output against the Hugging Face fp32 vision encoder on the exporter's reference images, worst case, norm-weighted cosine similarity.

compute tier	cosine
GPU	0.99999
Neural Engine	0.99958

The tempo9 server runs the encoder on Core ML's GPU tier. The Neural Engine row is for apps that move it there to keep the GPU free for the language model.

Checked

Apple M5 Pro, 24 GB, tempo9 1.2.2, lmstudio-community/Qwen3.8-27B-GGUF Q4_K_M, default settings, greedy, thinking off, two synthetic images:

• A sign reading "OPEN 9-5", "Closed Mondays" and "Tel 0571-8823-4410" beside a yellow triangle: all three lines transcribed exactly, and the shape and color named correctly.
• A red circle, a blue square and the text "PIER 37": shapes, colors and the text all right.

tempo9 1.2.1 with AS_METAL_DENSE_RESIDENCY=0 gave the same two answers. Decode ran at about 14 tokens per second.

License

Apache 2.0, inherited from the base model.