Qwen3-VL-30B-A3B-Instruct-Vision-CoreML-fp16

The vision encoder of Qwen/Qwen3-VL-30B-A3B-Instruct, DeepStack outputs included, compiled for Core ML in fp16. It lets a Qwen3-VL-30B-A3B-Instruct GGUF answer questions about images in tempo9: a GGUF carries the language model, and tempo9 runs the vision encoder through Core ML instead of from an mmproj file.

Use

With tempo9 1.2.2 or later, --tower auto reads the GGUF's header and fetches this encoder:

tempo9 --hf Qwen/Qwen3-VL-30B-A3B-Instruct-GGUF:Q4_K_M --tower auto

To keep the encoder in a directory of your own, download it and pass the directory:

hf download thinkspread/Qwen3-VL-30B-A3B-Instruct-Vision-CoreML-fp16 \
  --local-dir ~/models/Qwen3-VL-30B-A3B-Instruct-Vision-CoreML-fp16
tempo9 --hf Qwen/Qwen3-VL-30B-A3B-Instruct-GGUF:Q4_K_M \
  --tower ~/models/Qwen3-VL-30B-A3B-Instruct-Vision-CoreML-fp16

The Q4_K_M file (17.3 GB) and this encoder fit a 24 GB Mac together at the default settings.

Then send the image as an image_url data URI to /v1/chat/completions. Any OpenAI client works; with nothing but the standard library:

python3 - photo.png <<'EOF'
import base64, json, sys, urllib.request
img = base64.b64encode(open(sys.argv[1], "rb").read()).decode()
body = {"model": "local", "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "data:image/png;base64," + img}},
    {"type": "text", "text": "What is in this picture?"}]}]}
req = urllib.request.Request("http://127.0.0.1:11435/v1/chat/completions",
    json.dumps(body).encode(), {"content-type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"]["content"])
EOF

• Pair it with Qwen3-VL-30B-A3B-Instruct only. It emits 2048-wide embeddings, the same width as Qwen3.5-35B-A3B, so a mix-up between the two is not caught by width. How the GGUF is quantized does not matter. For Qwen3-VL-4B-Instruct use thinkspread/Qwen3-VL-4B-Instruct-Vision-CoreML-fp16.
• One image per request, as a data: URI.
• An image costs about one prompt token per 32×32 pixels, plus two. Images larger than about one megapixel (4096 patches of 16×16) are scaled down, aspect ratio kept.
• Apple Silicon, macOS 26 or later.

Files

file	what it is
qwen3vl_vit_b4096.mlmodelc	encoder for images up to 4096 patches (≈ 1024×1024 px)
tower_meta.json	shapes, layout and DeepStack layer indexes
tower_pos_embed.f32, tower_aux.npz	position-embedding table

Its one output carries the language-model embedding and the three DeepStack feature maps side by side; tempo9 splits them and injects the DeepStack maps into the early decoder layers.

Accuracy

Measured at export: Core ML fp16 on the Neural Engine against the same encoder in fp32, on a synthetic photo-like input, norm-weighted cosine similarity. The two halves of the output are judged separately so that a broken DeepStack head cannot hide behind a healthy main embedding.

output	cosine
language-model embedding	0.99875
DeepStack features	0.99953

The tempo9 server runs the encoder on Core ML's GPU tier.

Checked

Apple M5 Pro, 24 GB, tempo9 1.2.1, Qwen/Qwen3-VL-30B-A3B-Instruct-GGUF Q4_K_M, greedy, two synthetic images:

• A sign reading "OPEN 9-5", "Closed Mondays" and "Tel 0571-8823-4410" beside a yellow triangle: all three lines transcribed exactly, and the shape and color named correctly.
• A red circle, a blue square and the text "PIER 37": shapes and colors right; the text was read as "FREED".

How it was made

Converted from a BF16 llama.cpp mmproj of Qwen3-VL-30B-A3B-Instruct with coremltools 9, as one 4096-patch bucket, by the same exporter as the 4B encoder with no code changes: every hyperparameter is read from the mmproj header. The exporter checks its wrapper against the Hugging Face Qwen3VLVisionModel loaded with the same weights (cosine 1.000000 in fp32), then the Core ML fp16 output against that fp32 wrapper (the table above).

License

Apache 2.0, inherited from the base model.