Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Ternii 
posted an update 1 day ago
Post
86
Qwen3.6-35B-A3B now runs on a 16 GB MacBook Air at about 35 tokens/s.

One line in Terminal downloads everything and starts a local chat in your browser:

curl -fsSL https://raw.githubusercontent.com/ternii-ai/tritq-vllm/main/mac/qwen36-35b.sh | bash

How it fits in 16 GB:
- The model file, Ternii/Qwen3.6-35B-A3B-TR2.25-G-GGUF, is 16.49 GB: 2.25 trits per weight on average, no retraining.
- On a 16 GB Mac the experts stay on the SSD and an 8,400 MiB cache in memory holds the busy ones. With 24 GB or more the script loads the whole model into memory.

Measured on a MacBook Air M5 with 16 GB:
- Decode is about 35 tokens/s (35.1–36.3 over 6 runs on two days, including an independent re-test). Our previous build ran at 23.3–25.2 on the same Mac.
- Up to 12.1 GB wired, so close other large apps first.
- Accuracy in the 16 GB mode: GSM8K 94.0, HumanEval 92.1, MBPP 87.6 vs 95.0 / 92.7 / 87.2 with the model fully in memory, no significant difference (measured on an M5 Max at a 16 GB budget).

Caveats: experimental, and only one 16 GB Mac has been measured. The speed-up costs about 35 % more energy per token. It needs our patched llama.cpp; LM Studio and Ollama can't load it.

Ternii/Qwen3.6-35B-A3B-TR2.25-G-GGUF
In this post