spicyneuron/Qwen3.5-35B-A3B-MLX-4.8-bit

Qwen3.5-35B-A3B optimized for MLX. This quant does not support image input.

For vision support: https://huggingface.co/spicyneuron/Qwen3.5-35B-A3B-MLX-vision-4.9-bit

Usage

# Start server at http://localhost:8080/v1/chat/completions
uvx --from mlx-lm mlx_lm.server --host 127.0.0.1 --port 8080 \
  --model spicyneuron/Qwen3.5-35B-A3B-MLX-4.8-bit

Methodology

Quantized using a custom script inspired by Unsloth/AesSedai/ubergarm style mixed-precision GGUFs. MLX quantization options differ than llama.cpp, but the principles are the same:

• Sensitive layers like MoE routing, attention, and output embeddings get higher precision
• More tolerant layers like MoE experts get lower precision