⚠️ DEPRECATED — Recommended: leonsarmiento/Nex-N2-mini-5bit-XL-mlx

This model has been superseded by the BaseQuant_XL variant, which keeps routing-critical layers (MoE router gate, shared expert, lm_head) in full bf16 precision for improved quality. Benchmark comparisons are available in the XL model card.

→ Download leonsarmiento/Nex-N2-mini-5bit-XL-mlx

leonsarmiento/Nex-N2-mini-5bit-mlx

This model was converted to MLX format from nex-agi/Nex-N2-mini using mixed 5/8-bit quantization optimized for Apple Silicon. The vision encoder is preserved and quantized at 5-bit, making this a full multimodal model.

Nex-N2-mini is a 35B-parameter MoE (Mixture of Experts) model fine-tuned from Qwen3.5-35B-A3B-Base by Nex-AGI, featuring 256 experts (8 active per token + 1 shared expert), hybrid full + linear (Gated DeltaNet) attention, an "Agentic Thinking" framework (Adaptive Thinking + Coherent Thinking), and a vision encoder for multimodal input. Despite 35B total parameters, only ~3B are activated per token for efficient inference.

Benchmark Highlights (5-bit, text-only tests)

Benchmark	Nex-N2-mini 5-bit	Qwen3.6-27B (SOTA)	Qwen3.6-35B-A3B
HumanEval (n=30)	90.0%	93.3%	66.7%
MBPP (n=30)	76.7%	—	—

Nex-N2-mini narrows the gap to Qwen3.6-27B on coding benchmarks while maintaining a 3B-active MoE speed advantage.

Use with mlx

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/Nex-N2-mini-5bit-mlx --max-tokens 256 --temperature 0.7 --top-p 0.95 --prompt "Hello"

Mixed Quantization Strategy

Bit Depth	Layers	Rationale
8-bit	embed_tokens, lm_head, router gate, shared_expert_gate, shared_expert, self_attn (full attention), linear_attn (DeltaNet)	Every token passes through these — routing accuracy, shared representation, and sequence modeling are non-negotiable
5-bit	vision_tower, switch_mlp (routed experts)	Bulk of parameters, only 8 of 256 experts active per token — natural redundancy tolerates lower precision

Quantization Details

Layer	Bits	Group Size
embed_tokens	8	64
lm_head	8	64
mlp.gate (router)	8	64
shared_expert_gate	8	64
shared_expert	8	64
self_attn (full attention)	8	64
linear_attn (DeltaNet)	8	64
vision_tower	5	64
switch_mlp (routed experts)	5	64
Default fallback	8	64

• Bits per weight: 5.750
• Total size: ~24 GB (5 shards)
• Quantization type: Mixed 5/8-bit (multimodal, vision preserved)
• Group size: 64
• Method: Custom quant_predicate via mlx_vlm

Recommended Inference Parameters

Parameter	Value
temperature	0.7
top_p	0.95
top_k	40
min_p	0.01
repeat_penalty	1.05

Note: This is a Qwen3.5-based model — preserve_thinking is not applicable.