Irfanuruchi/SmolLM-360M-Instruct-MLX-5bit

SmolLM-360M-Instruct (MLX 5-bit)

A 5-bit MLX quantized build of HuggingFaceTB/SmolLM-360M-Instruct targeting a better quality/footprint balance than 3-bit.

Benchmark Environment

• Device: MacBook Pro (M3 Pro)
• Runtime: MLX
• Quantization: 5-bit

Performance (Measured)

• Disk size: ~241 MB
• Peak memory: ~0.29 GB
• Generation speed: ~296 tokens/sec

These numbers were measured on macOS (M3 Pro).
iPhone / iPad performance will vary depending on hardware and memory.

Usage

mlx_lm.generate \
  --model Irfanuruchi/SmolLM-360M-Instruct-MLX-5bit \
  --prompt "Reply with exactly 3 bullet points, 4-8 words each: what can you do offline?" \
  --max-tokens 80

License

Upstream SmolLM is released under Apache-2.0. Preserve attribution and the original license terms.