Domyn-Edge-SFT (MLX 4-bit)

MLX 4-bit quantized version of domyn/Domyn-Edge-SFT.

  • ~2.5 GB
  • Hybrid attention (sliding window + full attention)
  • Runs on 16 GB Apple Silicon

Important

This model uses a custom architecture (domynedge).
You must have the matching domynedge.py file in your local mlx-lm checkout for it to load.

Usage

# After placing domynedge.py in mlx_lm/models/
mlx_lm.generate --model your-username/Domyn-Edge-SFT-4bit --prompt "What is a dolphin?" --max-tokens 200

Quantization

Converted with mlx_lm.convert using 4-bit quantization, group size 64.

Limitations

Chat template handling is incomplete Generation quality is still rough Requires the custom architecture file

Downloads last month
74
Safetensors
Model size
4B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for dragosroua/Domyn-Edge-SFT-4bit

Quantized
(3)
this model