Hy4-flash MLX 2-bit

Our smallest Hy4-flash pack: 85.5 GB, 3.12 effective bits per weight.

A Nace quant of Tencent Hy4-flash, with 2-bit routed experts. The preserved model pack contains 85,458,831,601 bytes across 56 files.

The Engram tables retain IQ4_NL precision. Other weights retain their selected precision.

Results

Hy4-flash quantization quality across 37 measured variants

37 measured Hy4-flash variants on the same 11 texts, 5,621 scored tokens. Chart sizes use text-weight payload, including required Engram tables, excluding vision/MTP. Complete weight-file sizes and exact scores are in the comparison data.

Run it

Use Python 3.11 or newer and the bundled MLX loader.

hf download nace-ai/Hy4-flash-MLX-3.1bit --local-dir Hy4-flash-MLX-3.1bit
cd Hy4-flash-MLX-3.1bit
python3 -m venv .venv
source .venv/bin/activate

Install dependencies on Apple silicon:

python3 -m pip install -r requirements-macos.txt

On Linux with a CUDA 13 GPU, use requirements-linux.txt instead.

python3 run_mlx.py --model . --prompt "What is the capital of France?" --max-tokens 64

License

Tencent Hy License, Version 1.0.

Downloads last month
-
Safetensors
Model size
19B params
Tensor type
U32
·
F32
·
BF16
·
I32
·
F16
·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for nace-ai/Hy4-flash-MLX-3.1bit

Quantized
(40)
this model

Collection including nace-ai/Hy4-flash-MLX-3.1bit