chexone-4bit-mlx

MLX quantization of StanfordAIMI/CheXOne for Apple Silicon.

Variant: Affine int4
Disk size: 2945 MB
Quantized by: sahilchachra

Note on effective bpw: mlx-vlm's quantizers only act on the language tower's linear weights. The vision encoder and embeddings stay at the source dtype (bf16), so the headline variant name reflects the LM-tower quantization while the on-disk size averages the two halves of the model.

Benchmark results

Evaluated on Apple M4 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

	This model	FP16 baseline
Decode tok/s (avg, long traces)	116.94	38.67
Peak memory (GB)	3.84	7.876
Disk size (MB)	2945	7175

Quality

Benchmark	This model	FP16 baseline	n
VQA-RAD (radiology VQA, accuracy)	20.0%	36.7%	30

Context scaling (decode tok/s)

Context length	Decode tok/s
~128 tokens	118.4
~256 tokens	116.2
~512 tokens	117.4
~1024 tokens	115.8

Usage

pip install mlx-vlm

from mlx_vlm import load, generate

model, processor = load("sahilchachra/chexone-4bit-mlx")
response = generate(model, processor, prompt="Describe this image.",
                    image="path/to/image.jpg", max_tokens=256, verbose=True)

All variants in this collection

Model	Variant
sahilchachra/chexone-4bit-mlx	Affine int4 ← this model
sahilchachra/chexone-8bit-mlx	Affine int8
sahilchachra/chexone-mxfp4-mlx	Block float MX FP4

Notes

• Requires Apple Silicon (M1 or later) with MLX
• Benchmarks run on Apple M4 Pro, 24 GB unified memory
• License: see StanfordAIMI/CheXOne for the original model's license

Original model

See StanfordAIMI/CheXOne for full model details and intended use.