---
license: apache-2.0
base_model: mistralai/Voxtral-Mini-3B-2507
tags:
  - voxtral
  - quantized
  - mlx
  - voxtral-mini-3b-2507
library_name: mlx
---

# Voxtral Mini 3B — 2507 — Quantized (MLX)

Public quantized weights based on MLX bf16 from `mlx-community/Voxtral-Mini-3B-2507-bf16`.
Upstream model: [`mistralai/Voxtral-Mini-3B-2507`](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507).

## Variants (quantization profiles)
- Q4: folder `mlx-q4/`
- Q5: folder `mlx-q5/`
- Q6: folder `mlx-q6/`
- Q8: folder `mlx-q8/`

Published variants appear as subfolders at the top of this repo when available.

## Quantization notes
- Only inference weights are quantized (Q4/Q5/Q6/Q8 as above).
- Embeddings are NOT quantized to preserve shape compatibility. Therefore, any "bits per weight" metric may exceed the nominal target (informational, not an error).

## Quickstart (MLX)
```python
from mlx_lm import load, generate
model, tokenizer = load("NeoRoth/voxtral-3b-quantized")
print(generate(model, tokenizer, "Hello!", max_tokens=64))
```

## Integrity (SHA256)
- Q4 `model-00001-of-00001.safetensors`:
  - `eec98aef078b3db2c226943d38558d814b10ec387dc5359d333eeed4be5298d2`
- Q8 `model-00001-of-00001.safetensors`:
  - `37999e4a9dda52a0aedb593636be6c12e69dd8b8457f15ce48134f88b1ccebd3`

## License
- Apache-2.0 (see `LICENSE.txt`).

## Credits
- Upstream model: [`mistralai/Voxtral-Mini-3B-2507`](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507)
- MLX bf16 base used for quantization: [`mlx-community/Voxtral-Mini-3B-2507-bf16`](https://huggingface.co/mlx-community/Voxtral-Mini-3B-2507-bf16)