Voxtral Mini 3B MLX

MLX builds of Voxtral Mini 3B, made by Moonbox for Apple Silicon Macs. Each precision is a folder; keep a folder's files together. Only the parts listed below are converted: the rest of the pipeline comes from the original repository unless the section below says otherwise.

Files

The last column is how far each build is from the original PyTorch model on the same inputs (see the section below for what was measured).

Precision	File	Size	sha256	Checked against the original
bf16	`bf16/config.json`	1.5 kB	`228eec39354d`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
bf16	`bf16/generation_config.json`	0.1 kB	`cebeed3d4a16`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
bf16	`bf16/model-00001-of-00002.safetensors`	5.36 GB	`d830758e4223`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
bf16	`bf16/model-00002-of-00002.safetensors`	3.99 GB	`09315657e9bb`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
bf16	`bf16/model.safetensors.index.json`	71.1 kB	`f682c18bfcd6`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
bf16	`bf16/params.json`	0.7 kB	`4a37e19f2524`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
bf16	`bf16/preprocessor_config.json`	0.4 kB	`86d67d926e17`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
bf16	`bf16/tekken.json`	14.9 MB	`4aaf3836c2a5`	WER 1.15 % (original 1.15 %); transcripts identical to the original's
fp16	`fp16/config.json`	1.5 kB	`228eec39354d`	WER 1.15 %; transcripts identical to the original's
fp16	`fp16/generation_config.json`	0.1 kB	`cebeed3d4a16`	WER 1.15 %; transcripts identical to the original's
fp16	`fp16/model-00001-of-00002.safetensors`	5.36 GB	`482968139d78`	WER 1.15 %; transcripts identical to the original's
fp16	`fp16/model-00002-of-00002.safetensors`	3.99 GB	`11277c832b17`	WER 1.15 %; transcripts identical to the original's
fp16	`fp16/model.safetensors.index.json`	71.1 kB	`f682c18bfcd6`	WER 1.15 %; transcripts identical to the original's
fp16	`fp16/params.json`	0.7 kB	`4a37e19f2524`	WER 1.15 %; transcripts identical to the original's
fp16	`fp16/preprocessor_config.json`	0.4 kB	`86d67d926e17`	WER 1.15 %; transcripts identical to the original's
fp16	`fp16/tekken.json`	14.9 MB	`4aaf3836c2a5`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/config.json`	1.7 kB	`3a5aecd2c4ec`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/generation_config.json`	0.1 kB	`cebeed3d4a16`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/model-00001-of-00002.safetensors`	5.37 GB	`cf91a1b58fa3`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/model-00002-of-00002.safetensors`	197.3 MB	`5a4e719ef2c4`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/model.safetensors.index.json`	0.1 MB	`f50814443ebb`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/params.json`	0.7 kB	`4a37e19f2524`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/preprocessor_config.json`	0.4 kB	`86d67d926e17`	WER 1.15 %; transcripts identical to the original's
8bit	`8bit/tekken.json`	14.9 MB	`4aaf3836c2a5`	WER 1.15 %; transcripts identical to the original's
6bit	`6bit/config.json`	1.7 kB	`d4bf1a368f37`	WER 0.96 %; 0.2 % of the original's words read differently
6bit	`6bit/generation_config.json`	0.1 kB	`cebeed3d4a16`	WER 0.96 %; 0.2 % of the original's words read differently
6bit	`6bit/model.safetensors`	4.56 GB	`0283b107fe41`	WER 0.96 %; 0.2 % of the original's words read differently
6bit	`6bit/model.safetensors.index.json`	96.0 kB	`0da4be47bad2`	WER 0.96 %; 0.2 % of the original's words read differently
6bit	`6bit/params.json`	0.7 kB	`4a37e19f2524`	WER 0.96 %; 0.2 % of the original's words read differently
6bit	`6bit/preprocessor_config.json`	0.4 kB	`86d67d926e17`	WER 0.96 %; 0.2 % of the original's words read differently
6bit	`6bit/tekken.json`	14.9 MB	`4aaf3836c2a5`	WER 0.96 %; 0.2 % of the original's words read differently
4bit	`4bit/config.json`	1.7 kB	`a113d6595258`	WER 1.15 %; 0.4 % of the original's words read differently
4bit	`4bit/generation_config.json`	0.1 kB	`cebeed3d4a16`	WER 1.15 %; 0.4 % of the original's words read differently
4bit	`4bit/model.safetensors`	3.55 GB	`4edb404f259e`	WER 1.15 %; 0.4 % of the original's words read differently
4bit	`4bit/model.safetensors.index.json`	96.0 kB	`f848f5c8ba10`	WER 1.15 %; 0.4 % of the original's words read differently
4bit	`4bit/params.json`	0.7 kB	`4a37e19f2524`	WER 1.15 %; 0.4 % of the original's words read differently
4bit	`4bit/preprocessor_config.json`	0.4 kB	`86d67d926e17`	WER 1.15 %; 0.4 % of the original's words read differently
4bit	`4bit/tekken.json`	14.9 MB	`4aaf3836c2a5`	WER 1.15 %; 0.4 % of the original's words read differently

How to run

Voxtral Mini 3B (Mistral's speech model: transcription in 8 languages, plus audio question answering and summarization on the Ministral 3B decoder) for mlx-audio. Each folder is a complete mlx-audio model: model.safetensors (the Whisper-style audio encoder, the projector and the language model), config.json, tekken.json (the tokenizer) and the processor config. Tested with mlx-audio==0.6.5 (Voxtral's processor needs mistral-common[audio]>=1.11.5):

from mlx_audio.stt.utils import load_model

model = load_model("8bit")              # a downloaded folder
print(model.generate("speech.wav", language="en", max_tokens=256).text)

8bit/, 6bit/ and 4bit/ quantize the language model and projector (MLX affine, group size 64); the audio encoder stays bf16.

How they were checked. 24 LibriSpeech test-clean utterances in transcription mode, greedy decoding, with the original in PyTorch (bfloat16, the checkpoint's dtype; transformers' VoxtralForConditionalGeneration) and every MLX build: WER against the reference transcripts and against the original's own output. Pass: WER at most 1 point above the original's. English only: the lab's other test language (Chinese) isn't one of the model's.

Precision	Size	WER	Words read differently from the original	Real-time factor, M5 24 GB
original (PyTorch bf16, MPS)	-	1.15 %	-	0.28
`bf16/`	9.37 GB	1.15 %	0	0.29
`fp16/`	9.37 GB	1.15 %	0	0.29
`8bit/`	5.58 GB	1.15 %	0	0.19
`6bit/`	4.57 GB	0.96 %	0.2 %	0.19
`4bit/`	3.56 GB	1.15 %	0.4 %	0.16

Real-time factor = transcription time / audio length (lower is faster), one utterance at a time. bf16/ needs about 10.5 GB of memory; 8bit/ 6.7 GB.

How these were made

From Voxtral Mini 3B at revision 3060fe34b3: the original weights loaded in MLX and stored as bf16 or fp16, or quantized with MLX's affine quantization (8bit, 6bit, 4bit, group size 64; layers MLX can't quantize, such as convolutions and norms, stay bf16). Every build was checked against the original before upload; the sha256 column above is what the files must hash to.

License

Apache-2.0. These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged.

Original model card

Voxtral Mini 1.0 (3B) - 2507

Voxtral Mini is an enhancement of Ministral 3B, incorporating state-of-the-art audio input capabilities while retaining best-in-class text performance. It excels at speech transcription, translation and audio understanding.

Learn more about Voxtral in our blog post here and our research paper.

Key Features

Voxtral builds upon Ministral-3B with powerful audio understanding capabilities.

Dedicated transcription mode: Voxtral can operate in a pure speech transcription mode to maximize performance. By default, Voxtral automatically predicts the source audio language and transcribes the text accordingly
Long-form context: With a 32k token context length, Voxtral handles audios up to 30 minutes for transcription, or 40 minutes for understanding
Built-in Q&A and summarization: Supports asking questions directly through audio. Analyze audio and generate structured summaries without the need for separate ASR and language models
Natively multilingual: Automatic language detection and state-of-the-art performance in the world’s most widely used languages (English, Spanish, French, Portuguese, Hindi, German, Dutch, Italian)
Function-calling straight from voice: Enables direct triggering of backend functions, workflows, or API calls based on spoken user intents
Highly capable at text: Retains the text understanding capabilities of its language model backbone, Ministral-3B

Benchmark Results

Audio

Average word error rate (WER) over the FLEURS, Mozilla Common Voice and Multilingual LibriSpeech benchmarks:

Text

Usage

The model can be used with the following frameworks;

vllm (recommended): See here
Transformers 🤗: See here

Notes:

temperature=0.2 and top_p=0.95 for chat completion (e.g. Audio Understanding) and temperature=0.0 for transcription
Multiple audios per message and multiple user turns with audio are supported
System prompts are not yet supported

vLLM (recommended)

We recommend using this model with vLLM.

Installation

Make sure to install vllm >= 0.10.0, we recommend using uv:

uv pip install -U "vllm[audio]" --system

Doing so should automatically install mistral_common >= 1.8.1.

To check:

python -c "import mistral_common; print(mistral_common.__version__)"

Offline

You can test that your vLLM setup works as expected by cloning the vLLM repo:

git clone https://github.com/vllm-project/vllm && cd vllm

and then running:

python examples/offline_inference/audio_language.py --num-audios 2 --model-type voxtral

Serve

We recommend that you use Voxtral-Small-24B-2507 in a server/client setting.

Spin up a server:

vllm serve mistralai/Voxtral-Mini-3B-2507 --tokenizer_mode mistral --config_format mistral --load_format mistral

Note: Running Voxtral-Mini-3B-2507 on GPU requires ~9.5 GB of GPU RAM in bf16 or fp16.

To ping the client you can use a simple Python snippet. See the following examples.

Audio Instruct

Leverage the audio capabilities of Voxtral-Mini-3B-2507 to chat.

Make sure that your client has mistral-common with audio installed:

pip install --upgrade mistral_common\[audio\]

Python snippet

from mistral_common.protocol.instruct.messages import TextChunk, AudioChunk, UserMessage, AssistantMessage, RawAudio
from mistral_common.audio import Audio
from huggingface_hub import hf_hub_download

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://<your-server-host>:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

obama_file = hf_hub_download("patrickvonplaten/audio_samples", "obama.mp3", repo_type="dataset")
bcn_file = hf_hub_download("patrickvonplaten/audio_samples", "bcn_weather.mp3", repo_type="dataset")

def file_to_chunk(file: str) -> AudioChunk:
    audio = Audio.from_file(file, strict=False)
    return AudioChunk.from_audio(audio)

text_chunk = TextChunk(text="Which speaker is more inspiring? Why? How are they different from each other?")
user_msg = UserMessage(content=[file_to_chunk(obama_file), file_to_chunk(bcn_file), text_chunk]).to_openai()

print(30 * "=" + "USER 1" + 30 * "=")
print(text_chunk.text)
print("\n\n")

response = client.chat.completions.create(
    model=model,
    messages=[user_msg],
    temperature=0.2,
    top_p=0.95,
)
content = response.choices[0].message.content

print(30 * "=" + "BOT 1" + 30 * "=")
print(content)
print("\n\n")
# The speaker who is more inspiring is the one who delivered the farewell address, as they express
# gratitude, optimism, and a strong commitment to the nation and its citizens. They emphasize the importance of
# self-government and active citizenship, encouraging everyone to participate in the democratic process. In contrast,
# the other speaker provides a factual update on the weather in Barcelona, which is less inspiring as it
# lacks the emotional and motivational content of the farewell address.

# **Differences:**
# - The farewell address speaker focuses on the values and responsibilities of citizenship, encouraging active participation in democracy.
# - The weather update speaker provides factual information about the temperature in Barcelona, without any emotional or motivational content.


messages = [
    user_msg,
    AssistantMessage(content=content).to_openai(),
    UserMessage(content="Ok, now please summarize the content of the first audio.").to_openai()
]
print(30 * "=" + "USER 2" + 30 * "=")
print(messages[-1]["content"])
print("\n\n")

response = client.chat.completions.create(
    model=model,
    messages=messages,
    temperature=0.2,
    top_p=0.95,
)
content = response.choices[0].message.content
print(30 * "=" + "BOT 2" + 30 * "=")
print(content)

Transcription

Voxtral-Mini-3B-2507 has powerful transcription capabilities!

Make sure that your client has mistral-common with audio installed:

pip install --upgrade mistral_common\[audio\]

Python snippet

from mistral_common.protocol.transcription.request import TranscriptionRequest
from mistral_common.protocol.instruct.messages import RawAudio
from mistral_common.audio import Audio
from huggingface_hub import hf_hub_download

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://<your-server-host>:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

obama_file = hf_hub_download("patrickvonplaten/audio_samples", "obama.mp3", repo_type="dataset")
audio = Audio.from_file(obama_file, strict=False)

audio = RawAudio.from_audio(audio)
req = TranscriptionRequest(model=model, audio=audio, language="en", temperature=0.0).to_openai(exclude=("top_p", "seed"))

response = client.audio.transcriptions.create(**req)
print(response)

Transformers 🤗

Starting with transformers >= 4.54.0 and above, you can run Voxtral natively!

Install Transformers:

pip install -U transformers

Make sure to have mistral-common >= 1.8.1 installed with audio dependencies:

pip install --upgrade "mistral-common[audio]"

Audio Instruct

➡️ multi-audio + text instruction

from transformers import VoxtralForConditionalGeneration, AutoProcessor
import torch

device = "cuda"
repo_id = "mistralai/Voxtral-Mini-3B-2507"

processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralForConditionalGeneration.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map=device)

conversation = [
    {
        "role": "user",
        "content": [
            {
                "type": "audio",
                "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/mary_had_lamb.mp3",
            },
            {
                "type": "audio",
                "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/winning_call.mp3",
            },
            {"type": "text", "text": "What sport and what nursery rhyme are referenced?"},
        ],
    }
]

inputs = processor.apply_chat_template(conversation)
inputs = inputs.to(device, dtype=torch.bfloat16)

outputs = model.generate(**inputs, max_new_tokens=500)
decoded_outputs = processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("\nGenerated response:")
print("=" * 80)
print(decoded_outputs[0])
print("=" * 80)

➡️ multi-turn

from transformers import VoxtralForConditionalGeneration, AutoProcessor
import torch

device = "cuda"
repo_id = "mistralai/Voxtral-Mini-3B-2507"

processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralForConditionalGeneration.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map=device)

conversation = [
    {
        "role": "user",
        "content": [
            {
                "type": "audio",
                "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/obama.mp3",
            },
            {
                "type": "audio",
                "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/bcn_weather.mp3",
            },
            {"type": "text", "text": "Describe briefly what you can hear."},
        ],
    },
    {
        "role": "assistant",
        "content": "The audio begins with the speaker delivering a farewell address in Chicago, reflecting on his eight years as president and expressing gratitude to the American people. The audio then transitions to a weather report, stating that it was 35 degrees in Barcelona the previous day, but the temperature would drop to minus 20 degrees the following day.",
    },
    {
        "role": "user",
        "content": [
            {
                "type": "audio",
                "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/winning_call.mp3",
            },
            {"type": "text", "text": "Ok, now compare this new audio with the previous one."},
        ],
    },
]

inputs = processor.apply_chat_template(conversation)
inputs = inputs.to(device, dtype=torch.bfloat16)

outputs = model.generate(**inputs, max_new_tokens=500)
decoded_outputs = processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("\nGenerated response:")
print("=" * 80)
print(decoded_outputs[0])
print("=" * 80)

➡️ text only

from transformers import VoxtralForConditionalGeneration, AutoProcessor
import torch

device = "cuda"
repo_id = "mistralai/Voxtral-Mini-3B-2507"

processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralForConditionalGeneration.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map=device)

conversation = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Why should AI models be open-sourced?",
            },
        ],
    }
]

inputs = processor.apply_chat_template(conversation)
inputs = inputs.to(device, dtype=torch.bfloat16)

outputs = model.generate(**inputs, max_new_tokens=500)
decoded_outputs = processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("\nGenerated response:")
print("=" * 80)
print(decoded_outputs[0])
print("=" * 80)

➡️ audio only

from transformers import VoxtralForConditionalGeneration, AutoProcessor
import torch

device = "cuda"
repo_id = "mistralai/Voxtral-Mini-3B-2507"

processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralForConditionalGeneration.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map=device)

conversation = [
    {
        "role": "user",
        "content": [
            {
                "type": "audio",
                "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/winning_call.mp3",
            },
        ],
    }
]

inputs = processor.apply_chat_template(conversation)
inputs = inputs.to(device, dtype=torch.bfloat16)

outputs = model.generate(**inputs, max_new_tokens=500)
decoded_outputs = processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("\nGenerated response:")
print("=" * 80)
print(decoded_outputs[0])
print("=" * 80)

➡️ batched inference

from transformers import VoxtralForConditionalGeneration, AutoProcessor
import torch

device = "cuda"
repo_id = "mistralai/Voxtral-Mini-3B-2507"

processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralForConditionalGeneration.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map=device)

conversations = [
    [
        {
            "role": "user",
            "content": [
                {
                    "type": "audio",
                    "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/obama.mp3",
                },
                {
                    "type": "audio",
                    "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/bcn_weather.mp3",
                },
                {
                    "type": "text",
                    "text": "Who's speaking in the speach and what city's weather is being discussed?",
                },
            ],
        }
    ],
    [
        {
            "role": "user",
            "content": [
                {
                    "type": "audio",
                    "path": "https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/winning_call.mp3",
                },
                {"type": "text", "text": "What can you tell me about this audio?"},
            ],
        }
    ],
]

inputs = processor.apply_chat_template(conversations)
inputs = inputs.to(device, dtype=torch.bfloat16)

outputs = model.generate(**inputs, max_new_tokens=500)
decoded_outputs = processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("\nGenerated responses:")
print("=" * 80)
for decoded_output in decoded_outputs:
    print(decoded_output)
    print("=" * 80)

Transcription

➡️ transcribe

from transformers import VoxtralForConditionalGeneration, AutoProcessor
import torch

device = "cuda"
repo_id = "mistralai/Voxtral-Mini-3B-2507"

processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralForConditionalGeneration.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map=device)

inputs = processor.apply_transcription_request(language="en", audio="https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/obama.mp3", model_id=repo_id)
inputs = inputs.to(device, dtype=torch.bfloat16)

outputs = model.generate(**inputs, max_new_tokens=500)
decoded_outputs = processor.batch_decode(outputs[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)

print("\nGenerated responses:")
print("=" * 80)
for decoded_output in decoded_outputs:
    print(decoded_output)
    print("=" * 80)

Downloads last month: -; Downloads are not tracked for this model. How to track

MLX

Hardware compatibility

Quantized

Model tree for moonbox-dev/Voxtral-Mini-3B-2507-MLX

Base model

mistralai/Voxtral-Mini-3B-2507

Quantized

(34)

this model

Paper for moonbox-dev/Voxtral-Mini-3B-2507-MLX

Voxtral

Paper • 2507.13264 • Published Jul 17, 2025 • 33