base_model: MiniMaxAI/MiniMax-Music3
license: other
license_name: minimax-music3-community-license
license_link: https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/LICENSE
pipeline_tag: text-to-audio
library_name: mlx
tags:
- mlx
- apple-silicon
- quantized
- music-generation
- text-to-music
- minimax-music3
MiniMax-Music3 language model, MLX 4-bit
The autoregressive language model of MiniMaxAI/MiniMax-Music3,
converted to MLX and quantized, plus the bridge code that plugs it back into the
original diffusers pipeline.
Upstream states that inference requires CUDA and that CPU and MPS are not supported. On a 24 GB Apple Silicon machine the bf16 pipeline does not merely run slowly, it never reaches generation: 21.6 GB of weights plus activations plus the OS puts the machine into full swap. This repo removes that wall by replacing the single component responsible for 80% of the weight footprint.
This is a memory port, not a speed port. Read the benchmarks before you decide whether it is useful to you.
What is in here
| File | Content |
|---|---|
model.safetensors*, config.json |
Qwen3 8.6B language model, MLX quantized, 5.65 GB |
mlx_bridge.py |
Torch-facing shim so the diffusers pipeline can call the MLX model |
convert_lm_mlx.py |
The conversion script, if you want to redo it at other bit widths |
generate.py |
End-to-end generation script |
LICENSE |
MiniMax-Music3 Community License, unchanged |
The other components (flow-matching transformer, RVQ depth decoder, Flow-VAE vocoder, condition encoder) are not redistributed here. They are used unmodified in bf16 from the upstream repo.
Why only the language model
The pipeline touches Qwen3ForCausalLM at exactly three points:
language_model.model.embed_tokens(ids)
language_model.model(inputs_embeds=..., past_key_values=..., use_cache=True)
language_model.lm_head(hidden)
That is a small enough surface to reimplement on top of an MLX model, which is what
mlx_bridge.py does. Everything else stays in torch/MPS, so the numerical path of
the diffusion stage is untouched.
| Component | Upstream bf16 | Here |
|---|---|---|
| language model (Qwen3 8.6B) | 17.2 GB | 5.65 GB, MLX |
| transformer, RVQ decoder, vocoder, condition encoder | 5.6 GB | unchanged, torch/MPS |
| resident weights | 21.6 GB | 11.3 GB |
Peak process footprint during a 30 s generation: 15.7 GB, no swap.
Quantization
| Tensors | Bits | Group size |
|---|---|---|
| transformer block linears | 4 | 64 |
embed_tokens, lm_head |
8 | 64 |
5.264 bits per weight overall. The two embedding tables are kept at 8 bits because every audio code the model emits is looked up and scored through them, and they are only 1.6 GB of the total at that width.
Benchmarks
Apple M4 Pro, 24 GB unified memory, macOS 25.6, torch 2.13, MLX 0.32.
30 s of audio, 30 denoising steps, measured with explicit torch.mps.synchronize().
Component load takes 10.5 s. Generation of 30 s of audio takes 356 to 413 s across runs, which is 12x to 14x slower than realtime. A 90 s track takes roughly 20 minutes.
Where that time goes, per stage:
| Stage | Time | Share | Backend |
|---|---|---|---|
| flow-matching transformer | 265.7 s | 74.6% | torch/MPS |
| autoregressive loop, total | 81.0 s | 22.7% | mixed |
| of which RVQ depth decoder | 56.9 s | 16.0% | torch/MPS |
| of which MLX language model | 16.9 s | 4.7% | MLX |
of which lm_head |
3.1 s | 0.9% | MLX |
| of which top-k sampling | 3.9 s | 1.1% | torch/MPS |
| vocoder | 9.2 s | 2.6% | torch/MPS |
| condition encoder | 0.3 s | 0.1% | torch/MPS |
Per-operation costs behind those totals:
| Operation | Cost |
|---|---|
| flow transformer, one denoising forward (batch 1) | 572.7 ms |
| RVQ depth decoder, one depth step | 10.0 ms |
| MLX language model, one frame | 22.5 ms |
| vocoder, one window | 1078.8 ms |
| condition encoder, one window | 3.3 ms |
The language model is not the bottleneck once quantized: MLX accounts for about 6% of wall time. The flow-matching transformer dominates, running 2 forwards per denoising step because classifier-free guidance is executed as two separate batch-1 passes. Batching them into one batch-2 forward was measured and gains 1.1%, so that stage is compute-bound rather than launch-bound. Making this fast on Apple Silicon means porting the flow transformer to MLX, which this repo does not do. Second target after that would be the RVQ depth decoder, whose 7 sequential steps per audio frame cost 3x more than the language model they feed.
Measure with torch.mps.synchronize() around anything you time here. Torch is
asynchronous on MPS, and without explicit syncs the cost of the denoising loop is
attributed to whichever later call happens to drain the queue. An earlier version of
this profile reported 219 s in the condition encoder, a 25M-parameter Conv1d whose
real cost is 0.3 s.
If you have a 24 GB CUDA GPU, use the upstream repo instead. It will be far faster.
Usage
# 1. upstream components (diffusers subset, ~28.5 GB)
hf download MiniMaxAI/MiniMax-Music3 --local-dir models/MiniMax-Music3 \
--include "condition_encoder/*" "language_model/*" "rvq_depth_decoder/*" \
"scheduler/*" "tokenizer/*" "transformer/*" "vocoder/*"
hf download MiniMaxAI/MiniMax-Music3 modular_model_index.json config.json \
--local-dir models/MiniMax-Music3
# 2. this repo
hf download elishabjm/MiniMax-Music3-MLX --local-dir models/lm-mlx
# 3. dependencies
pip install "git+https://github.com/huggingface/diffusers@dafe3733fcfdbf3c48915fe77be3aef65b5d6a2d" \
transformers accelerate soundfile torch mlx mlx-lm
# 4. generate
python generate.py -p caption.txt -l lyrics.txt -d 60 --steps 30 -o song.wav
You can skip downloading language_model/* in step 1 if you only ever use the MLX
path. It is listed above because --lm torch falls back to it.
Two gotchas that will cost you an hour otherwise
modular_model_index.json points every component at the Hub repo id rather than at
your local directory, so the pipeline re-downloads 28 GB even though the weights are
already on disk. generate.py rewrites those paths on startup.
The decoder returns (channels, samples); soundfile expects (samples, channels).
Writing the array as-is fails with Format not recognised.
Prompting
Unchanged from upstream. The caption drives the sound, the lyrics only get sung. The
official music-caption-rewriter skill in the
MiniMax-AI/MiniMax-Music3 GitHub repo
carries a genre router and 1000 reference captions, and is worth reading before
writing prompts by hand.
Quality
No formal evaluation was run against the bf16 reference, because the bf16 reference does not fit on the hardware this port targets. Spectral checks on 30 s outputs are consistent with music rather than noise (13.9% of energy below 200 Hz, 34.6% in 200 Hz to 2 kHz, 17 dB crest factor, structured RMS envelope). Treat that as a sanity check, not as a quality claim. If you have a machine that can run both, an A/B would be a genuinely useful contribution.
License and attribution
MiniMax-Music3 Community License, reproduced unchanged in LICENSE. The copyright
notice must travel with any copy. Commercial use requires displaying
"MiniMax-Music3" in the product interface, and separate written authorization from
MiniMax above 20M USD yearly revenue.
Upstream components carry their own lineage: the language model derives from Qwen3-8B (Apache 2.0), the DiT from Stable Audio Open, the VAE from a modified Stability AI design. See the upstream LICENSE for the full chain.
All credit for the model belongs to MiniMax. This repo contributes a quantization and a compatibility shim, nothing more.