README.md

---
base_model: MiniMaxAI/MiniMax-Music3
license: other
license_name: minimax-music3-community-license
license_link: https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/LICENSE
pipeline_tag: text-to-audio
library_name: mlx
tags:
  - mlx
  - apple-silicon
  - quantized
  - music-generation
  - text-to-music
  - minimax-music3
---

# MiniMax-Music3 language model, MLX 4-bit

The autoregressive language model of [MiniMaxAI/MiniMax-Music3](https://huggingface.co/MiniMaxAI/MiniMax-Music3),
converted to MLX and quantized, plus the bridge code that plugs it back into the
original `diffusers` pipeline.

Upstream states that inference requires CUDA and that CPU and MPS are not supported.
On a 24 GB Apple Silicon machine the bf16 pipeline does not merely run slowly, it
never reaches generation: 21.6 GB of weights plus activations plus the OS puts the
machine into full swap. This repo removes that wall by replacing the single
component responsible for 80% of the weight footprint.

**This is a memory port, not a speed port.** Read the benchmarks before you decide
whether it is useful to you.

## What is in here

| File | Content |
|---|---|
| `model.safetensors*`, `config.json` | Qwen3 8.6B language model, MLX quantized, 5.65 GB |
| `mlx_bridge.py` | Torch-facing shim so the `diffusers` pipeline can call the MLX model |
| `convert_lm_mlx.py` | The conversion script, if you want to redo it at other bit widths |
| `generate.py` | End-to-end generation script |
| `LICENSE` | MiniMax-Music3 Community License, unchanged |

The other components (flow-matching transformer, RVQ depth decoder, Flow-VAE
vocoder, condition encoder) are **not** redistributed here. They are used unmodified
in bf16 from the upstream repo.

## Why only the language model

The pipeline touches `Qwen3ForCausalLM` at exactly three points:

```python
language_model.model.embed_tokens(ids)
language_model.model(inputs_embeds=..., past_key_values=..., use_cache=True)
language_model.lm_head(hidden)
```

That is a small enough surface to reimplement on top of an MLX model, which is what
`mlx_bridge.py` does. Everything else stays in torch/MPS, so the numerical path of
the diffusion stage is untouched.

| Component | Upstream bf16 | Here |
|---|---|---|
| language model (Qwen3 8.6B) | 17.2 GB | 5.65 GB, MLX |
| transformer, RVQ decoder, vocoder, condition encoder | 5.6 GB | unchanged, torch/MPS |
| **resident weights** | **21.6 GB** | **11.3 GB** |

Peak process footprint during a 30 s generation: **15.7 GB**, no swap.

## Quantization

| Tensors | Bits | Group size |
|---|---|---|
| transformer block linears | 4 | 64 |
| `embed_tokens`, `lm_head` | 8 | 64 |

5.264 bits per weight overall. The two embedding tables are kept at 8 bits because
every audio code the model emits is looked up and scored through them, and they are
only 1.6 GB of the total at that width.

## Benchmarks

Apple M4 Pro, 24 GB unified memory, macOS 25.6, torch 2.13, MLX 0.32.
30 s of audio, 30 denoising steps, measured with explicit `torch.mps.synchronize()`.

Component load takes 10.5 s. Generation of 30 s of audio takes **356 to 413 s**
across runs, which is **12x to 14x slower than realtime**. A 90 s track takes
roughly 20 minutes.

Where that time goes, per stage:

| Stage | Time | Share | Backend |
|---|---|---|---|
| flow-matching transformer | 265.7 s | 74.6% | torch/MPS |
| autoregressive loop, total | 81.0 s | 22.7% | mixed |
|   of which RVQ depth decoder | 56.9 s | 16.0% | torch/MPS |
|   of which MLX language model | 16.9 s | 4.7% | MLX |
|   of which `lm_head` | 3.1 s | 0.9% | MLX |
|   of which top-k sampling | 3.9 s | 1.1% | torch/MPS |
| vocoder | 9.2 s | 2.6% | torch/MPS |
| condition encoder | 0.3 s | 0.1% | torch/MPS |

Per-operation costs behind those totals:

| Operation | Cost |
|---|---|
| flow transformer, one denoising forward (batch 1) | 572.7 ms |
| RVQ depth decoder, one depth step | 10.0 ms |
| MLX language model, one frame | 22.5 ms |
| vocoder, one window | 1078.8 ms |
| condition encoder, one window | 3.3 ms |

The language model is **not** the bottleneck once quantized: MLX accounts for about
6% of wall time. The flow-matching transformer dominates, running 2 forwards per
denoising step because classifier-free guidance is executed as two separate batch-1
passes. Batching them into one batch-2 forward was measured and gains 1.1%, so that
stage is compute-bound rather than launch-bound. Making this fast on Apple Silicon
means porting the flow transformer to MLX, which this repo does not do. Second
target after that would be the RVQ depth decoder, whose 7 sequential steps per audio
frame cost 3x more than the language model they feed.

Measure with `torch.mps.synchronize()` around anything you time here. Torch is
asynchronous on MPS, and without explicit syncs the cost of the denoising loop is
attributed to whichever later call happens to drain the queue. An earlier version of
this profile reported 219 s in the condition encoder, a 25M-parameter Conv1d whose
real cost is 0.3 s.

If you have a 24 GB CUDA GPU, use the upstream repo instead. It will be far faster.

## Usage

```bash
# 1. upstream components (diffusers subset, ~28.5 GB)
hf download MiniMaxAI/MiniMax-Music3 --local-dir models/MiniMax-Music3 \
  --include "condition_encoder/*" "language_model/*" "rvq_depth_decoder/*" \
  "scheduler/*" "tokenizer/*" "transformer/*" "vocoder/*"
hf download MiniMaxAI/MiniMax-Music3 modular_model_index.json config.json \
  --local-dir models/MiniMax-Music3

# 2. this repo
hf download elishabjm/MiniMax-Music3-MLX --local-dir models/lm-mlx

# 3. dependencies
pip install "git+https://github.com/huggingface/diffusers@dafe3733fcfdbf3c48915fe77be3aef65b5d6a2d" \
  transformers accelerate soundfile torch mlx mlx-lm

# 4. generate
python generate.py -p caption.txt -l lyrics.txt -d 60 --steps 30 -o song.wav
```

You can skip downloading `language_model/*` in step 1 if you only ever use the MLX
path. It is listed above because `--lm torch` falls back to it.

### Two gotchas that will cost you an hour otherwise

`modular_model_index.json` points every component at the Hub repo id rather than at
your local directory, so the pipeline re-downloads 28 GB even though the weights are
already on disk. `generate.py` rewrites those paths on startup.

The decoder returns `(channels, samples)`; `soundfile` expects `(samples, channels)`.
Writing the array as-is fails with `Format not recognised`.

## Prompting

Unchanged from upstream. The caption drives the sound, the lyrics only get sung. The
official `music-caption-rewriter` skill in the
[MiniMax-AI/MiniMax-Music3](https://github.com/MiniMax-AI/MiniMax-Music3) GitHub repo
carries a genre router and 1000 reference captions, and is worth reading before
writing prompts by hand.

## Quality

No formal evaluation was run against the bf16 reference, because the bf16 reference
does not fit on the hardware this port targets. Spectral checks on 30 s outputs are
consistent with music rather than noise (13.9% of energy below 200 Hz, 34.6% in
200 Hz to 2 kHz, 17 dB crest factor, structured RMS envelope). Treat that as a
sanity check, not as a quality claim. If you have a machine that can run both, an
A/B would be a genuinely useful contribution.

## License and attribution

MiniMax-Music3 Community License, reproduced unchanged in `LICENSE`. The copyright
notice must travel with any copy. Commercial use requires displaying
"MiniMax-Music3" in the product interface, and separate written authorization from
MiniMax above 20M USD yearly revenue.

Upstream components carry their own lineage: the language model derives from
Qwen3-8B (Apache 2.0), the DiT from Stable Audio Open, the VAE from a modified
Stability AI design. See the upstream LICENSE for the full chain.

All credit for the model belongs to MiniMax. This repo contributes a quantization
and a compatibility shim, nothing more.