--- library_name: cortiq license: apache-2.0 base_model: - MiniMaxAI/MiniMax-Music3 base_model_relation: quantized pipeline_tag: text-to-audio tags: - cmf - cortiq - music - audio - 4-bit --- # MiniMax-Music-3 → CMF — 20.3 GB of weights in one 5.55 GB file ```bash cortiq music minimax-music3-q4tp.cmf \ --prompt "Classic 1960s soul, passionate male tenor with rich vibrato, \ lush female backing vocals, gospel choir harmonies, vintage Motown and \ Stax atmosphere, groovy bassline, warm Hammond organ, horn section, \ analog tape saturation, romantic nighttime mood." \ --lyrics "[verse] Baby when the midnight comes around I still hear your footsteps on the ground [chorus] Oh, come back to me" \ --seconds 20 --steps 32 --seed 7 --out song.wav ``` **[Listen to that command's output.](https://huggingface.co/infosave/MiniMax-Music-3-cmf/resolve/main/samples/soul_20s_32steps.wav)** One binary, one file, no Python. The lyrics are not English-only — the same command [in Russian](https://huggingface.co/infosave/MiniMax-Music-3-cmf/resolve/main/samples/russian_20s_32steps.wav), caption and all. And steps buy audible quality: the [same soul prompt at 16](https://huggingface.co/infosave/MiniMax-Music-3-cmf/resolve/main/samples/soul_16steps.wav) is where the sibilance stops being distracting, 32 is where it settles. [MiniMax-Music-3](https://huggingface.co/MiniMaxAI/MiniMax-Music3) generates music with vocals from a caption and lyrics. This is its [Comfy-Org repackage](https://huggingface.co/Comfy-Org/MiniMax-Music-3) converted into the [CMF container](https://github.com/infosave2007/cmf): one memory-mapped file holding the AR stack, the DiT and the vocoder, read by `cortiq`, a Rust binary with no ML framework underneath. | | source | here | |---|---|---| | AR stack (Qwen3-8B + RVQ depth decoder) | 15.56 GiB bf16 | — | | flow-matching DiT | 4.58 GiB fp16 | — | | DAV vocoder | 207 MiB | — | | **total** | **20.34 GiB, three files** | **5.55 GiB, one file** | 915 tensors, `cortiq verify` clean. The tokenizer travels inside it. ## What is in the file | stack | packed as | status | |---|---|---| | **AR** — Qwen3-8B backbone (36 layers, 32 q heads / 8 kv of 128), three embedding tables, pruned audio head, 4-layer RVQ depth decoder | `q4tp`, norms exact | **implemented, tested** | | **DiT** — 36 layers, 32×64 heads, GEGLU 8192, flow matching | `q4tp`, norms and both 1×1 convs exact | **implemented, tested** | | **DAV vocoder** — ×8 ×8 ×4 ×2 to 44.1 kHz stereo | **exact throughout** | **implemented, tested** | The vocoder is not quantized on purpose. It buys tens of megabytes and costs audible hiss — a lesson the [H3 conversion](https://huggingface.co/infosave/MiniMax-H3-Turbo-cmf) already paid for once. ## What the conversion had to get right None of these are visible in a tensor name, and each is wrong in a way that still produces plausible output. They came from reading ComfyUI's `comfy/ldm/minimax_music/`, and several of them corrected a guess made from the weights alone. **The vocoder's residual dilations are 1, 3, 9** — not BigVGAN's usual 1, 3, 5. **Its Snake activation reads α verbatim.** The H3 vocoder in the same engine keeps α and β in log scale and exponentiates on load; feeding one model's parameters to the other's loader silently raises the activation to an exponent. Music-3 gets its own plain `Snake` rather than a shared loader. **The 128 latent channels are a stereo pair of 64.** `decode()` folds `[b,128,t]` to `[b·2,64,t]` and unfolds to `[b,2,-1]`. The vocoder config's `latent_channels: 128` against `dec_in_proj [1024,64,1]` reads like evidence of a second VAE sitting between them. There is none — it is left and right. **The DiT's input is `[x | zeros_like(x) | condition]` on the channel axis**, 128 + 128 + 2048 = 2304, which is what `preprocess_conv`'s width was saying. The middle plane is a slot the reference leaves empty, not padding. **Both 1×1 convs are residual** (`conv(x) + x`), the **timestep embedding is prepended as a token** — carried through all 36 blocks, dropped before `project_out`, and shifting every latent frame's rotary position by one — and **the output is negated**. A sampler stepping the wrong way still moves and still decodes. **RoPE covers only the first 32 of each head's 64 dims**, split-half: the pair for `i < 16` is `x[i], x[i+16]`. **The two fused projections are not the same shape.** Every projection in the AR checkpoint is fused where the engine wants them split, and the LM's `qkv_proj` is a GQA fuse — 32 query heads and 8 key/value heads of 128, hence 6144 rows rather than 3×4096 — while the depth decoder's is 12288 = 3×4096 with no grouping. Splitting the first as if it were the second gives a model that loads, runs and is wrong, so the packer DERIVES the key/value head count from the row count instead of assuming it. ## What is verified, and how Not "it produced something". Each stack is gated on quantities the reference fixes exactly, so a stack that has quietly lost an input fails rather than degrades. **Vocoder** — 12 latent frames must decode to exactly 6144 samples per side (512 per frame), inside a `tanh` range, above silence, and **left must differ from right**. That last one is what catches the stereo fold being read as one wide latent, which decodes noise at half the length. Measured: 6144 samples/side, rms 0.065, L−R max 0.177. **DiT** — the velocity must be `[128, n]`, finite, and must *respond*: zeroing the condition has to change it (catches a wrong concatenation) and moving the timestep has to change it (catches a dropped token). Measured: rms 0.563, d/dcond 0.254, d/dt 2.445. **Depth decoder** — it must be CAUSAL, since that is the only thing its attention mask does and losing it lets a codebook level attend to its own answer while the model keeps sampling plausible codes. Measured: 8 codebooks, head spread 6.188. **The chain** — noise → sampler → vocoder has to land on the sample count all three agree about, `frames × 512 × 2`, because the σ walk, the DiT's timestep convention and the vocoder's hop all feed it. And then the ear, which is the only judge of the last mile. The sample above is what came out; by numbers it is 3382 zero crossings a second (music sits in the low thousands, white noise above ten), band energy 0.42/0.25/0.25/0.08 from low to high, and an envelope with real onsets — sd/mean 0.71, 124 of 149 windows active. ## How it generates The AR stack does not encode the prompt, it **generates** the conditioning. A Qwen3-8B backbone is prefilled at batch two — the words, and a copy whose middle is replaced by `<|audio_cfg|>` — then sampled one audio frame at a time at 25 fps: `c0` from the pruned head under classifier-free guidance at 1.5, with the top-k mask taken from the CONDITIONED logits, then seven more codebooks through the depth decoder, each fed back through its own embedding table. The eight hidden states of that frame are what the DiT sees, softmax-mixed by `cond_layer_logits`. Then an ordinary Euler flow walk over the latent — σ from 1 to 0, the DiT asked at `1 − σ`, windowed 689 frames at a time with a 344 hop and the overlap averaged, exactly as the reference does it — and the vocoder turns each latent frame into 512 stereo samples. Two places where this deliberately is not the reference, both marked in the source: - **The top-k sampler is a plain xorshift, not torch's seeded `Generator`.** Reproducing `torch.multinomial` bit-for-bit is its own project, and nothing here needs one seed to mean the same song across implementations — only that a seed means one song in this one. - **The lyrics normaliser skips the reference's markdown scrubbing.** That step only ever removes characters a caption should not carry. ### What it costs Every render prints where its time went. On an RTX 3090, 4 s at 4 steps: | stage | GPU | CPU only | |---|---|---| | AR | 52.3 s | 44.2 s | | denoise | 32.9 s | 36.2 s | | vocoder | **18.2 s** | 31.3 s | | total | **103.5 s** | 111.8 s | The vocoder was 54.0 s until its convolutions stopped shipping their column matrix across the bus — the host built it, transposed it into a second buffer of the same size and uploaded that, up to 2.37 GB for a 20-second song, when the input it expands from is `k` times smaller. It is expanded on the card now. Two things that table will not tell you. The CPU column is a 256-core EPYC, so an ordinary machine's fallback is far slower than this and the device gap far wider. And the shape that matters for real songs is not this one: attention over latent frames is quadratic, so a 20-second render at 32 steps spends about **80% of its time in the denoise**, around 32-40 s a step. `CMF_MUSIC3_PROF=1` splits a step four ways if you want to see it. Earlier, on an Apple M4, 5 s at 8 steps took 206 s: AR 0.45 s/frame, denoise 6.5 s/step over 430 latent frames, vocoder 19 s — and the vocoder was 75 s before it was handed the thread pool. ## Running it ```bash cargo install cortiq-cli # 0.5.74+ hf download infosave/MiniMax-Music-3-cmf minimax-music3-q4tp.cmf --local-dir . cortiq verify minimax-music3-q4tp.cmf # → ✓ all tensor hashes match cortiq music minimax-music3-q4tp.cmf --prompt "..." --lyrics "..." \ --seconds 20 --steps 32 --seed 7 --out song.wav ``` `--seconds` is a ceiling: the model can stop earlier. Same seed, same prompt, same song. Repacking from the original sources: ```bash cortiq animate-pack \ --music-te minimax_music3_text_encoder_pruned_bf16.safetensors \ --music-dit minimax_music3_dit_fp16.safetensors \ --music-vae minimax_music3_dav.safetensors \ --quant q4tp --out minimax-music3-q4tp.cmf ``` ## Provenance Weights derive from MiniMax's Music-3 release as repackaged by Comfy-Org, under their own licence. The conversion conventions were read from ComfyUI's `comfy/ldm/minimax_music/` and MiniMax's own configs. The CMF container and the cortiq runtime are Apache-2.0.