File size: 3,258 Bytes
be2c293 f97f2d7 be2c293 bbfb389 be2c293 f97f2d7 be2c293 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
license: cc-by-nc-4.0
base_model: m-a-p/YuE2-3B
base_model_relation: quantized
library_name: mlx
pipeline_tag: text-to-audio
tags:
- mlx
- music-generation
- yue2
language:
- en
- zh
---
# YuE2-3B MLX-with-Hum-Encoder
Native [MLX](https://github.com/ml-explore/mlx) port of [m-a-p/YuE2-3B](https://huggingface.co/m-a-p/YuE2-3B) (AR/NAR
Mixture-of-Transformers music generation) with the [YuE2-Vae](https://huggingface.co/m-a-p/YuE2-Vae) decoder folded in.
**This Repo contains an additional Vae Encoder model**, which is used for the Hum lora feature in my pinokio app: https://github.com/Rdx-ai-art/yue2-mlx.pinokio.
Runs on Apple Silicon with no PyTorch dependency. Lyrics + style prompt in, 48 kHz stereo song out, with optional
editable ABC score planning.
Upstream repo : https://huggingface.co/ahmadw/YuE2-3B-MLX
## Variants
| Folder | Backbone | Size | AR logits / NAR velocity error vs torch bf16 |
|---|---|---|---|
| `bf16/` | bf16 | 7.0 GB | 1.4% / 1.6% |
| `8bit/` | 8-bit affine, group size 64, all backbone Linear layers | 4.2 GB | 1.0% / 1.9% |
| `4bit/` | 4-bit AR path, 8-bit NAR path, group size 64 | 3.4 GB | 8.1% / 4.4% |
The NAR (flow-matching) path stays at 8 bits in the 4-bit variant because fully 4-bit flow matching drifts
(12% velocity error). VAE decode is bit-exact in all variants. `8bit/` is the recommended default: near-bf16 output
at roughly twice the decode speed.
Each folder contains `model.safetensors`, `vae.safetensors`, `config.json`, `vae_config.json`, `qwen.tiktoken`
and `yue2_generation_config.json`. The Python files at the repo root are the inference code.
## Usage
```bash
pip install mlx tiktoken numpy huggingface_hub
hf download ahmadw/YuE2-3B-MLX --include "*.py" "8bit/*" --local-dir YuE2-3B-MLX
cd YuE2-3B-MLX
python generate.py --model 8bit \
--style "English, indie pop, bright acoustic guitar, soft drums, warm lead vocal" \
--lyrics "[Verse]
Soft morning light is touching the window.
[Chorus]
Stay with the rhythm, let it carry us home." \
--cot full --seed 831001 --out song.wav
```
Options: `--cot off|melody|full` (symbolic ABC planning; `full` is the upstream default), `--abc-file` to supply your
own score, `--cfg-scale`, `--steps` (NAR midpoint steps, default 32), `--max-semantic-tokens` (song length cap, 25
frames per second), `--decode-latents <out>.latents.npy` to re-decode a saved run. A generated ABC score is written
next to the wav.
Measured on an M-series Mac: bf16 decodes at about 70 tokens/s (35 with CFG), 8bit at about 80 with CFG.
A three-minute song takes a few minutes end to end.
## Converting yourself
```bash
python convert.py --output YuE2-3B-MLX/8bit -q --bits 8 # or --bits 4 (NAR stays 8-bit), or no -q for bf16
python test_parity.py --mlx YuE2-3B-MLX/8bit # needs torch + transformers
```
Sampling uses MLX's RNG, so seeds reproduce within MLX only.
## License
Weights are derived from m-a-p/YuE2-3B and YuE2-Vae and inherit their **CC BY-NC 4.0** license (non-commercial).
The SnakeBeta activation and stable-audio-tools decoder code are MIT-licensed by NVIDIA and Stability AI respectively;
see the upstream `THIRD_PARTY_NOTICES.md`. Please cite the original YuE2 authors (m-a-p) when using these weights.
|