A community fine-tune of YuE2-3B (~1,860 hours), public merges, and what a pilot taught us · Ruach Studio
Thank you again for YuE2. Since our first note, Ruach Studio has grown from a studio into a small lab around the model. This thread is about the weights: what we merged, what we are training, and what we measured on the way. All of it is open and keeps your licence (CC BY-NC 4.0).
Merges you can try (goldhub/YuE2-3B-Ruach_Studio-IRA-Merge):
- IRA: Mothersuperior's Instrumental on the music half (0.75) and Real Audio v9 on the sound half (0.85).
- IRAE, IRAEv2, IRAEv3: IRA with monsterovich's Epic Trailer adapter on top, at 0.32 / 0.32, 0.28 / 0.24 and 0.5 / 0.375 (music / sound).
- Safetensors in your layout (the IRAE ones in subfolders) and GGUF BF16, Q8_0 and Q6_K for yue2.cpp.
- Folding a LoRA into the weights gives the same audio as loading it at runtime, byte for byte, with the same seed and score.
A first wave of fine-tuning, YuE2-RS-FW (it starts on 22 October 2026; the results will land in goldhub/YuE2-RS-Finetunes):
- About 32,600 tracks, about 1,860 hours: MTG-Jamendo and FMA above a production-quality bar (Audiobox Aesthetics), 5,000 songs with lyrics from community sets on the Hub, and a guard of sung and spoken material so the model keeps what it already does well.
- Every track has its own ABC score (SheetSage2), so the wave also teaches Full mode. The lyrics come from the sources, else from Whisper, checked against LRCLIB where it had them.
- The style tags follow the model's order and carry emotion words read from the audio by LAION's Empathic Insight heads.
- LoRA on both experts: rank 192 on the music half, 384 on the sound half, two epochs on three RTX 3090s. The music half is held near the base by a KL term and half the learning rate, so the words stay put. At rank 192/384 a card peaks at 21.1 GB, 3.7 s a step.
What a pilot taught us (1,000 tracks, rank 128/256, three short epochs; judged blind by ear, with a hidden copy of the stock model among the numbered takes):
- The sound half is the one that overcooks. With music 0.75 / sound 0.75 the pilot scored +9; music 1.0 / sound 0.5 scored +5; music 0.5 / sound 1.0 scored −2, the same as the hidden stock model. So we merge with the sound half at 0.75 or below.
- By ear it brought a bass and sub-bass the stock weights don't have.
- Full beat Direct clearly in the same kind of test (55 points against 28).
- Machine aesthetics scores did not follow the ear: Audiobox's correlation with the listener was about −0.1. We use such scores to filter data, never to choose what we release.
Other findings since #29:
- The score's Vocal line is written 1.5–2 octaves above what is sung (measured on our reference takes). An absolute note range read from the score says little about the singer.
- Adapters' music halves have ceilings. Epic Trailer broke 9 scores in 9 at 1.0 (no
K:line) and held 18 in 18 at 0.5 and below. An industrial-rock adapter's music half broke plans above about 0.45. - YuE2 runs on an Intel Arc Pro B50 through yue2.cpp's SYCL build (a beta in our studio). It needs
SYCL_CACHE_PERSISTENT=0: the persistent kernel cache crashed it. On the Arc, BF16 was faster than Q8_0 and Q5_K_M.
A question for m-a-p. The official audio-to-semantic tokenizer is still unreleased; training conditioning today comes from a community head (Mothersuperior's realaudio tokenizer). If you plan to release yours, we would train the next wave on it and share what we find.
Thanks to Mothersuperior and monsterovich for their adapters, to Serveurperso for yue2.cpp, to Ostris for AI-Toolkit and to IronWolve for the YuE2 Kit.