File size: 2,478 Bytes
9781faf | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
license: mit
---
# Audio DiT
Tag-conditioned music generation focused on instrumental game and anime songs. Choose 1-8 tags (recommend to start with 1). Generates 95s songs.
Set a seed to reproduce a generation, or leave it at `-1` for a random seed.
## Run it
Needs [uv](https://docs.astral.sh/uv/). CUDA, MPS or CPU.
```bash
hf download Localsong/Localsong --local-dir Localsong
uv run webui.py # http://127.0.0.1:7860
```
## Architecture
```
tags ──► AudioDiT ──► 256×1024 latents ──► SAME-L decoder ──► 95 s of stereo audio
0.84B params
```
**Latents.** [SAME-L](https://huggingface.co/stabilityai/SAME-L) autoencoder by Stability AI.
**Denoiser.** `audio_dit.py` is a 1-D latent transformer DiT.
RMSNorm and QK-RMSNorm, RoPE, SwiGLU
Timestep conditioning is adaLN-single (PixArt)
offset. Tags condition via cross-attention.
**Sparse-dense fusion** ([SPRINT](https://arxiv.org/abs/2510.21986)). The blocks
are split into a dense 2-block encoder, a 20-block middle stack, and a dense
2-block decoder. The encoder output is concatenated with the middle output and
projected back down before the decoder:
```
frames ──► encoder ──┬─────────────────────────► f ──┐
└─► middle (20 blocks) ──► g ──► linear([f;g]) ──► decoder ──► velocity
```
Training ran the middle stack on a random 25 % of frames and sometimes dropped
it entirely, which makes the shallow encoder→fusion→decoder route a usable weak
model on its own. Inference runs the middle stack on every frame
(`AudioDiT.forward`) and keeps the shallow route as the guidance branch
(`AudioDiT.shallow`).
**Sampling.** Rectified flow: v-prediction, 50 steps Euler shift=2
Guidance alternates between normal CFG and the shallow-path PDG.
## License
MIT, see [LICENSE](LICENSE).
`same_l_decoder.py` is adapted from Stability AI's
[stable-audio-3](https://github.com/Stability-AI/stable-audio-3) (MIT). The
SAME-L weights it loads are released by Stability AI under the Stable Audio
Community License; commercial use of them is subject to
<https://stability.ai/license>.
The model also builds on SPRINT ([2510.21986](https://arxiv.org/abs/2510.21986)),
PixArt-α ([2310.00426](https://arxiv.org/abs/2310.00426)),
DiT ([2212.09748](https://arxiv.org/abs/2212.09748)) and
rectified flow ([2209.03003](https://arxiv.org/abs/2209.03003)). |