File size: 3,514 Bytes
02bebe1
 
5415f1a
 
 
 
 
 
02bebe1
5415f1a
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ecf394b
 
5415f1a
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ecf394b
5415f1a
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
---
license: cc-by-nc-4.0
base_model: facebook/musicgen-medium
pipeline_tag: text-to-audio
tags:
- music-generation
- emotion-control
- arxiv:2510.05875
---
# LARA-Gen

Inference release for [LARA-Gen: Enabling Continuous Emotion Control for Music Generation Models via Latent Affective Representation Alignment](https://arxiv.org/abs/2510.05875).

`model.safetensors` and `av_encoder.safetensors` together contain the final-paper main checkpoint: experiment `59dc26ab`, epoch 30, embedded `best_state`. Both files are required. The included `laragen/` code implements the corresponding streaming Transformer and continuous VA conditioning.

## Install and generate

Use Python 3.9 or 3.10, install FFmpeg with your system package manager, then run from this directory:

```bash
python -m pip install -r requirements.txt
python generate.py --arousal 7.4 --valence 5.8 --seed 204000 --output outputs/example.wav
```

Use matched PyTorch/torchaudio wheels for your platform. CUDA is recommended; `--device cpu` supports slow CPU inference. The WAV is 30 seconds, 32 kHz, mono, with loudness normalization. `--duration` accepts 0.5–30 seconds. A JSON sidecar records the inputs and seed.

Attention uses native PyTorch SDPA with `backend='torch'` and `memory_efficient=true`. No xformers installation is required. Streaming generation retains its KV cache, including support for decoding multiple new tokens per call.

Arousal and valence are continuous values in **[1, 9]**. Numeric conditioning uses `[arousal, valence]`, normalized by `(x-5)/4`. The original prompt spelling is preserved: `Genarate an instrumental music with arousal 7.40 and valence 5.80.` Sampling uses temperature 1, top-k 250, top-p 0 and CFG 3. Paper samples use seeds `204000 + sample index`.

```python
from laragen import LaraGen
model = LaraGen('.', device='cuda')
audio = model.generate(arousal=7.4, valence=5.8, seed=204000)
# [1, 1, 960000], before WAV loudness normalization
```

## Frozen dependencies

On first use the loader downloads the original frozen components from their public repositories:

- T5-base: `t5-base`, revision `a9723ea7f1b39c1eae772870f3b547bf6ef7e6c1`.
- EnCodec: `facebook/encodec_32khz`, revision `d0c45384f6c44db055f78200cfdcb9c1c8706727`.

For offline inference, download these snapshots on a connected machine:

```bash
hf download t5-base --revision a9723ea7f1b39c1eae772870f3b547bf6ef7e6c1 --include config.json model.safetensors spiece.model tokenizer.json tokenizer_config.json special_tokens_map.json --local-dir dependencies/t5-base
hf download facebook/encodec_32khz --revision d0c45384f6c44db055f78200cfdcb9c1c8706727 --include config.json model.safetensors --local-dir dependencies/encodec
export LARAGEN_T5_PATH="$PWD/dependencies/t5-base"
export LARAGEN_CODEC_PATH="$PWD/dependencies/encodec"
export HF_HUB_OFFLINE=1
export TRANSFORMERS_OFFLINE=1
```

Then run `generate.py` as above. The loader uses FP32 modules with FP16 LM autocast on CUDA. Validated without xformers using PyTorch 2.1.1: CPU attention/streaming checks, CUDA FP16 streaming checks, and full 30-second generation on an NVIDIA A100 (32 kHz, mono, 960,000 samples).

## License

Model weights: [CC BY-NC 4.0](LICENSE), inherited from [MusicGen](https://huggingface.co/facebook/musicgen-medium). Code: [MIT](LICENSE-CODE), with Meta/AudioCraft notices retained in adapted sources. External [T5](https://huggingface.co/google-t5/t5-base) and [EnCodec](https://huggingface.co/facebook/encodec_32khz) retain their own licenses.