Improve model card and add sample usage

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +36 -18
README.md CHANGED
@@ -1,48 +1,67 @@
1
  ---
2
- license: cc-by-nc-4.0
3
  language:
4
  - zh
5
  - en
6
  - ja
7
  - ko
8
  - es
 
9
  pipeline_tag: text-to-audio
10
  tags:
11
  - music
12
  - art
13
  ---
14
 
15
- # HeartMuLa
16
 
 
 
 
 
 
 
17
 
18
  ## Model Details
19
 
20
- ### Model Description
 
 
 
 
21
 
22
- <!-- Provide a longer summary of what this model is. -->
23
 
24
- - **Developed by:** [Ario Scale Global]
25
- - **License:** [CC-BY-NC 4.0]
26
 
27
- ### Model Sources
 
 
 
 
28
 
29
- <!-- Provide the basic links for the model. -->
30
 
31
- - **Github Repo:** https://github.com/HeartMuLa/heartlib
32
- - **Paper:** https://arxiv.org/abs/2601.10547
33
- - **Demo:** https://heartmula.github.io/
34
- - **HeartMuLa-oss-3B:** https://huggingface.co/HeartMuLa/HeartMuLa-oss-3B
35
- - **HeartCodec-oss:** https://huggingface.co/HeartMuLa/HeartCodec-oss
36
- - **HeartTranscriptor-oss:** https://huggingface.co/HeartMuLa/HeartTranscriptor-oss
37
 
 
 
38
 
39
- ## Get Started
 
 
 
 
40
 
41
- Check our github repo https://github.com/HeartMuLa/heartlib for a quickstart and local deployment of HeartMuLa.
 
 
 
 
 
42
 
43
  ## Citation
44
 
45
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
46
  If you find HeartMuLa useful, please cite:
47
 
48
  ```bibtex
@@ -57,6 +76,5 @@ If you find HeartMuLa useful, please cite:
57
  }
58
  ```
59
 
60
-
61
  ## Contact
62
  If you are interested in HeartMuLa, feel free to reach us at heartmula.ai@gmail.com
 
1
  ---
 
2
  language:
3
  - zh
4
  - en
5
  - ja
6
  - ko
7
  - es
8
+ license: cc-by-nc-4.0
9
  pipeline_tag: text-to-audio
10
  tags:
11
  - music
12
  - art
13
  ---
14
 
15
+ # HeartMuLa: A Family of Open Sourced Music Foundation Models
16
 
17
+ HeartMuLa is a family of open-source music foundation models designed to advance large-scale music understanding and generation across diverse tasks and modalities. It is an LLM-based song generation model capable of synthesizing high-fidelity music under rich, user-controllable conditions (e.g., textual style descriptions, lyrics, and reference audio).
18
+
19
+ - **Project Page:** [https://heartmula.github.io/](https://heartmula.github.io/)
20
+ - **Repository:** [https://github.com/HeartMuLa/heartlib](https://github.com/HeartMuLa/heartlib)
21
+ - **Paper:** [HeartMuLa: A Family of Open Sourced Music Foundation Models](https://arxiv.org/abs/2601.10547)
22
+ - **Demo:** [https://heartmula.github.io/](https://heartmula.github.io/)
23
 
24
  ## Model Details
25
 
26
+ The HeartMuLa framework consists of four major components:
27
+ 1. **HeartMuLa**: A music language model that generates music conditioned on lyrics and tags with multilingual support.
28
+ 2. **HeartCodec**: A low-frame-rate (12.5 Hz), high-fidelity music codec tokenizer that captures long-range musical structure.
29
+ 3. **HeartTranscriptor**: A robust lyric recognition model optimized for real-world music scenarios.
30
+ 4. **HeartCLAP**: An audio-text alignment model for music descriptions and cross-modal retrieval.
31
 
32
+ ## Installation
33
 
34
+ We recommend using `python=3.10` for local deployment. Clone the repository and install locally:
 
35
 
36
+ ```bash
37
+ git clone https://github.com/HeartMuLa/heartlib.git
38
+ cd heartlib
39
+ pip install -e .
40
+ ```
41
 
42
+ ## Sample Usage
43
 
44
+ ### Download Checkpoints
45
+ First, download the pretrained checkpoints into a `./ckpt` folder as described in the [GitHub README](https://github.com/HeartMuLa/heartlib).
 
 
 
 
46
 
47
+ ### Inference
48
+ To generate music conditioned on lyrics and tags, run the following command:
49
 
50
+ ```bash
51
+ python ./examples/run_music_generation.py --model_path=./ckpt --version="3B"
52
+ ```
53
+
54
+ By default, this command generates a piece of music based on the lyrics and tags provided in the `./assets` folder. The output will be saved as `./assets/output.mp3`.
55
 
56
+ **Key Parameters:**
57
+ - `--model_path`: Path to the pretrained model checkpoint.
58
+ - `--lyrics`: Path to the lyrics file (e.g., `./assets/lyrics.txt`).
59
+ - `--tags`: Path to the tags file (e.g., `./assets/tags.txt`).
60
+ - `--save_path`: Output audio file path.
61
+ - `--version`: The version of HeartMuLa (choose `3B`).
62
 
63
  ## Citation
64
 
 
65
  If you find HeartMuLa useful, please cite:
66
 
67
  ```bibtex
 
76
  }
77
  ```
78
 
 
79
  ## Contact
80
  If you are interested in HeartMuLa, feel free to reach us at heartmula.ai@gmail.com