nielsr HF Staff commited on
Commit
34f5ee4
·
verified ·
1 Parent(s): 6d36de0

Improve model card and add sample usage

Browse files

Hi! I'm Niels from the Hugging Face community science team.

This PR improves the model card for HeartMuLa by:
- Adding a more detailed description of the model components (HeartMuLa, HeartCodec, etc.).
- Organizing links to the paper, code, and project page.
- Adding **Installation** and **Sample Usage** sections based on the official GitHub repository to help users get started with local deployment and music generation.

Please review and merge if this looks good to you!

Files changed (1) hide show
  1. README.md +36 -18
README.md CHANGED
@@ -1,48 +1,67 @@
1
  ---
2
- license: cc-by-nc-4.0
3
  language:
4
  - zh
5
  - en
6
  - ja
7
  - ko
8
  - es
 
9
  pipeline_tag: text-to-audio
10
  tags:
11
  - music
12
  - art
13
  ---
14
 
15
- # HeartMuLa
16
 
 
 
 
 
 
 
17
 
18
  ## Model Details
19
 
20
- ### Model Description
 
 
 
 
21
 
22
- <!-- Provide a longer summary of what this model is. -->
23
 
24
- - **Developed by:** [Ario Scale Global]
25
- - **License:** [CC-BY-NC 4.0]
26
 
27
- ### Model Sources
 
 
 
 
28
 
29
- <!-- Provide the basic links for the model. -->
30
 
31
- - **Github Repo:** https://github.com/HeartMuLa/heartlib
32
- - **Paper:** https://arxiv.org/abs/2601.10547
33
- - **Demo:** https://heartmula.github.io/
34
- - **HeartMuLa-oss-3B:** https://huggingface.co/HeartMuLa/HeartMuLa-oss-3B
35
- - **HeartCodec-oss:** https://huggingface.co/HeartMuLa/HeartCodec-oss
36
- - **HeartTranscriptor-oss:** https://huggingface.co/HeartMuLa/HeartTranscriptor-oss
37
 
 
 
38
 
39
- ## Get Started
 
 
 
 
40
 
41
- Check our github repo https://github.com/HeartMuLa/heartlib for a quickstart and local deployment of HeartMuLa.
 
 
 
 
 
42
 
43
  ## Citation
44
 
45
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
46
  If you find HeartMuLa useful, please cite:
47
 
48
  ```bibtex
@@ -57,6 +76,5 @@ If you find HeartMuLa useful, please cite:
57
  }
58
  ```
59
 
60
-
61
  ## Contact
62
  If you are interested in HeartMuLa, feel free to reach us at heartmula.ai@gmail.com
 
1
  ---
 
2
  language:
3
  - zh
4
  - en
5
  - ja
6
  - ko
7
  - es
8
+ license: cc-by-nc-4.0
9
  pipeline_tag: text-to-audio
10
  tags:
11
  - music
12
  - art
13
  ---
14
 
15
+ # HeartMuLa: A Family of Open Sourced Music Foundation Models
16
 
17
+ HeartMuLa is a family of open-source music foundation models designed to advance large-scale music understanding and generation across diverse tasks and modalities. It is an LLM-based song generation model capable of synthesizing high-fidelity music under rich, user-controllable conditions (e.g., textual style descriptions, lyrics, and reference audio).
18
+
19
+ - **Project Page:** [https://heartmula.github.io/](https://heartmula.github.io/)
20
+ - **Repository:** [https://github.com/HeartMuLa/heartlib](https://github.com/HeartMuLa/heartlib)
21
+ - **Paper:** [HeartMuLa: A Family of Open Sourced Music Foundation Models](https://arxiv.org/abs/2601.10547)
22
+ - **Demo:** [https://heartmula.github.io/](https://heartmula.github.io/)
23
 
24
  ## Model Details
25
 
26
+ The HeartMuLa framework consists of four major components:
27
+ 1. **HeartMuLa**: A music language model that generates music conditioned on lyrics and tags with multilingual support.
28
+ 2. **HeartCodec**: A low-frame-rate (12.5 Hz), high-fidelity music codec tokenizer that captures long-range musical structure.
29
+ 3. **HeartTranscriptor**: A robust lyric recognition model optimized for real-world music scenarios.
30
+ 4. **HeartCLAP**: An audio-text alignment model for music descriptions and cross-modal retrieval.
31
 
32
+ ## Installation
33
 
34
+ We recommend using `python=3.10` for local deployment. Clone the repository and install locally:
 
35
 
36
+ ```bash
37
+ git clone https://github.com/HeartMuLa/heartlib.git
38
+ cd heartlib
39
+ pip install -e .
40
+ ```
41
 
42
+ ## Sample Usage
43
 
44
+ ### Download Checkpoints
45
+ First, download the pretrained checkpoints into a `./ckpt` folder as described in the [GitHub README](https://github.com/HeartMuLa/heartlib).
 
 
 
 
46
 
47
+ ### Inference
48
+ To generate music conditioned on lyrics and tags, run the following command:
49
 
50
+ ```bash
51
+ python ./examples/run_music_generation.py --model_path=./ckpt --version="3B"
52
+ ```
53
+
54
+ By default, this command generates a piece of music based on the lyrics and tags provided in the `./assets` folder. The output will be saved as `./assets/output.mp3`.
55
 
56
+ **Key Parameters:**
57
+ - `--model_path`: Path to the pretrained model checkpoint.
58
+ - `--lyrics`: Path to the lyrics file (e.g., `./assets/lyrics.txt`).
59
+ - `--tags`: Path to the tags file (e.g., `./assets/tags.txt`).
60
+ - `--save_path`: Output audio file path.
61
+ - `--version`: The version of HeartMuLa (choose `3B`).
62
 
63
  ## Citation
64
 
 
65
  If you find HeartMuLa useful, please cite:
66
 
67
  ```bibtex
 
76
  }
77
  ```
78
 
 
79
  ## Contact
80
  If you are interested in HeartMuLa, feel free to reach us at heartmula.ai@gmail.com