Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,92 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-sa-4.0
|
| 3 |
+
datasets:
|
| 4 |
+
- SyMuPe/PERiScoPe
|
| 5 |
+
tags:
|
| 6 |
+
- music
|
| 7 |
+
- piano
|
| 8 |
+
- midi
|
| 9 |
+
- expressive-performance
|
| 10 |
+
- transformer
|
| 11 |
+
- MLM
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# SyMuPe: MLM baseline
|
| 15 |
+
|
| 16 |
+
**MLM-base** is a Transformer-based masked language modeling baseline for expressive piano performance rendering.
|
| 17 |
+
|
| 18 |
+
Introduced in the paper: [**SyMuPe: Affective and Controllable Symbolic Music Performance**](https://arxiv.org/abs/2511.03425).
|
| 19 |
+
|
| 20 |
+
- **GitHub:** https://github.com/ilya16/SyMuPe
|
| 21 |
+
- **Website:** https://ilya16.github.io/SyMuPe
|
| 22 |
+
- **Dataset:** https://huggingface.co/datasets/SyMuPe/PERiScoPe
|
| 23 |
+
|
| 24 |
+
## Architecture
|
| 25 |
+
|
| 26 |
+
- **Type:** Transformer Encoder
|
| 27 |
+
- **Objective:** Masked Performance Modeling (MLM)
|
| 28 |
+
- **Inputs:**
|
| 29 |
+
- **Score features (y):** `Pitch`, `Position`, `PositionShift`, `Duration`
|
| 30 |
+
- **Performance features (x):** `Velocity`, `TimeShift`, `TimeDuration`, `TimeDurationSustain`
|
| 31 |
+
- **Conditioning (c_s):** `Velocity` and `Tempo` score tokens for tempo and dynamics.
|
| 32 |
+
- **Outputs:** Categorical distributions for unmasked performance tokens.
|
| 33 |
+
- **Training:** Trained for 300,000 iterations on the [PERiScoPe v1.0](https://huggingface.co/datasets/SyMuPe/PERiScoPe) dataset as described in the paper.
|
| 34 |
+
|
| 35 |
+
## Quick Start
|
| 36 |
+
|
| 37 |
+
To use this model, ensure you have the `symupe` library installed (refer to the [GitHub repo](https://github.com/ilya16/SyMuPe) for installation instructions).
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
import torch
|
| 41 |
+
from symusic import Score
|
| 42 |
+
|
| 43 |
+
from symupe.data.tokenizers import SyMuPe
|
| 44 |
+
from symupe.inference import AutoGenerator, perform_score, save_performances
|
| 45 |
+
from symupe.models import AutoModel
|
| 46 |
+
|
| 47 |
+
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 48 |
+
|
| 49 |
+
# Load the model and tokenizer directly from the Hub
|
| 50 |
+
model = AutoModel.from_pretrained("SyMuPe/MLM-base").to(device)
|
| 51 |
+
tokenizer = SyMuPe.from_pretrained("SyMuPe/MLM-base")
|
| 52 |
+
|
| 53 |
+
# Prepare generator for the model
|
| 54 |
+
generator = AutoGenerator.from_model(model, tokenizer, device=device)
|
| 55 |
+
|
| 56 |
+
# Load score MIDI
|
| 57 |
+
score_midi = Score("score.mid")
|
| 58 |
+
|
| 59 |
+
# Perform score MIDI (tokenization is handled inside)
|
| 60 |
+
gen_results = perform_score(
|
| 61 |
+
generator=generator,
|
| 62 |
+
score=score_midi,
|
| 63 |
+
use_score_context=True,
|
| 64 |
+
num_samples=8,
|
| 65 |
+
seed=23
|
| 66 |
+
)
|
| 67 |
+
# gen_results[i] is PerformanceRenderingResult(...) containing:
|
| 68 |
+
# - score_midi, score_seq, gen_seq, perf_seq, perf_midi, perf_midi_sus
|
| 69 |
+
|
| 70 |
+
# Save performed MIDI files in a single directory
|
| 71 |
+
save_performances(gen_results, out_dir="samples/mlm", save_midi=True)
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
## License
|
| 75 |
+
|
| 76 |
+
The model weights are distributed under the [CC-BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en) license.
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
## Citation
|
| 80 |
+
|
| 81 |
+
If you use the dataset, please cite the paper:
|
| 82 |
+
|
| 83 |
+
```bibtex
|
| 84 |
+
@inproceedings{borovik2025symupe,
|
| 85 |
+
title = {{SyMuPe: Affective and Controllable Symbolic Music Performance}},
|
| 86 |
+
author = {Borovik, Ilya and Gavrilev, Dmitrii and Viro, Vladimir},
|
| 87 |
+
year = {2025},
|
| 88 |
+
booktitle = {Proceedings of the 33rd ACM International Conference on Multimedia},
|
| 89 |
+
pages = {10699--10708},
|
| 90 |
+
doi = {10.1145/3746027.3755871}
|
| 91 |
+
}
|
| 92 |
+
```
|