File size: 4,306 Bytes
bedfc1d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | ---
license: cc-by-nc-4.0
base_model:
- MuScriptor/muscriptor-small
- MuScriptor/muscriptor-medium
library_name: onnxruntime
tags:
- music
- music-transcription
- automatic-music-transcription
- audio-to-midi
- onnx
- onnxruntime
- int8
---
# MuScriptor β ONNX (small & medium, INT8)
ONNX Runtime conversions of [MuScriptor Small](https://huggingface.co/MuScriptor/muscriptor-small) (β100M params)
and [MuScriptor Medium](https://huggingface.co/MuScriptor/muscriptor-medium) (β300M params), the multi-instrument
music transcription model by **Mirelo Γ Kyutai**. Made for running the model on CPU through ONNX Runtime
(e.g. `onnxruntime-node`) without PyTorch. **Format conversion only β nothing retrained.** See [NOTICE.md](NOTICE.md)
for attribution, the list of changes and the upstream conditions of use.
> **License: CC BY-NC 4.0 β non-commercial use only**, plus the upstream conditions of use in [NOTICE.md](NOTICE.md).
> By using these files you accept them.
## Files
| File | Size | What |
|---|---:|---|
| `small/cond.onnx` | 23.6 MB | conditioner: audio β prefix embeddings (log-mel inside the graph) |
| `small/lm-int8.onnx` | 108.5 MB | decoder, INT8 weights, KV cache in/out |
| `medium/cond.onnx` | 25.1 MB | conditioner |
| `medium/lm-int8.onnx` | 315.2 MB | decoder, INT8 weights |
| `SHA256SUMS` | | checksums of the four graphs |
| `tools/` | | export + quantization scripts that produced these files |
## Graph interface (both sizes)
`cond.onnx`
| | |
|---|---|
| `audio` | `f32 [1, 80000]` β one 5 s chunk at 16 kHz mono, zero-padded |
| `instrument` | `i64 [1, K]` β MT3_FULL_PLUS instrument group ids; `-1` = unconditional |
| `dataset` | `i64 [1, 1]` β always `-1` |
| β `prefix` | `f32 [1, 501 + 1 + K, dim]` |
The mel spectrogram (reflect pad, STFT as a DFT convolution, HTK mel bank, `log(x + 1e-6)`) runs **inside** the graph
and uses the checkpoint's own window and filterbank buffers, which differ from the textbook formulas by about 2.4e-4.
`lm-int8.onnx`
| | |
|---|---|
| `tokens` | `i64 [1, S]` |
| `prefix` | `f32 [1, P, dim]` β conditioner output on the first call; `P = 0` afterwards |
| `past_k`, `past_v` | `f32 [L, 1, T_past, H, 64]` β empty (`T_past = 0`) on the first call |
| β `logits` | `f32 [1, card]` β last position only |
| β `present_k`, `present_v` | the cache grown by this call |
| | dim | heads (H) | layers (L) | card (vocab) | initial token |
|---|---:|---:|---:|---:|---:|
| small | 768 | 12 | 14 | 1393 | 1393 |
| medium | 1024 | 16 | 24 | 1395 | 1395 |
Positions continue from `T_past`; the same graph serves prefill and single-token decoding. Decoding is greedy over the
first 1393 logits, 5 s chunks, MT3 token vocabulary, with the tie-prologue handling described in the
[reference implementation](https://github.com/muscriptor/muscriptor). Prefill the first call with
`[initial_token]` (plus a forced tie prologue for chunks after the first).
## Fidelity
fp32 exports vs the PyTorch reference, same 5 s chunk: conditioner output max |Ξ| 5.7e-5 (small) / 7.1e-5 (medium);
greedy token sequences identical (24/24 and 27/27 tokens).
INT8 decoders, teacher-forced top-1 agreement with the fp32 reference over the reference's greedy tokens
(5 short musical clips, 175 tokens for small and 216 for medium β a small sample, not a benchmark):
| | fp32 | INT8 (these files) |
|---|---:|---:|
| small | 100% | 96.6% |
| medium | 100% | 98.6% |
INT4 weight-only variants were also tried (small 96.6%, medium 94.0%) and are not published here.
## Reproducing
```bash
python tools/export_muscriptor_onnx.py --size small --checkpoint model.safetensors --reference <muscriptor repo> --out out/small
python tools/quantize_int8.py out/small/lm.onnx out/small/lm-int8.onnx --card 1393
```
The upstream weights are gated on Hugging Face: accept their conditions on the model pages first.
## Citation
```bibtex
@misc{rouard2026muscriptoropenmodelmultiinstrument,
title={MuScriptor: An Open Model for Multi-Instrument Music Transcription},
author={Simon Rouard and Michael Krause and Axel Roebel and Carl-Johann Simon-Gabriel and Alexandre D\'efossez},
year={2026},
eprint={2607.08168},
archivePrefix={arXiv},
primaryClass={cs.SD},
url={https://arxiv.org/abs/2607.08168}
}
```
|