File size: 4,306 Bytes
bedfc1d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
---
license: cc-by-nc-4.0
base_model:
  - MuScriptor/muscriptor-small
  - MuScriptor/muscriptor-medium
library_name: onnxruntime
tags:
  - music
  - music-transcription
  - automatic-music-transcription
  - audio-to-midi
  - onnx
  - onnxruntime
  - int8
---

# MuScriptor β€” ONNX (small & medium, INT8)

ONNX Runtime conversions of [MuScriptor Small](https://huggingface.co/MuScriptor/muscriptor-small) (β‰ˆ100M params)
and [MuScriptor Medium](https://huggingface.co/MuScriptor/muscriptor-medium) (β‰ˆ300M params), the multi-instrument
music transcription model by **Mirelo Γ— Kyutai**. Made for running the model on CPU through ONNX Runtime
(e.g. `onnxruntime-node`) without PyTorch. **Format conversion only β€” nothing retrained.** See [NOTICE.md](NOTICE.md)
for attribution, the list of changes and the upstream conditions of use.

> **License: CC BY-NC 4.0 β€” non-commercial use only**, plus the upstream conditions of use in [NOTICE.md](NOTICE.md).
> By using these files you accept them.

## Files

| File | Size | What |
|---|---:|---|
| `small/cond.onnx` | 23.6 MB | conditioner: audio β†’ prefix embeddings (log-mel inside the graph) |
| `small/lm-int8.onnx` | 108.5 MB | decoder, INT8 weights, KV cache in/out |
| `medium/cond.onnx` | 25.1 MB | conditioner |
| `medium/lm-int8.onnx` | 315.2 MB | decoder, INT8 weights |
| `SHA256SUMS` | | checksums of the four graphs |
| `tools/` | | export + quantization scripts that produced these files |

## Graph interface (both sizes)

`cond.onnx`

| | |
|---|---|
| `audio` | `f32 [1, 80000]` β€” one 5 s chunk at 16 kHz mono, zero-padded |
| `instrument` | `i64 [1, K]` β€” MT3_FULL_PLUS instrument group ids; `-1` = unconditional |
| `dataset` | `i64 [1, 1]` β€” always `-1` |
| β†’ `prefix` | `f32 [1, 501 + 1 + K, dim]` |

The mel spectrogram (reflect pad, STFT as a DFT convolution, HTK mel bank, `log(x + 1e-6)`) runs **inside** the graph
and uses the checkpoint's own window and filterbank buffers, which differ from the textbook formulas by about 2.4e-4.

`lm-int8.onnx`

| | |
|---|---|
| `tokens` | `i64 [1, S]` |
| `prefix` | `f32 [1, P, dim]` β€” conditioner output on the first call; `P = 0` afterwards |
| `past_k`, `past_v` | `f32 [L, 1, T_past, H, 64]` β€” empty (`T_past = 0`) on the first call |
| β†’ `logits` | `f32 [1, card]` β€” last position only |
| β†’ `present_k`, `present_v` | the cache grown by this call |

| | dim | heads (H) | layers (L) | card (vocab) | initial token |
|---|---:|---:|---:|---:|---:|
| small | 768 | 12 | 14 | 1393 | 1393 |
| medium | 1024 | 16 | 24 | 1395 | 1395 |

Positions continue from `T_past`; the same graph serves prefill and single-token decoding. Decoding is greedy over the
first 1393 logits, 5 s chunks, MT3 token vocabulary, with the tie-prologue handling described in the
[reference implementation](https://github.com/muscriptor/muscriptor). Prefill the first call with
`[initial_token]` (plus a forced tie prologue for chunks after the first).

## Fidelity

fp32 exports vs the PyTorch reference, same 5 s chunk: conditioner output max |Ξ”| 5.7e-5 (small) / 7.1e-5 (medium);
greedy token sequences identical (24/24 and 27/27 tokens).

INT8 decoders, teacher-forced top-1 agreement with the fp32 reference over the reference's greedy tokens
(5 short musical clips, 175 tokens for small and 216 for medium β€” a small sample, not a benchmark):

| | fp32 | INT8 (these files) |
|---|---:|---:|
| small | 100% | 96.6% |
| medium | 100% | 98.6% |

INT4 weight-only variants were also tried (small 96.6%, medium 94.0%) and are not published here.

## Reproducing

```bash
python tools/export_muscriptor_onnx.py --size small --checkpoint model.safetensors --reference <muscriptor repo> --out out/small
python tools/quantize_int8.py out/small/lm.onnx out/small/lm-int8.onnx --card 1393
```

The upstream weights are gated on Hugging Face: accept their conditions on the model pages first.

## Citation

```bibtex
@misc{rouard2026muscriptoropenmodelmultiinstrument,
  title={MuScriptor: An Open Model for Multi-Instrument Music Transcription},
  author={Simon Rouard and Michael Krause and Axel Roebel and Carl-Johann Simon-Gabriel and Alexandre D\'efossez},
  year={2026},
  eprint={2607.08168},
  archivePrefix={arXiv},
  primaryClass={cs.SD},
  url={https://arxiv.org/abs/2607.08168}
}
```