--- license: cc-by-nc-4.0 base_model: - MuScriptor/muscriptor-small - MuScriptor/muscriptor-medium library_name: onnxruntime tags: - music - music-transcription - automatic-music-transcription - audio-to-midi - onnx - onnxruntime - int8 --- # MuScriptor — ONNX (small & medium, INT8) ONNX Runtime conversions of [MuScriptor Small](https://huggingface.co/MuScriptor/muscriptor-small) (≈100M params) and [MuScriptor Medium](https://huggingface.co/MuScriptor/muscriptor-medium) (≈300M params), the multi-instrument music transcription model by **Mirelo × Kyutai**. Made for running the model on CPU through ONNX Runtime (e.g. `onnxruntime-node`) without PyTorch. **Format conversion only — nothing retrained.** See [NOTICE.md](NOTICE.md) for attribution, the list of changes and the upstream conditions of use. > **License: CC BY-NC 4.0 — non-commercial use only**, plus the upstream conditions of use in [NOTICE.md](NOTICE.md). > By using these files you accept them. ## Files | File | Size | What | |---|---:|---| | `small/cond.onnx` | 23.6 MB | conditioner: audio → prefix embeddings (log-mel inside the graph) | | `small/lm-int8.onnx` | 108.5 MB | decoder, INT8 weights, KV cache in/out | | `medium/cond.onnx` | 25.1 MB | conditioner | | `medium/lm-int8.onnx` | 315.2 MB | decoder, INT8 weights | | `SHA256SUMS` | | checksums of the four graphs | | `tools/` | | export + quantization scripts that produced these files | ## Graph interface (both sizes) `cond.onnx` | | | |---|---| | `audio` | `f32 [1, 80000]` — one 5 s chunk at 16 kHz mono, zero-padded | | `instrument` | `i64 [1, K]` — MT3_FULL_PLUS instrument group ids; `-1` = unconditional | | `dataset` | `i64 [1, 1]` — always `-1` | | → `prefix` | `f32 [1, 501 + 1 + K, dim]` | The mel spectrogram (reflect pad, STFT as a DFT convolution, HTK mel bank, `log(x + 1e-6)`) runs **inside** the graph and uses the checkpoint's own window and filterbank buffers, which differ from the textbook formulas by about 2.4e-4. `lm-int8.onnx` | | | |---|---| | `tokens` | `i64 [1, S]` | | `prefix` | `f32 [1, P, dim]` — conditioner output on the first call; `P = 0` afterwards | | `past_k`, `past_v` | `f32 [L, 1, T_past, H, 64]` — empty (`T_past = 0`) on the first call | | → `logits` | `f32 [1, card]` — last position only | | → `present_k`, `present_v` | the cache grown by this call | | | dim | heads (H) | layers (L) | card (vocab) | initial token | |---|---:|---:|---:|---:|---:| | small | 768 | 12 | 14 | 1393 | 1393 | | medium | 1024 | 16 | 24 | 1395 | 1395 | Positions continue from `T_past`; the same graph serves prefill and single-token decoding. Decoding is greedy over the first 1393 logits, 5 s chunks, MT3 token vocabulary, with the tie-prologue handling described in the [reference implementation](https://github.com/muscriptor/muscriptor). Prefill the first call with `[initial_token]` (plus a forced tie prologue for chunks after the first). ## Fidelity fp32 exports vs the PyTorch reference, same 5 s chunk: conditioner output max |Δ| 5.7e-5 (small) / 7.1e-5 (medium); greedy token sequences identical (24/24 and 27/27 tokens). INT8 decoders, teacher-forced top-1 agreement with the fp32 reference over the reference's greedy tokens (5 short musical clips, 175 tokens for small and 216 for medium — a small sample, not a benchmark): | | fp32 | INT8 (these files) | |---|---:|---:| | small | 100% | 96.6% | | medium | 100% | 98.6% | INT4 weight-only variants were also tried (small 96.6%, medium 94.0%) and are not published here. ## Reproducing ```bash python tools/export_muscriptor_onnx.py --size small --checkpoint model.safetensors --reference --out out/small python tools/quantize_int8.py out/small/lm.onnx out/small/lm-int8.onnx --card 1393 ``` The upstream weights are gated on Hugging Face: accept their conditions on the model pages first. ## Citation ```bibtex @misc{rouard2026muscriptoropenmodelmultiinstrument, title={MuScriptor: An Open Model for Multi-Instrument Music Transcription}, author={Simon Rouard and Michael Krause and Axel Roebel and Carl-Johann Simon-Gabriel and Alexandre D\'efossez}, year={2026}, eprint={2607.08168}, archivePrefix={arXiv}, primaryClass={cs.SD}, url={https://arxiv.org/abs/2607.08168} } ```