README.md

---
license: cc-by-nc-4.0
base_model: MuScriptor/muscriptor-small
tags:
  - music
  - transcription
  - audio-to-midi
  - onnx
library_name: onnx
---

# MuScriptor small — ONNX conversion

An ONNX Runtime conversion of [`MuScriptor/muscriptor-small`](https://huggingface.co/MuScriptor/muscriptor-small),
the 103M-parameter multi-instrument music transcription model by **Kyutai ×
Mirelo × IRCAM**. The weights are theirs and unmodified apart from the
conversion; all credit for the model belongs to them.

- Original model and paper: <https://github.com/muscriptor/muscriptor>, [arXiv:2607.08168](https://arxiv.org/abs/2607.08168)
- **Licence: CC BY-NC 4.0** — non-commercial use only, inherited from the
  original weights.

It exists so that [Lepton Hum](https://midi.engineering/lepton-hum/), a
desktop practice tool for musicians, can run the model through ONNX Runtime
without shipping PyTorch. Anyone else running ONNX Runtime can use it the
same way, under the same non-commercial terms.

## Files

| file | what it is |
|---|---|
| `muscriptor-small-cond.onnx` | conditioner: audio → prefix embeddings |
| `muscriptor-small-lm.onnx` | the decoder-only transformer, KV cache in and out |

### `muscriptor-small-cond.onnx`

| | |
|---|---|
| `audio` | `f32 [1, 80000]` — one 5-second chunk at 16 kHz, zero-padded |
| `instrument` | `i64 [1, K]` — instrument class ids (group + 2; `1` = unconditional) |
| `dataset` | `i64 [1, 1]` — always `1`, the null class |
| → `prefix` | `f32 [1, 501 + 1 + K, 768]` |

The spectrogram is **inside this graph** on purpose. The checkpoint carries
its own STFT window and mel filterbank as buffers, and they are stored
quantized — about 2.4e-4 away from a freshly computed periodic Hann window
and htk filterbank. That rounding raises the window's sidelobe floor by
roughly 70 dB, and since the model reads log-magnitude features, bins that
the textbook formulas put at the `log(1e-6)` floor sit near −4 for this
model. Recomputing the front end therefore feeds it a spectrum it was not
trained on, so the conversion keeps the checkpoint's own.

### `muscriptor-small-lm.onnx`

| | |
|---|---|
| `tokens` | `i64 [1, S]` |
| `prefix` | `f32 [1, P, 768]` — the conditioner's output; `P = 0` after the first step |
| `past_k`, `past_v` | `f32 [14, 1, T_past, 12, 64]` |
| → `logits` | `f32 [1, 1393]` — last position only |
| → `present_k`, `present_v` | the cache, grown by this step |

Positions are not an input: everything in the cache precedes the call, so
`T_past` is where the new tokens start. The graph is branch-free, so one
exported model serves both the prefill and single-token decoding.

Greedy decoding, MT3 token vocabulary (1393 tokens), 5-second chunks with a
tie prologue carrying notes across chunk boundaries — see the original
repository for the decoding rules.

## Fidelity

Checked against the reference PyTorch implementation on the same 3.7-minute
mix, both given identical 16 kHz audio: **drums 1103 notes vs 1103 (exact),
bass 522 vs 514, guitar 1730 vs 1845**. Greedy decoding turns the ~1e-5
logit differences between ONNX Runtime and PyTorch into an occasional
flipped argmax that then cascades, so agreement within a few percent per
instrument is what parity looks like here.

Layer-level checks from the conversion script: conditioner max|Δ| 7.7e-3
(the STFT's numerics), prefill logits 2.7e-5, three decode steps ≤ 5.4e-5.

## Reproducing

`scripts/export_muscriptor_onnx.py` in the Lepton Hum repository. It fetches
the original gated weights (accept the licence on the model page first),
exports both graphs, and asserts parity against the reference modules before
writing anything.