Upload folder using huggingface_hub
Browse files- README.md +88 -0
- muscriptor-small-cond.onnx +3 -0
- muscriptor-small-lm.onnx +3 -0
README.md
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
base_model: MuScriptor/muscriptor-small
|
| 4 |
+
tags:
|
| 5 |
+
- music
|
| 6 |
+
- transcription
|
| 7 |
+
- audio-to-midi
|
| 8 |
+
- onnx
|
| 9 |
+
library_name: onnx
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# MuScriptor small β ONNX conversion
|
| 13 |
+
|
| 14 |
+
An ONNX Runtime conversion of [`MuScriptor/muscriptor-small`](https://huggingface.co/MuScriptor/muscriptor-small),
|
| 15 |
+
the 103M-parameter multi-instrument music transcription model by **Kyutai Γ
|
| 16 |
+
Mirelo Γ IRCAM**. The weights are theirs and unmodified apart from the
|
| 17 |
+
conversion; all credit for the model belongs to them.
|
| 18 |
+
|
| 19 |
+
- Original model and paper: <https://github.com/muscriptor/muscriptor>, [arXiv:2607.08168](https://arxiv.org/abs/2607.08168)
|
| 20 |
+
- **Licence: CC BY-NC 4.0** β non-commercial use only, inherited from the
|
| 21 |
+
original weights.
|
| 22 |
+
|
| 23 |
+
It exists so that [Lepton Hum](https://midi.engineering/lepton-hum/), a
|
| 24 |
+
desktop practice tool for musicians, can run the model through ONNX Runtime
|
| 25 |
+
without shipping PyTorch. Anyone else running ONNX Runtime can use it the
|
| 26 |
+
same way, under the same non-commercial terms.
|
| 27 |
+
|
| 28 |
+
## Files
|
| 29 |
+
|
| 30 |
+
| file | what it is |
|
| 31 |
+
|---|---|
|
| 32 |
+
| `muscriptor-small-cond.onnx` | conditioner: audio β prefix embeddings |
|
| 33 |
+
| `muscriptor-small-lm.onnx` | the decoder-only transformer, KV cache in and out |
|
| 34 |
+
|
| 35 |
+
### `muscriptor-small-cond.onnx`
|
| 36 |
+
|
| 37 |
+
| | |
|
| 38 |
+
|---|---|
|
| 39 |
+
| `audio` | `f32 [1, 80000]` β one 5-second chunk at 16 kHz, zero-padded |
|
| 40 |
+
| `instrument` | `i64 [1, K]` β instrument class ids (group + 2; `1` = unconditional) |
|
| 41 |
+
| `dataset` | `i64 [1, 1]` β always `1`, the null class |
|
| 42 |
+
| β `prefix` | `f32 [1, 501 + 1 + K, 768]` |
|
| 43 |
+
|
| 44 |
+
The spectrogram is **inside this graph** on purpose. The checkpoint carries
|
| 45 |
+
its own STFT window and mel filterbank as buffers, and they are stored
|
| 46 |
+
quantized β about 2.4e-4 away from a freshly computed periodic Hann window
|
| 47 |
+
and htk filterbank. That rounding raises the window's sidelobe floor by
|
| 48 |
+
roughly 70 dB, and since the model reads log-magnitude features, bins that
|
| 49 |
+
the textbook formulas put at the `log(1e-6)` floor sit near β4 for this
|
| 50 |
+
model. Recomputing the front end therefore feeds it a spectrum it was not
|
| 51 |
+
trained on, so the conversion keeps the checkpoint's own.
|
| 52 |
+
|
| 53 |
+
### `muscriptor-small-lm.onnx`
|
| 54 |
+
|
| 55 |
+
| | |
|
| 56 |
+
|---|---|
|
| 57 |
+
| `tokens` | `i64 [1, S]` |
|
| 58 |
+
| `prefix` | `f32 [1, P, 768]` β the conditioner's output; `P = 0` after the first step |
|
| 59 |
+
| `past_k`, `past_v` | `f32 [14, 1, T_past, 12, 64]` |
|
| 60 |
+
| β `logits` | `f32 [1, 1393]` β last position only |
|
| 61 |
+
| β `present_k`, `present_v` | the cache, grown by this step |
|
| 62 |
+
|
| 63 |
+
Positions are not an input: everything in the cache precedes the call, so
|
| 64 |
+
`T_past` is where the new tokens start. The graph is branch-free, so one
|
| 65 |
+
exported model serves both the prefill and single-token decoding.
|
| 66 |
+
|
| 67 |
+
Greedy decoding, MT3 token vocabulary (1393 tokens), 5-second chunks with a
|
| 68 |
+
tie prologue carrying notes across chunk boundaries β see the original
|
| 69 |
+
repository for the decoding rules.
|
| 70 |
+
|
| 71 |
+
## Fidelity
|
| 72 |
+
|
| 73 |
+
Checked against the reference PyTorch implementation on the same 3.7-minute
|
| 74 |
+
mix, both given identical 16 kHz audio: **drums 1103 notes vs 1103 (exact),
|
| 75 |
+
bass 522 vs 514, guitar 1730 vs 1845**. Greedy decoding turns the ~1e-5
|
| 76 |
+
logit differences between ONNX Runtime and PyTorch into an occasional
|
| 77 |
+
flipped argmax that then cascades, so agreement within a few percent per
|
| 78 |
+
instrument is what parity looks like here.
|
| 79 |
+
|
| 80 |
+
Layer-level checks from the conversion script: conditioner max|Ξ| 7.7e-3
|
| 81 |
+
(the STFT's numerics), prefill logits 2.7e-5, three decode steps β€ 5.4e-5.
|
| 82 |
+
|
| 83 |
+
## Reproducing
|
| 84 |
+
|
| 85 |
+
`scripts/export_muscriptor_onnx.py` in the Lepton Hum repository. It fetches
|
| 86 |
+
the original gated weights (accept the licence on the model page first),
|
| 87 |
+
exports both graphs, and asserts parity against the reference modules before
|
| 88 |
+
writing anything.
|
muscriptor-small-cond.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bfdd8edb30affe9f085cf145400cbd3b320822946468a42d88664d2cd1d91d6b
|
| 3 |
+
size 6799538
|
muscriptor-small-lm.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:88231c3343ecb11f8e3cba4dbf22835e7398122425025a5c00c60e4dfd2e1c66
|
| 3 |
+
size 405211928
|