unifolia commited on
Commit
15dd43d
Β·
verified Β·
1 Parent(s): 964f4c6

Upload folder using huggingface_hub

Browse files
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ base_model: MuScriptor/muscriptor-small
4
+ tags:
5
+ - music
6
+ - transcription
7
+ - audio-to-midi
8
+ - onnx
9
+ library_name: onnx
10
+ ---
11
+
12
+ # MuScriptor small β€” ONNX conversion
13
+
14
+ An ONNX Runtime conversion of [`MuScriptor/muscriptor-small`](https://huggingface.co/MuScriptor/muscriptor-small),
15
+ the 103M-parameter multi-instrument music transcription model by **Kyutai Γ—
16
+ Mirelo Γ— IRCAM**. The weights are theirs and unmodified apart from the
17
+ conversion; all credit for the model belongs to them.
18
+
19
+ - Original model and paper: <https://github.com/muscriptor/muscriptor>, [arXiv:2607.08168](https://arxiv.org/abs/2607.08168)
20
+ - **Licence: CC BY-NC 4.0** β€” non-commercial use only, inherited from the
21
+ original weights.
22
+
23
+ It exists so that [Lepton Hum](https://midi.engineering/lepton-hum/), a
24
+ desktop practice tool for musicians, can run the model through ONNX Runtime
25
+ without shipping PyTorch. Anyone else running ONNX Runtime can use it the
26
+ same way, under the same non-commercial terms.
27
+
28
+ ## Files
29
+
30
+ | file | what it is |
31
+ |---|---|
32
+ | `muscriptor-small-cond.onnx` | conditioner: audio β†’ prefix embeddings |
33
+ | `muscriptor-small-lm.onnx` | the decoder-only transformer, KV cache in and out |
34
+
35
+ ### `muscriptor-small-cond.onnx`
36
+
37
+ | | |
38
+ |---|---|
39
+ | `audio` | `f32 [1, 80000]` β€” one 5-second chunk at 16 kHz, zero-padded |
40
+ | `instrument` | `i64 [1, K]` β€” instrument class ids (group + 2; `1` = unconditional) |
41
+ | `dataset` | `i64 [1, 1]` β€” always `1`, the null class |
42
+ | β†’ `prefix` | `f32 [1, 501 + 1 + K, 768]` |
43
+
44
+ The spectrogram is **inside this graph** on purpose. The checkpoint carries
45
+ its own STFT window and mel filterbank as buffers, and they are stored
46
+ quantized β€” about 2.4e-4 away from a freshly computed periodic Hann window
47
+ and htk filterbank. That rounding raises the window's sidelobe floor by
48
+ roughly 70 dB, and since the model reads log-magnitude features, bins that
49
+ the textbook formulas put at the `log(1e-6)` floor sit near βˆ’4 for this
50
+ model. Recomputing the front end therefore feeds it a spectrum it was not
51
+ trained on, so the conversion keeps the checkpoint's own.
52
+
53
+ ### `muscriptor-small-lm.onnx`
54
+
55
+ | | |
56
+ |---|---|
57
+ | `tokens` | `i64 [1, S]` |
58
+ | `prefix` | `f32 [1, P, 768]` β€” the conditioner's output; `P = 0` after the first step |
59
+ | `past_k`, `past_v` | `f32 [14, 1, T_past, 12, 64]` |
60
+ | β†’ `logits` | `f32 [1, 1393]` β€” last position only |
61
+ | β†’ `present_k`, `present_v` | the cache, grown by this step |
62
+
63
+ Positions are not an input: everything in the cache precedes the call, so
64
+ `T_past` is where the new tokens start. The graph is branch-free, so one
65
+ exported model serves both the prefill and single-token decoding.
66
+
67
+ Greedy decoding, MT3 token vocabulary (1393 tokens), 5-second chunks with a
68
+ tie prologue carrying notes across chunk boundaries β€” see the original
69
+ repository for the decoding rules.
70
+
71
+ ## Fidelity
72
+
73
+ Checked against the reference PyTorch implementation on the same 3.7-minute
74
+ mix, both given identical 16 kHz audio: **drums 1103 notes vs 1103 (exact),
75
+ bass 522 vs 514, guitar 1730 vs 1845**. Greedy decoding turns the ~1e-5
76
+ logit differences between ONNX Runtime and PyTorch into an occasional
77
+ flipped argmax that then cascades, so agreement within a few percent per
78
+ instrument is what parity looks like here.
79
+
80
+ Layer-level checks from the conversion script: conditioner max|Ξ”| 7.7e-3
81
+ (the STFT's numerics), prefill logits 2.7e-5, three decode steps ≀ 5.4e-5.
82
+
83
+ ## Reproducing
84
+
85
+ `scripts/export_muscriptor_onnx.py` in the Lepton Hum repository. It fetches
86
+ the original gated weights (accept the licence on the model page first),
87
+ exports both graphs, and asserts parity against the reference modules before
88
+ writing anything.
muscriptor-small-cond.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bfdd8edb30affe9f085cf145400cbd3b320822946468a42d88664d2cd1d91d6b
3
+ size 6799538
muscriptor-small-lm.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88231c3343ecb11f8e3cba4dbf22835e7398122425025a5c00c60e4dfd2e1c66
3
+ size 405211928