| --- |
| license: cc-by-nc-4.0 |
| library_name: onnxruntime |
| base_model: MuScriptor/muscriptor-medium |
| tags: |
| - muscriptor |
| - music-transcription |
| - automatic-music-transcription |
| - audio-to-midi |
| - onnx |
| - onnxruntime |
| - int4 |
| - hqq |
| --- |
| |
| # MuScriptor Medium — ONNX Runtime FP16 and W4A16 |
|
|
| This is an **unofficial derivative ONNX conversion** of |
| [`MuScriptor/muscriptor-medium`](https://huggingface.co/MuScriptor/muscriptor-medium), |
| developed by Mirelo × Kyutai. It contains an FP16 export and a smaller W4A16 |
| export for ONNX Runtime. This repository is not affiliated with or endorsed by |
| the MuScriptor authors. |
|
|
| The conversion uses upstream revision |
| `f32236969308476e01fd3aae67357de5feb05a2d`. The source checkpoint used for the |
| export has SHA-256 |
| `ac80adbdf85d87231735fd948af7013441c0afced316c4e9067fd5d8a7fb97ec`. |
|
|
| ## Variants |
|
|
| | Directory | Transformer backbone | Embeddings | LM head | Activations | Weight files | |
| |---|---:|---:|---:|---:|---:| |
| | `onnx/fp16` | FP16 | FP16 | FP16 | FP16 | 613,150,680 bytes | |
| | `onnx/w4a16` | HQQ UINT4 | FP16 | FP16 | FP16 | 169,649,866 bytes | |
|
|
| The W4A16 graph quantizes only the 96 constant-weight MatMul operations in the |
| 24-layer Transformer backbone. It uses ONNX Runtime `MatMulNBits`, HQQ |
| quantization, and block size 128. The token embedding, all other embedding |
| tables, conditioner, and LM head deliberately remain FP16. |
|
|
| `decoder.onnx.data` is external tensor data and **must remain next to** |
| `onnx/w4a16/decoder.onnx`. |
|
|
| ## ONNX interface |
|
|
| The conditioner consumes a 5-second, 16 kHz mono log-mel spectrogram plus |
| instrument and dataset IDs. The decoder exposes an autoregressive interface |
| with explicit `past_key`/`past_value` inputs and `present_key`/`present_value` |
| outputs. Log-mel extraction is intentionally outside the ONNX graph. |
|
|
| ## Quick smoke test |
|
|
| Download this public repository: |
|
|
| ```bash |
| hf download happyme531/muscriptor-medium-onnx --local-dir muscriptor-medium-onnx |
| cd muscriptor-medium-onnx |
| uv venv |
| source .venv/bin/activate |
| uv pip install -r runtime/requirements.txt |
| python runtime/test_onnxruntime.py --model-dir onnx/w4a16 --provider cpu |
| ``` |
|
|
| Use `--audio path/to/audio.wav` to replace the generated 440 Hz test tone. For |
| CUDA, install a matching `onnxruntime-gpu` package instead of `onnxruntime`, |
| and pass `--provider cuda`. PyTorch is optional; if installed, the helper |
| imports it first so its CUDA/cuDNN libraries can be preloaded. |
|
|
| The included script validates conditioner/decoder execution and greedy token |
| generation. It is a low-level ONNX Runtime smoke test, not a complete |
| audio-to-MIDI application. |
|
|
| ## Validation |
|
|
| Both models pass `onnx.checker.check_model`. The FP16 and W4A16 exports were |
| executed with ONNX Runtime on CUDA and x86-64 CPU; on a real audio sample their |
| first eight greedy tokens matched. ARM64 execution was not validated and |
| depends on whether the target ONNX Runtime build/provider implements the |
| `MatMulNBits` graph used by the W4A16 decoder. |
|
|
| | File | SHA-256 | |
| |---|---| |
| | `onnx/fp16/conditioner.onnx` | `cc94f58ba8023f339b4b8eba06a3f3e7623079c8234fa1be075612ee96efcf81` | |
| | `onnx/fp16/decoder.onnx` | `6832149920078feb4562c1c211a0dcac52a4ea10d7b715459e2c6fd0aee83b25` | |
| | `onnx/w4a16/conditioner.onnx` | `cc94f58ba8023f339b4b8eba06a3f3e7623079c8234fa1be075612ee96efcf81` | |
| | `onnx/w4a16/decoder.onnx` | `2d3b8a35c353b581982a1567eea1629ce70d4b899de1d020f4fa9cb85757943e` | |
| | `onnx/w4a16/decoder.onnx.data` | `8d92ff62bed61814593222a9fd8fcb731f0aa83cbd56591c56cc845b6e16b61d` | |
|
|
| ## License, attribution, and conditions |
|
|
| The model weights and these derivative ONNX weights are distributed under |
| [Creative Commons Attribution-NonCommercial 4.0 International](LICENSE). |
| Commercial use is not permitted under that license. You must also comply with |
| the upstream supplemental conditions summarized below, including |
| having all necessary rights to any audio you transcribe and complying with all |
| applicable laws. |
|
|
| > **Summary of upstream supplemental conditions of use:** MuScriptor and any generated |
| > content by MuScriptor are provided as is without any warranty of any kind, |
| > including but not limited to any warranty of non-infringement. Use of |
| > MuScriptor and its output must comply with all applicable laws and must not |
| > result in, involve, or facilitate any illegal or unauthorized activity. |
| > Prohibited uses include, without limitation, inputting music files and |
| > transcribing them to MIDI/music sheet without having all the necessary |
| > rights, including intellectual property rights, under applicable laws. |
| > Accordingly, users undertake and warrant that they have all necessary rights, |
| > including intellectual property rights, in connection with their use of |
| > MuScriptor and its output. Mirelo and Kyutai disclaim liability for |
| > non-compliant use; users shall indemnify, defend, and hold them harmless from |
| > claims, damages, losses, liabilities, and expenses arising from failure to |
| > comply with CC BY-NC 4.0 and/or these specific conditions. The exact original |
| > wording is preserved in `UPSTREAM_MODEL_CARD.md`. |
|
|
| See [NOTICE.md](NOTICE.md) for attribution and |
| [UPSTREAM_MODEL_CARD.md](UPSTREAM_MODEL_CARD.md) for the original model card, |
| intended uses, limitations, supplemental conditions, author list, and citation. |
| The small runtime helpers are covered by [their MIT license](runtime/LICENSE). |
|
|
