happyme531/muscriptor-medium-onnx

MuScriptor Medium — ONNX Runtime FP16 and W4A16

This is an unofficial derivative ONNX conversion of MuScriptor/muscriptor-medium, developed by Mirelo × Kyutai. It contains an FP16 export and a smaller W4A16 export for ONNX Runtime. This repository is not affiliated with or endorsed by the MuScriptor authors.

The conversion uses upstream revision f32236969308476e01fd3aae67357de5feb05a2d. The source checkpoint used for the export has SHA-256 ac80adbdf85d87231735fd948af7013441c0afced316c4e9067fd5d8a7fb97ec.

Variants

Directory	Transformer backbone	Embeddings	LM head	Activations	Weight files
onnx/fp16	FP16	FP16	FP16	FP16	613,150,680 bytes
onnx/w4a16	HQQ UINT4	FP16	FP16	FP16	169,649,866 bytes

The W4A16 graph quantizes only the 96 constant-weight MatMul operations in the 24-layer Transformer backbone. It uses ONNX Runtime MatMulNBits, HQQ quantization, and block size 128. The token embedding, all other embedding tables, conditioner, and LM head deliberately remain FP16.

decoder.onnx.data is external tensor data and must remain next to onnx/w4a16/decoder.onnx.

ONNX interface

The conditioner consumes a 5-second, 16 kHz mono log-mel spectrogram plus instrument and dataset IDs. The decoder exposes an autoregressive interface with explicit past_key/past_value inputs and present_key/present_value outputs. Log-mel extraction is intentionally outside the ONNX graph.

Quick smoke test

Download this public repository:

hf download happyme531/muscriptor-medium-onnx --local-dir muscriptor-medium-onnx
cd muscriptor-medium-onnx
uv venv
source .venv/bin/activate
uv pip install -r runtime/requirements.txt
python runtime/test_onnxruntime.py --model-dir onnx/w4a16 --provider cpu

Use --audio path/to/audio.wav to replace the generated 440 Hz test tone. For CUDA, install a matching onnxruntime-gpu package instead of onnxruntime, and pass --provider cuda. PyTorch is optional; if installed, the helper imports it first so its CUDA/cuDNN libraries can be preloaded.

The included script validates conditioner/decoder execution and greedy token generation. It is a low-level ONNX Runtime smoke test, not a complete audio-to-MIDI application.

Validation

Both models pass onnx.checker.check_model. The FP16 and W4A16 exports were executed with ONNX Runtime on CUDA and x86-64 CPU; on a real audio sample their first eight greedy tokens matched. ARM64 execution was not validated and depends on whether the target ONNX Runtime build/provider implements the MatMulNBits graph used by the W4A16 decoder.

File	SHA-256
onnx/fp16/conditioner.onnx	cc94f58ba8023f339b4b8eba06a3f3e7623079c8234fa1be075612ee96efcf81
onnx/fp16/decoder.onnx	6832149920078feb4562c1c211a0dcac52a4ea10d7b715459e2c6fd0aee83b25
onnx/w4a16/conditioner.onnx	cc94f58ba8023f339b4b8eba06a3f3e7623079c8234fa1be075612ee96efcf81
onnx/w4a16/decoder.onnx	2d3b8a35c353b581982a1567eea1629ce70d4b899de1d020f4fa9cb85757943e
onnx/w4a16/decoder.onnx.data	8d92ff62bed61814593222a9fd8fcb731f0aa83cbd56591c56cc845b6e16b61d

License, attribution, and conditions

The model weights and these derivative ONNX weights are distributed under Creative Commons Attribution-NonCommercial 4.0 International. Commercial use is not permitted under that license. You must also comply with the upstream supplemental conditions summarized below, including having all necessary rights to any audio you transcribe and complying with all applicable laws.

Summary of upstream supplemental conditions of use: MuScriptor and any generated content by MuScriptor are provided as is without any warranty of any kind, including but not limited to any warranty of non-infringement. Use of MuScriptor and its output must comply with all applicable laws and must not result in, involve, or facilitate any illegal or unauthorized activity. Prohibited uses include, without limitation, inputting music files and transcribing them to MIDI/music sheet without having all the necessary rights, including intellectual property rights, under applicable laws. Accordingly, users undertake and warrant that they have all necessary rights, including intellectual property rights, in connection with their use of MuScriptor and its output. Mirelo and Kyutai disclaim liability for non-compliant use; users shall indemnify, defend, and hold them harmless from claims, damages, losses, liabilities, and expenses arising from failure to comply with CC BY-NC 4.0 and/or these specific conditions. The exact original wording is preserved in UPSTREAM_MODEL_CARD.md.

See NOTICE.md for attribution and UPSTREAM_MODEL_CARD.md for the original model card, intended uses, limitations, supplemental conditions, author list, and citation. The small runtime helpers are covered by their MIT license.