PGSCOM/MuQ-MuLan-large-onnx

MuQ-MuLan-large (ONNX)

ONNX conversion of OpenMuQ/MuQ-MuLan-large (Tencent AI Lab), a music-text joint embedding model. The original weights are licensed CC BY-NC 4.0, so this conversion is too (non-commercial use, attribution to the MuQ authors). Changes from the original: exported to ONNX (opset 18/21) and quantized; no retraining.

Used by github.com/PGSCOM/music-similarity (export_web.py) and the Space PGSCOM/MusicSearcher.

File	Input	Output
mel.onnx (fp32)	wav float32 [1, 240000] (10 s mono at 24 kHz)	mel [1, 128, 1000] normalized mel spectrogram
audio_q4f16.onnx / audio_q4.onnx	mel float32 [1, 128, 1000]	embedding [1, 512], L2-normalized
texto_fp16.onnx / texto_q4.onnx	input_ids, attention_mask int64 [1, n] (tokenizer of the original model)	embedding [1, 512], L2-normalized

Song embedding: split the first 30 s into three 10 s windows, run mel + audio on each and average (same as MuQMuLan in PyTorch). Similarity = dot product between audio and/or text vectors.

Citation

@article{zhu2025muq,
  title={MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization},
  author={Zhu, Haina and Zhou, Yizhi and Chen, Hangting and Yu, Jianwei and Ma, Ziyang and Gu, Rongzhi and Luo, Yi and Tan, Wei and Chen, Xie},
  journal={arXiv preprint arXiv:2501.01108},
  year={2025}
}