MuQ-MuLan-large-GGUF

GGUF conversions of OpenMuQ/MuQ-MuLan-large, a joint music/text embedding model supporting English and Chinese text. Native inference uses gorse-io/clap.cpp, not llama.cpp. Both text and audio outputs have 512 components. The tokenizer, frontend buffers and inference weights are included in each GGUF; no separate backbone download or Python installation is needed at inference.

Files

File	Bytes	Storage
muq-mulan-f16.gguf	1,302,180,544	Multidimensional parameters in IEEE FP16; frontend buffers and 1D parameters remain FP32
muq-mulan-f32.gguf	2,592,676,032	FP32
muq-mulan-q8_0.gguf	837,017,824	Canonical Q8_0 eligible dense/embedding tensors; ineligible tensors unchanged
muq-mulan-q4_k_m.gguf	586,099,520	Encoder Q4_K_M: Q4_K plus sensitive Q6_K; ineligible tensors unchanged

F16/F32 are preserved floating-point conversions. Q8_0 and Q4_K_M additions use the separately identified runtime below. F16 reduces download/storage size, but the MuQ backbone expands noneligible FP16 weights to FP32 compute weights; it does not halve runtime RAM requirements.

Native usage

Build the quantization-compatible runtime identified below. Historical float conversion/validation revisions remain recorded under provenance and do not identify the new quantization implementation.

Download a GGUF variant from this repository. Assuming it is in the current directory:

./build/clap-model --model muq-mulan-f16.gguf --threads 2 \
  --text 'classical genres, hopeful mood, piano.'
./build/clap-model --model muq-mulan-f16.gguf --threads 2 \
  --text '一首适合海边风景的小提琴曲，节奏欢快'
./build/clap-model --model muq-mulan-f16.gguf --threads 2 --audio music.flac

Each command prints a JSON array. Use clap-model; the validated revision's clap embedding and HTTP server are dummy interfaces, not real inference endpoints. Windows multi-configuration builds place the executable under build/Release/.

Audio is mixed to mono and resampled to 24 kHz. Successive ten-second clips follow the official prefix-tail behavior, and normalized clip embeddings are averaged without a final normalization. Multi-clip audio vectors can therefore have norm below one. Text is normalized. The official cross-modal similarity is the dot product. Native resampling is not bit-identical to librosa/torchaudio; use identical pre-resampled 24 kHz PCM when comparing implementations. The published text context is at most 512 tokenizer IDs including special tokens; oversized input is rejected. This is a single-item CPU inference path, not batched or end-to-end GPU inference.

Provenance and reproducibility

• Original weights/config: OpenMuQ/MuQ-MuLan-large at 2e01c796b71dca71b45251384c04cd7b237c9020.
• Original pytorch_model.bin SHA-256: d42ae3f7cb9b66759ee0089ddc70e2f28b130c2d8ba621457358272d32dd0444.
• Official MuQ implementation: tencent-ailab/MuQ at 28847ea50cd31ac4b8b6a7dacc051ad7d1c7606a.
• Converter and validated runtime: gorse-io/clap.cpp at ebbb9a46840a94fc5f94fed6cd594256e4492e5b.
• XLM-R config/tokenizer revision: e73636d4f797dec63c3081bb6ed5c7b0bb3f2089.

Conversion uses local configs/tokenizer and torch.load(weights_only=True, mmap=True) without executing checkpoint source. To reproduce, provide the pinned source checkpoint, MuQ dependency config and XLM-R config/tokenizer:

python -m pip install -r requirements.txt
python convert_hf_to_gguf.py /path/to/muq-mulan muq-mulan-f16.gguf \
  --dtype f16 \
  --muq-mulan-revision 2e01c796b71dca71b45251384c04cd7b237c9020 \
  --muq-config /path/to/muq/config.json \
  --text-config /path/to/xlm-roberta-base/config.json \
  --tokenizer /path/to/xlm-roberta-base/tokenizer.json
# Replace --dtype f16 with --dtype f32 and change the destination for F32.

Validation and limitations

Published-checkpoint numerical checks used five text probes (English, Chinese, empty, combining Unicode, special token) and identical 24 kHz synthetic broadband PCM spanning two clips:

Storage	Worst text absolute error	Audio absolute error
F32	3.50177e-7	2.45869e-7
F16	3.58094e-7	2.11410e-7

The F16 oracle uses identically storage-rounded weights with FP32 activations, not the untouched F32 checkpoint. These values isolate runtime parity; they are not a measurement of F16 degradation against the original model.

A separate short two-tone diagnostic measured audio errors of 1.84137e-4 (F32) and 1.89982e-4 (F16). FFT/noise-floor and resampling differences mean the broadband error is not a universal waveform bound. These synthetic probes do not assess music retrieval accuracy.

Both historical floating-point files were subsequently rechecked with native Chinese-text and audio inference: all four outputs contained exactly 512 finite components. Multi-clip audio norms were 0.979717080 (F32) and 0.979694188 (F16), as expected from clip averaging. Details are in validation.json; source hashes are in provenance.json.

Verify downloads with:

sha256sum -c SHA256SUMS

Quantization runtime and measured loss

The compatible public runtime is gorse-io/clap.cpp commit b59e0cc69dfbeb03840f41e45ca5efd8edd5a18d (implementation tree 22f24b2d4caa80481604dfd980a0f912a88be25c). This is an immutable public feature-branch commit, not a claim that the pull request is merged. The artifacts and recorded native validation were produced from this exact implementation tree before the public commit existed; the later commit has the same tree. Generation and validation binary SHA-256 values remain in quantization-provenance.json. Historical float-only runtime pins are not sufficient for these quantized files.

Build on Linux with a C++17 compiler, CMake, and ICU development libraries installed (for example, libicu-dev on Debian/Ubuntu):

git clone https://github.com/gorse-io/clap.cpp.git
cd clap.cpp
git checkout b59e0cc69dfbeb03840f41e45ca5efd8edd5a18d
git submodule update --init --recursive
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DCLAP_USE_SYSTEM_ICU=ON
cmake --build build --target clap-model clap-cli -j2

No historical runtime patch needs applying to this commit; the required MERT2 FFT/F32-activation corrections are included. Download a quantized GGUF from this model repository and use build/clap-model as below. To reproduce quantization from the preserved F32 file, use ./build/clap quantize INPUT_F32.gguf OUTPUT.gguf --type q8_0 or --type q4_k_m (choose a new output path).

Q4_K_M is the encoder preset q4_k_m-v2-encoder-dense-v-ffn-down-projection-q6_k, not uniform Q4_K and not llama.cpp's decoder-layer schedule. Eligible block-aligned native dense/embedding matrices use Q4_K, with all architecture-selected value/FFN-down/final projection roles protected by Q6_K. Convolutions, normalization, frontend/GRN tensors, noneligible roles and unaligned row widths retain identical source floating-point bytes. The full per-tensor type/role/raw-byte audit is recorded in provenance. Canonical padded tensor shapes are preserved; the writer can omit trailing singleton dimensions in the serialized rank (notably 24 unchanged MERT2 GRN vectors originally stored as [channels,1,1]). Every such change is recorded; float types, element counts and raw bytes remain identical. Q8_0 uses canonical Q8_0 on eligible 32-aligned rows. Quantization started from the verified preserved F32 file, not F16.

Weights remain mapped; each current dense operand is decoded into operation-local FP32 scratch with FP32 activation semantics. This is not whole-model dequantization on load, nor a claim of proportional RAM savings or faster inference. Noneligible FP16 promotion behavior remains relevant when loading F16 sources. The new artifacts were tested serially using two CPU threads with real full-checkpoint native inference on the saved waveform/text fixtures. Audio-only models have no text support.

Native commands (after building the approved runtime):

./build/clap-model --model QUANTIZED_FILE --audio music.flac --threads 2
./build/clap-model --model QUANTIZED_FILE --text '一首适合海边风景的小提琴曲，节奏欢快' --threads 2

The following are losses against the same quantization-compatible runtime loading the preserved F32 GGUF, not strict storage-rounded oracle parity and not official-model/retrieval-quality acceptance. Numerical equality is not expected. Frame-sequence metrics compare the flattened sequence, not a minimum over individual frames. Historical F16/F32 parity evidence and known narrowband/near-noise-floor limitations remain historical; no universal waveform error bound or music quality claim is made.

Preset	Cases	Minimum cosine	Maximum relative L2	Maximum absolute error
Q8_0	6	0.999103319	0.0423480953	0.00608719699
Q4_K_M	6	0.955176485	0.299411136	0.0412917696

All output dimensions, flattened frame/value counts and finite-value checks passed. Norm ranges, fixture hashes, exact commands and per-case losses are in quantization-validation.json; model identity, licensing, source checkpoints, tensor audits, output sizes and hashes are in quantization-provenance.json. F16/F32 files and their original provenance/validation are not replaced.

License and attribution

The original model weights and these converted weights are licensed CC-BY-NC-4.0 (non-commercial). Conversion does not change or remove that restriction. The upstream MuQ implementation is MIT; clap.cpp has its own code license, which does not relicense the weights. This repository is a format conversion by Gorse, not an official OpenMuQ release.

Credit: Haina Zhu, Yizhi Zhou, Hangting Chen, Jianwei Yu, Ziyang Ma, Rongzhi Gu, Yi Luo, Wei Tan and Xie Chen, MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization (2025), arXiv:2501.01108. See the original model card and official source.