MuQ-large-msd-iter-GGUF

F16/F32 GGUF conversions and Q8_0/Q4_K_M derivatives of OpenMuQ/MuQ-large-msd-iter, for native inference with gorse-io/clap.cpp.

This is an audio-only representation model, not MuQ-MuLan. It produces 1024-dimensional final-layer audio features, not joint text/audio embeddings. Text input is unsupported. These GGUF files are not intended for llama.cpp.

Files

File	Bytes	Storage
muq-f16.gguf	634,043,104	IEEE FP16 multidimensional parameters, FP32 vectors and frontend buffers
muq-f32.gguf	1,266,066,144	FP32
muq-q8_0.gguf	477,537,088	Canonical Q8_0 eligible dense/embedding tensors; ineligible tensors unchanged
muq-q4_k_m.gguf	376,841,120	Encoder Q4_K_M: Q4_K plus sensitive Q6_K; ineligible tensors unchanged

F16 saves download/storage space, not runtime weight RAM: the validated native MuQ loader expands F16 weights into FP32 compute buffers. Q8_0 and Q4_K_M additions require the separately identified runtime below. Each file contains the required inference weights and frontend buffers; no separate model is needed at inference.

Native inference

Build the quantization-compatible runtime identified below. Historical float conversion/validation revisions remain recorded under provenance and do not identify the new quantization implementation.

Download a GGUF variant, then run (assuming the GGUF is in the current directory):

# Mean-pool the final frames into one unnormalized 1024-vector:
./build/clap-model --model muq-f16.gguf --audio music.flac --threads 2
# Return the complete time-major final-layer frame sequence:
./build/clap-model --model muq-f16.gguf --audio music.flac --threads 2 --frames

Use clap-model, not the historical dummy clap embedding or HTTP server interfaces. Windows multi-configuration builds place the executable in build/Release/.

The default output is a JSON array of 1024 finite values. --frames prints an object with frames, dimension and flat time-major values. There is no L2 normalization. The pooled output is a convenience over MuQ representations, not a trained text/audio projection.

Audio is mixed to mono and resampled to 24 kHz. The complete input is processed without an implicit crop or chunk aggregation. At least 1025 samples at 24 kHz are required. For N native-rate samples, frame count is ceil(floor(N / 240) / 4) (about 25 Hz). Long inputs have quadratic attention compute cost and growing activation memory. Native resampling differs from librosa/torchaudio; compare implementations using identical pre-resampled 24 kHz PCM. No Python or runtime model download is required. This release does not claim full GPU inference, batching, RVQ/training codes, intermediate-layer APIs or text support.

Source and conversion

• Source model revision: 0562a57814f6f8bbd9fdea0a25921a2fce1a841a.
• Original model.safetensors: 1,333,825,096 bytes; SHA-256 273febab2be02872c37d2c37e48a9d6c52c1c9392f3eeeabd498efa281ccb7a6.
• Official MuQ implementation: tencent-ailab/MuQ at 28847ea50cd31ac4b8b6a7dacc051ad7d1c7606a.
• Converter: clap.cpp convert_hf_to_gguf.py, native MuQ feature lineage published as 011dc9b449f9ba5a8924c546b2cd370964f6680c (PR #15). These preserved artifacts were generated during feature development; their exact generation commit was not separately recorded. Their source identity and subsequent native numerical checks are recorded below and in the attached provenance/validation files.

Reproduce using the pinned local safetensors/config snapshot and the converter:

python -m pip install -r requirements.txt
python convert_hf_to_gguf.py /path/to/muq-checkpoint muq-f16.gguf \
  --dtype f16 --muq-revision 0562a57814f6f8bbd9fdea0a25921a2fce1a841a
python convert_hf_to_gguf.py /path/to/muq-checkpoint muq-f32.gguf \
  --dtype f32 --muq-revision 0562a57814f6f8bbd9fdea0a25921a2fce1a841a

Conversion streams safetensors locally, validates configuration/tensor contracts, and does not execute downloaded model source. The resulting inference artifact has 438 tensors, omitting known unused training/RVQ/classifier/CLS and positional-convolution tensors.

Numerical validation and limitations

Both preserved GGUFs were freshly exercised with native frame and pooled inference against saved, hash-checked official-reference arrays. Source checkpoint SHA-256 was also rechecked against the Hub manifest. The synthetic broadband probe has 5041 samples at 24 kHz, producing six frames of width 1024:

Storage	Frame max absolute error	Pooled max absolute error
F32	2.74181366e-5	9.29832458e-6
F16	2.70605087e-5	1.00135803e-5

All native values were finite, and exact frame/value counts were checked. F16 reference weights are identically storage-rounded with FP32 activations, so these numbers measure inference parity, not F16 quality loss against untouched FP32 weights.

Known limitation: a separate almost-pure-tone published-weight diagnostic measured a frame error of 0.0139627, exceeding the unchanged strict FP32 tolerance of 5e-4. Near the power-dB noise floor, native FFT versus PyTorch float32 FFT differences can be amplified. A passing broadband probe is not a universal waveform error bound or a music retrieval benchmark. The diagnostic failure is not counted as a passing test.

validation.json records the fresh numerical results; provenance.json records source/runtime details. Verify downloaded files with:

sha256sum -c SHA256SUMS

Quantization runtime and measured loss

The compatible public runtime is gorse-io/clap.cpp commit b59e0cc69dfbeb03840f41e45ca5efd8edd5a18d (implementation tree 22f24b2d4caa80481604dfd980a0f912a88be25c). This is an immutable public feature-branch commit, not a claim that the pull request is merged. The artifacts and recorded native validation were produced from this exact implementation tree before the public commit existed; the later commit has the same tree. Generation and validation binary SHA-256 values remain in quantization-provenance.json. Historical float-only runtime pins are not sufficient for these quantized files.

Build on Linux with a C++17 compiler, CMake, and ICU development libraries installed (for example, libicu-dev on Debian/Ubuntu):

git clone https://github.com/gorse-io/clap.cpp.git
cd clap.cpp
git checkout b59e0cc69dfbeb03840f41e45ca5efd8edd5a18d
git submodule update --init --recursive
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DCLAP_USE_SYSTEM_ICU=ON
cmake --build build --target clap-model clap-cli -j2

No historical runtime patch needs applying to this commit; the required MERT2 FFT/F32-activation corrections are included. Download a quantized GGUF from this model repository and use build/clap-model as below. To reproduce quantization from the preserved F32 file, use ./build/clap quantize INPUT_F32.gguf OUTPUT.gguf --type q8_0 or --type q4_k_m (choose a new output path).

Q4_K_M is the encoder preset q4_k_m-v2-encoder-dense-v-ffn-down-projection-q6_k, not uniform Q4_K and not llama.cpp's decoder-layer schedule. Eligible block-aligned native dense/embedding matrices use Q4_K, with all architecture-selected value/FFN-down/final projection roles protected by Q6_K. Convolutions, normalization, frontend/GRN tensors, noneligible roles and unaligned row widths retain identical source floating-point bytes. The full per-tensor type/role/raw-byte audit is recorded in provenance. Canonical padded tensor shapes are preserved; the writer can omit trailing singleton dimensions in the serialized rank (notably 24 unchanged MERT2 GRN vectors originally stored as [channels,1,1]). Every such change is recorded; float types, element counts and raw bytes remain identical. Q8_0 uses canonical Q8_0 on eligible 32-aligned rows. Quantization started from the verified preserved F32 file, not F16.

Weights remain mapped; each current dense operand is decoded into operation-local FP32 scratch with FP32 activation semantics. This is not whole-model dequantization on load, nor a claim of proportional RAM savings or faster inference. Noneligible FP16 promotion behavior remains relevant when loading F16 sources. The new artifacts were tested serially using two CPU threads with real full-checkpoint native inference on the saved waveform/text fixtures. Audio-only models have no text support.

Native commands (after building the approved runtime):

./build/clap-model --model QUANTIZED_FILE --audio music.flac --threads 2
./build/clap-model --model QUANTIZED_FILE --audio music.flac --frames --threads 2

The following are losses against the same quantization-compatible runtime loading the preserved F32 GGUF, not strict storage-rounded oracle parity and not official-model/retrieval-quality acceptance. Numerical equality is not expected. Frame-sequence metrics compare the flattened sequence, not a minimum over individual frames. Historical F16/F32 parity evidence and known narrowband/near-noise-floor limitations remain historical; no universal waveform error bound or music quality claim is made.

Preset	Cases	Minimum cosine	Maximum relative L2	Maximum absolute error
Q8_0	2	0.999947096	0.0102914809	0.0358814001
Q4_K_M	2	0.995333699	0.0971760828	0.38529563

All output dimensions, flattened frame/value counts and finite-value checks passed. Norm ranges, fixture hashes, exact commands and per-case losses are in quantization-validation.json; model identity, licensing, source checkpoints, tensor audits, output sizes and hashes are in quantization-provenance.json. F16/F32 files and their original provenance/validation are not replaced.

License and attribution

Original and converted model weights: CC-BY-NC-4.0 (non-commercial). Conversion does not remove this restriction. Upstream MuQ source is MIT; clap.cpp code has its own license and does not relicense these weights. This is a Gorse format conversion, not an official OpenMuQ release.

Credit: Haina Zhu, Yizhi Zhou, Hangting Chen, Jianwei Yu, Ziyang Ma, Rongzhi Gu, Yi Luo, Wei Tan and Xie Chen, MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization (2025), arXiv:2501.01108. See the original model card and official implementation.