CLaMP3 SAAS — single-file GGUF with bundled MERT

Unofficial conversion published by gorse-io. Each file embeds the pretrained MERT-v1-95M waveform frontend, CLaMP3 text/audio encoders, and tokenizer. Native inference needs only one GGUF: no Python, external MERT file, feature extraction step, tokenizer download, or model download is needed at runtime.

Non-commercial use only for the bundled release: CLaMP3 retains its MIT license (LICENSE); the bundled MERT weights are CC-BY-NC-4.0 (LICENSE-MERT-CC-BY-NC-4.0.txt, NOTICE-MERT.md). The model-card license reflects the MERT restriction, not a claim that CLaMP3 itself was relicensed.

Files

All four files contain 606 tensors: 397 CLaMP3 + 209 MERT. Outputs are 768-dimensional projections and are not L2-normalized.

File	Exact bytes	Storage
`clamp3-saas-f32.gguf`	1,841,789,696	606 F32 tensors
`clamp3-saas-f16.gguf`	926,206,208	232 F16 matrices, 374 F32 tensors
`clamp3-saas-q8_0.gguf`	773,876,096	150 Q8_0 CLaMP3 matrices, 456 F32 tensors
`clamp3-saas-q4_k_m.gguf`	610,655,456	100 Q4_K + 50 Q6_K CLaMP3 matrices, 456 F32 tensors

Quantized files are generated directly from the bundled F32 file. All 209 MERT tensors retain their exact F32 type, shape and bytes in both quantized files; only eligible CLaMP3 matrices are quantized. Q4_K_M uses clap.cpp's encoder policy q4_k_m-v1-all-v-ffn-down-projection-q6_k, not llama.cpp's decoder layer schedule. Attention-V, FFN-down and final projection roles use Q6_K; other eligible matrices use Q4_K. No calibration corpus or importance matrix was used.

manifest.json, SHA256SUMS, and PACKAGE_SHA256SUMS record artifact identities. F16 means FP16 matrix storage, not BF16 or an accuracy-equivalence claim.

Compatible runtime

Requires the bundled-MERT implementation merged in gorse-io/clap.cpp PR #25. The public merged commit is 27c75120f1bdd66ea44ce53df79567f547ace3cf. The validated implementation tree is 1aec95ca5b861f1c4404e425862d3f4d3808554c; the reviewed local commit eebca6c9d6f635b1233df0e1d51d1b6749e9174b has that same tree. Use the real clap-model binary, not the legacy dummy clap embedding or HTTP server. This architecture is not supported by generic llama.cpp, Ollama, Transformers GGUF loading, or Hub hosted inference.

git clone --recursive https://github.com/gorse-io/clap.cpp.git
cd clap.cpp
git checkout 27c75120f1bdd66ea44ce53df79567f547ace3cf
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCLAP_USE_SYSTEM_ICU=ON
cmake --build build --target clap-model clap-cli -j2
./build/clap-model --model /path/to/clamp3-saas-f16.gguf --threads 2 \
  --audio /path/to/music.wav
./build/clap-model --model /path/to/clamp3-saas-f16.gguf --threads 2 \
  --text 'A calm acoustic guitar melody.'

Audio and text are separate invocations (--audio and --text are mutually exclusive). WAV, MP3 and FLAC are decoded, mixed to mono and resampled internally. The mandatory MERT frontend computes the five-second window features consumed by the CLaMP3 audio branch. Text uses the embedded XLM-R tokenizer. CLaMP3 retains its boundary frames, padding/overlap, masked pooling, learned projection and weighted chunk aggregation. Normalize outputs yourself if your application requires cosine similarity.

Breaking compatibility: old 397-tensor feature-only files are unsupported by this runtime. Public audio_features() and --audio-features were removed; there is no external-MERT fallback or --mert-source conversion option. Download one of the replacement files above and pass raw audio instead of NPY features. The original feature-only release remains in Git history at commit 2fb4edd1e40c8e741261ced0f1aab58bdc648e46; its old evidence is not evidence for this release.

Symbolic encoders and ABC/MTF/MusicXML/image inputs are not included. Unused pooler/audio word-embedding and symbolic checkpoint tensors remain omitted. This is not the entire original multimodal checkpoint.

Validation and limitations

All four formats were executed through the production CPU CLI with two threads: English and Chinese text, broadband waveform, two-window waveform, pre-resampled waveform and input requiring native resampling. Every invocation returned 768 finite values. These are synthetic waveform probes through real pretrained weights, not recorded music or a retrieval benchmark.

F32 text matches the retained independent original-weight Transformers references (maximum absolute error below 3e-7 for these two prompts).
A separately completed published-weight F32 oracle compared the bundled raw-audio path against independent MERT extraction followed by the private CLaMP3 feature encoder: maximum absolute error below 3.4e-6 over broadband, two-window and resampled cases. Its report and original log are retained in validation/published-f32-oracle*; this is not a four-precision Python oracle.
The 397 CLaMP3 tensors in F32/F16 match the previously source-audited tensors byte-for-byte at their respective storage precision. The retained source-tensor-audit.json is that branch-only historical audit, not an audit of the newly bundled MERT tensors. New full-file tensor inventories are in validation/tensors-*.json.
Both quantized files preserve all MERT tensor bytes from bundled F32.
Static and shared native regression suites each passed 36/36 tests on the validated tree; their logs are retained in validation/.

validation.json records each case's error versus the newly executed bundled F32 baseline, plus text comparisons with independent original-F32 references. F16-versus-F32 measurements include storage loss in both encoders; these are not storage-rounded-reference parity results. Q4_K_M has material text drift (e.g. cosine about 0.980 on the English probe); do not treat it as equivalent to F32/F16. Prefer F32/F16 as fidelity baselines and evaluate quantized retrieval on your own task. No retrieval scores, broad quality equivalence, GPU parity, or third-party runtime compatibility are claimed.

Sources and reproduction

CLaMP3 checkpoint: sander-wood/clamp3 revision 355625cc1c6f73726bbcd0eb9276ac7152d56426, SHA-256 5033f868e3977be3945ee416b5a1718d5589a173c7ba8982231d8c94a6441d80. MERT dependency: m-a-p/MERT-v1-95M revision 12af15fef9d0ac838c3f475bfbbf26d2060dd4f5, checkpoint SHA-256 a2b8b747f72c06e0595aeae41ae5473f4364938c6b39b2c58be38c48e6bd3fcd. Both local checkpoint hashes were reverified against immutable Hub LFS manifests. Tokenizer/config: FacebookAI/xlm-roberta-base revision e73636d4f797dec63c3081bb6ed5c7b0bb3f2089.

F32 was reused from the already completed published-weight conversion, generated before the reviewed commit was created but from the same exact implementation tree. F16 and both quantizations were generated for this release from the reviewed checkout. provenance.json separates generation tree, reviewed commit, public merged commit and binary identities. See REPRODUCE.md for conversion and validation instructions. Original CLaMP3: Sander Wood and contributors. MERT: Yizhi Li, Ruibin Yuan, Ge Zhang and the MERT/m-a-p contributors.