--- license: mit library_name: llama.cpp base_model: Diffuspace/Clover-35B-A3B-Preview base_model_relation: quantized language: - en tags: - llama.cpp - gguf - clover - clover_moe - prebuilt - runtime - linux - x86-64 - windows - universal - vulkan - cpu - cuda - cuda12 - converter - quantize - imatrix - diffusion - block-diffusion - moe --- # llama.cpp with Clover support (clover_moe) **Prebuilt Clover runtime by IsValorum**

Support on Ko-fi

> [!IMPORTANT] > **Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.** > > If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider [supporting on Ko-fi](https://ko-fi.com/isvalorum). Every contribution directly sustains future cloud compute hours and helps support my university studies. Prebuilt Linux and Windows x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview), built from llama.cpp PR [#31306](https://github.com/ggml-org/llama.cpp/pull/31306) at commit `4838ffb0c8ade92f34d9157deea38d92758ab824`. Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility. ## Clover artifacts for this runtime | Artifact | Size | Link | |---|---|---| | BF16 source | 69.38 GB | [Clover-35B-A3B-Preview-BF16-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-BF16-GGUF) | | Q8_0 | 36.91 GB | [Clover-35B-A3B-Preview-Q8_0-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF) | | 1.25M I-Matrix | 383,689,280 bytes | [Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF) | | APEX-I NanoPlus V2.1 | 12.56 GB / 11.70 GiB | [Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF) | | APEX-I MiniPlus V2.1 | 14.75 GB / 13.74 GiB | [Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF) | ## Files | File | Description | |---|---| | `llama-clover-4838ffb-linux-x86_64-cpu.tar.gz` | Linux CPU build, `GGML_NATIVE=OFF` | | `llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz` | CUDA 12 build, archs `80;86;89;90` | | `llama-clover-4838ffb-converter-python.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py | | `llama-clover-4838ffb-windows-x86_64-nvidia-amd-universal-cuda12.8-vulkan-cpu.zip` | Windows x86_64 universal: Nvidia CUDA 12.8 sm_50-120, AMD/Intel Vulkan, any CPU | | `BUILD_INFO.json` | Compiler, flags and SHA256 per archive | | `SHA256SUMS` | Checksums | | `config.json` | Machine-readable build descriptor used by Hub download tracking | ## Coverage | Platform | Target | Backend | Status | |---|---|---|---| | Linux x86_64 | Any x86-64 CPU | CPU, `GGML_NATIVE=OFF` | shipped | | Linux x86_64 | NVIDIA sm_80 / 86 / 89 / 90 (A100, A10G, L40S, RTX 4090, H100/H200) | CUDA 12.0 | shipped | | Linux x86_64 | NVIDIA Blackwell sm_100/120, AMD, Intel | none | not in this build | | Windows x86_64 | NVIDIA sm_50 through sm_120 (Maxwell to Blackwell) | CUDA 12.8 | shipped | | Windows x86_64 | AMD and Intel GPUs | Vulkan 1.4.363 | shipped | | Windows x86_64 | Any x86-64 CPU | per-CPU DLLs, AVX-512 off | shipped | | macOS, ARM64, ROCm on any OS | - | none | not published | Windows is native. The ZIP carries its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver; extract it and run `set PATH=%~dp0bin;%PATH%` from the archive root. It ships the same tool set as the Linux archives, including `llama-speculative-simple`, `llama-quantize` and `llama-imatrix`. The Linux x86_64 archives require `GLIBC_2.38` or newer, so Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build. Anything outside the table above, including macOS and ARM64, needs a build from the same branch and commit. ## Compatibility - Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors. - Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream. ## Usage All runtime archives are dynamically linked and ship `llama-cli`, `llama-server`, `llama-diffusion-cli`, `llama-quantize`, `llama-imatrix`, and `llama-perplexity`. Point the loader at the bundled libraries before running. The archive names changed in this build; the extract directories kept their old names. CPU: ```bash tar -xzf llama-clover-4838ffb-linux-x86_64-cpu.tar.gz cd llama-clover-b4838ffb-linux-x64-cpu export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH ./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello" ``` CUDA 12: ```bash tar -xzf llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz cd llama-clover-b4838ffb-linux-x64-cuda12 export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH ./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello" ./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99 ``` Windows x86_64 (universal ZIP, no separate runtime install): ```bat set PATH=%~dp0bin;%PATH% llama-completion.exe -m Clover-35B-A3B-Preview.APEX-I-NanoPlus-V2.1.gguf -p "hi" -n 1 -c 128 -b 16 -ub 16 -ngl 8 -no-cnv ``` Run both lines from the extracted archive root. Drop `-ngl 8` for CPU or Vulkan-only use; raise it to offload more layers on a large NVIDIA card. ## Conversion ```bash tar -xzf llama-clover-4838ffb-converter-python.tar.gz python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf ``` ## Quantization and importance matrix ```bash ./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0 ./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99 ``` The APEX-I V2.1 releases were produced with this build: `llama-quantize --imatrix Clover-VAL-1.25M-I-Matrix.gguf --tensor-type-file clover-bf16.gguf Q4_0 64`. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in `F32`. ## Notes The CUDA packages require a compatible NVIDIA driver and CUDA 12.x runtime; the Windows package bundles its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.