File size: 6,981 Bytes
ed14f02 dbea22a ed14f02 e31f211 dbea22a e31f211 ed14f02 e86d429 ed14f02 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | ---
license: mit
library_name: llama.cpp
base_model: Diffuspace/Clover-35B-A3B-Preview
base_model_relation: quantized
language:
- en
tags:
- llama.cpp
- gguf
- clover
- clover_moe
- prebuilt
- runtime
- linux
- x86-64
- windows
- universal
- vulkan
- cpu
- cuda
- cuda12
- converter
- quantize
- imatrix
- diffusion
- block-diffusion
- moe
---
# llama.cpp with Clover support (clover_moe)
**Prebuilt Clover runtime by IsValorum**
<p align="left">
<a href="https://ko-fi.com/isvalorum"><img src="https://ko-fi.com/img/githubbutton_sm.svg" alt="Support on Ko-fi" height="38"></a>
</p>
> [!IMPORTANT]
> **Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.**
>
> If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider [supporting on Ko-fi](https://ko-fi.com/isvalorum). Every contribution directly sustains future cloud compute hours and helps support my university studies.
Prebuilt Linux and Windows x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview), built from llama.cpp PR [#31306](https://github.com/ggml-org/llama.cpp/pull/31306) at commit `4838ffb0c8ade92f34d9157deea38d92758ab824`.
Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.
## Clover artifacts for this runtime
| Artifact | Size | Link |
|---|---|---|
| BF16 source | 69.38 GB | [Clover-35B-A3B-Preview-BF16-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-BF16-GGUF) |
| Q8_0 | 36.91 GB | [Clover-35B-A3B-Preview-Q8_0-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF) |
| 1.25M I-Matrix | 383,689,280 bytes | [Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF) |
| APEX-I NanoPlus V2.1 | 12.56 GB / 11.70 GiB | [Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF) |
| APEX-I MiniPlus V2.1 | 14.75 GB / 13.74 GiB | [Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF) |
## Files
| File | Description |
|---|---|
| `llama-clover-4838ffb-linux-x86_64-cpu.tar.gz` | Linux CPU build, `GGML_NATIVE=OFF` |
| `llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz` | CUDA 12 build, archs `80;86;89;90` |
| `llama-clover-4838ffb-converter-python.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py |
| `llama-clover-4838ffb-windows-x86_64-nvidia-amd-universal-cuda12.8-vulkan-cpu.zip` | Windows x86_64 universal: Nvidia CUDA 12.8 sm_50-120, AMD/Intel Vulkan, any CPU |
| `BUILD_INFO.json` | Compiler, flags and SHA256 per archive |
| `SHA256SUMS` | Checksums |
| `config.json` | Machine-readable build descriptor used by Hub download tracking |
## Coverage
| Platform | Target | Backend | Status |
|---|---|---|---|
| Linux x86_64 | Any x86-64 CPU | CPU, `GGML_NATIVE=OFF` | shipped |
| Linux x86_64 | NVIDIA sm_80 / 86 / 89 / 90 (A100, A10G, L40S, RTX 4090, H100/H200) | CUDA 12.0 | shipped |
| Linux x86_64 | NVIDIA Blackwell sm_100/120, AMD, Intel | none | not in this build |
| Windows x86_64 | NVIDIA sm_50 through sm_120 (Maxwell to Blackwell) | CUDA 12.8 | shipped |
| Windows x86_64 | AMD and Intel GPUs | Vulkan 1.4.363 | shipped |
| Windows x86_64 | Any x86-64 CPU | per-CPU DLLs, AVX-512 off | shipped |
| macOS, ARM64, ROCm on any OS | - | none | not published |
Windows is native. The ZIP carries its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver; extract it and run `set PATH=%~dp0bin;%PATH%` from the archive root. It ships the same tool set as the Linux archives, including `llama-speculative-simple`, `llama-quantize` and `llama-imatrix`.
The Linux x86_64 archives require `GLIBC_2.38` or newer, so Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build. Anything outside the table above, including macOS and ARM64, needs a build from the same branch and commit.
## Compatibility
- Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors.
- Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream.
## Usage
All runtime archives are dynamically linked and ship `llama-cli`, `llama-server`, `llama-diffusion-cli`, `llama-quantize`, `llama-imatrix`, and `llama-perplexity`. Point the loader at the bundled libraries before running. The archive names changed in this build; the extract directories kept their old names.
CPU:
```bash
tar -xzf llama-clover-4838ffb-linux-x86_64-cpu.tar.gz
cd llama-clover-b4838ffb-linux-x64-cpu
export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello"
```
CUDA 12:
```bash
tar -xzf llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz
cd llama-clover-b4838ffb-linux-x64-cuda12
export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello"
./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
```
Windows x86_64 (universal ZIP, no separate runtime install):
```bat
set PATH=%~dp0bin;%PATH%
llama-completion.exe -m Clover-35B-A3B-Preview.APEX-I-NanoPlus-V2.1.gguf -p "hi" -n 1 -c 128 -b 16 -ub 16 -ngl 8 -no-cnv
```
Run both lines from the extracted archive root. Drop `-ngl 8` for CPU or Vulkan-only use; raise it to offload more layers on a large NVIDIA card.
## Conversion
```bash
tar -xzf llama-clover-4838ffb-converter-python.tar.gz
python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
```
## Quantization and importance matrix
```bash
./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
```
The APEX-I V2.1 releases were produced with this build: `llama-quantize --imatrix Clover-VAL-1.25M-I-Matrix.gguf --tensor-type-file <recipe> clover-bf16.gguf <output> Q4_0 64`. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in `F32`.
## Notes
The CUDA packages require a compatible NVIDIA driver and CUDA 12.x runtime; the Windows package bundles its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.
|