README.md · IsValorum/llama.cpp-clover-moe at main

Remove test qualifiers from coverage

dbea22a verified 1 day ago

6.98 kB

---
license: mit
library_name: llama.cpp
base_model: Diffuspace/Clover-35B-A3B-Preview
base_model_relation: quantized
language:
- en
tags:
- llama.cpp
- gguf
- clover
- clover_moe
- prebuilt
- runtime
- linux
- x86-64
- windows
- universal
- vulkan
- cpu
- cuda
- cuda12
- converter
- quantize
- imatrix
- diffusion
- block-diffusion
- moe
---
# llama.cpp with Clover support (clover_moe)
**Prebuilt Clover runtime by IsValorum**
<p align="left">
  <a href="https://ko-fi.com/isvalorum"><img src="https://ko-fi.com/img/githubbutton_sm.svg" alt="Support on Ko-fi" height="38"></a>
</p>
> [!IMPORTANT]
> **Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.**
>
> If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider [supporting on Ko-fi](https://ko-fi.com/isvalorum). Every contribution directly sustains future cloud compute hours and helps support my university studies.
Prebuilt Linux and Windows x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview), built from llama.cpp PR [#31306](https://github.com/ggml-org/llama.cpp/pull/31306) at commit `4838ffb0c8ade92f34d9157deea38d92758ab824`.

Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.

## Clover artifacts for this runtime

| Artifact | Size | Link |
|---|---|---|
| BF16 source | 69.38 GB | [Clover-35B-A3B-Preview-BF16-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-BF16-GGUF) |
| Q8_0 | 36.91 GB | [Clover-35B-A3B-Preview-Q8_0-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF) |
| 1.25M I-Matrix | 383,689,280 bytes | [Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF) |
| APEX-I NanoPlus V2.1 | 12.56 GB / 11.70 GiB | [Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF) |
| APEX-I MiniPlus V2.1 | 14.75 GB / 13.74 GiB | [Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF) |
## Files
| File | Description |
|---|---|
| `llama-clover-4838ffb-linux-x86_64-cpu.tar.gz` | Linux CPU build, `GGML_NATIVE=OFF` |
| `llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz` | CUDA 12 build, archs `80;86;89;90` |
| `llama-clover-4838ffb-converter-python.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py |
| `llama-clover-4838ffb-windows-x86_64-nvidia-amd-universal-cuda12.8-vulkan-cpu.zip` | Windows x86_64 universal: Nvidia CUDA 12.8 sm_50-120, AMD/Intel Vulkan, any CPU |
| `BUILD_INFO.json` | Compiler, flags and SHA256 per archive |
| `SHA256SUMS` | Checksums |
| `config.json` | Machine-readable build descriptor used by Hub download tracking |

## Coverage

| Platform | Target | Backend | Status |
|---|---|---|---|
| Linux x86_64 | Any x86-64 CPU | CPU, `GGML_NATIVE=OFF` | shipped |
| Linux x86_64 | NVIDIA sm_80 / 86 / 89 / 90 (A100, A10G, L40S, RTX 4090, H100/H200) | CUDA 12.0 | shipped |
| Linux x86_64 | NVIDIA Blackwell sm_100/120, AMD, Intel | none | not in this build |
| Windows x86_64 | NVIDIA sm_50 through sm_120 (Maxwell to Blackwell) | CUDA 12.8 | shipped |
| Windows x86_64 | AMD and Intel GPUs | Vulkan 1.4.363 | shipped |
| Windows x86_64 | Any x86-64 CPU | per-CPU DLLs, AVX-512 off | shipped |
| macOS, ARM64, ROCm on any OS | - | none | not published |
Windows is native. The ZIP carries its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver; extract it and run `set PATH=%~dp0bin;%PATH%` from the archive root. It ships the same tool set as the Linux archives, including `llama-speculative-simple`, `llama-quantize` and `llama-imatrix`.
The Linux x86_64 archives require `GLIBC_2.38` or newer, so Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build. Anything outside the table above, including macOS and ARM64, needs a build from the same branch and commit.

## Compatibility

- Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors.
- Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream.

## Usage

All runtime archives are dynamically linked and ship `llama-cli`, `llama-server`, `llama-diffusion-cli`, `llama-quantize`, `llama-imatrix`, and `llama-perplexity`. Point the loader at the bundled libraries before running. The archive names changed in this build; the extract directories kept their old names.

CPU:

```bash
tar -xzf llama-clover-4838ffb-linux-x86_64-cpu.tar.gz
cd llama-clover-b4838ffb-linux-x64-cpu
export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello"
```

CUDA 12:

```bash
tar -xzf llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz
cd llama-clover-b4838ffb-linux-x64-cuda12
export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello"
./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
```

Windows x86_64 (universal ZIP, no separate runtime install):
```bat
set PATH=%~dp0bin;%PATH%
llama-completion.exe -m Clover-35B-A3B-Preview.APEX-I-NanoPlus-V2.1.gguf -p "hi" -n 1 -c 128 -b 16 -ub 16 -ngl 8 -no-cnv
```
Run both lines from the extracted archive root. Drop `-ngl 8` for CPU or Vulkan-only use; raise it to offload more layers on a large NVIDIA card.
## Conversion
```bash
tar -xzf llama-clover-4838ffb-converter-python.tar.gz
python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
```
## Quantization and importance matrix
```bash
./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
```
The APEX-I V2.1 releases were produced with this build: `llama-quantize --imatrix Clover-VAL-1.25M-I-Matrix.gguf --tensor-type-file <recipe> clover-bf16.gguf <output> Q4_0 64`. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in `F32`.

## Notes

The CUDA packages require a compatible NVIDIA driver and CUDA 12.x runtime; the Windows package bundles its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.

	---
	license: mit
	library_name: llama.cpp
	base_model: Diffuspace/Clover-35B-A3B-Preview
	base_model_relation: quantized
	language:
	- en
	tags:
	- llama.cpp
	- gguf
	- clover
	- clover_moe
	- prebuilt
	- runtime
	- linux
	- x86-64
	- windows
	- universal
	- vulkan
	- cpu
	- cuda
	- cuda12
	- converter
	- quantize
	- imatrix
	- diffusion
	- block-diffusion
	- moe
	---

	# llama.cpp with Clover support (clover_moe)

	Prebuilt Clover runtime by IsValorum

	<p align="left">
	<a href="https://ko-fi.com/isvalorum"><img src="https://ko-fi.com/img/githubbutton_sm.svg" alt="Support on Ko-fi" height="38"></a>
	</p>

	> [!IMPORTANT]
	> Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.
	>
	> If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider [supporting on Ko-fi](https://ko-fi.com/isvalorum). Every contribution directly sustains future cloud compute hours and helps support my university studies.

	Prebuilt Linux and Windows x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview), built from llama.cpp PR [#31306](https://github.com/ggml-org/llama.cpp/pull/31306) at commit `4838ffb0c8ade92f34d9157deea38d92758ab824`.

	Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.

	## Clover artifacts for this runtime

	\| Artifact \| Size \| Link \|
	\|---\|---\|---\|
	\| BF16 source \| 69.38 GB \| [Clover-35B-A3B-Preview-BF16-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-BF16-GGUF) \|
	\| Q8_0 \| 36.91 GB \| [Clover-35B-A3B-Preview-Q8_0-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF) \|
	\| 1.25M I-Matrix \| 383,689,280 bytes \| [Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF) \|
	\| APEX-I NanoPlus V2.1 \| 12.56 GB / 11.70 GiB \| [Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF) \|
	\| APEX-I MiniPlus V2.1 \| 14.75 GB / 13.74 GiB \| [Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF) \|

	## Files

	\| File \| Description \|
	\|---\|---\|
	\| `llama-clover-4838ffb-linux-x86_64-cpu.tar.gz` \| Linux CPU build, `GGML_NATIVE=OFF` \|
	\| `llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz` \| CUDA 12 build, archs `80;86;89;90` \|
	\| `llama-clover-4838ffb-converter-python.tar.gz` \| HF-to-GGUF converter, conversion code and gguf-py \|
	\| `llama-clover-4838ffb-windows-x86_64-nvidia-amd-universal-cuda12.8-vulkan-cpu.zip` \| Windows x86_64 universal: Nvidia CUDA 12.8 sm_50-120, AMD/Intel Vulkan, any CPU \|
	\| `BUILD_INFO.json` \| Compiler, flags and SHA256 per archive \|
	\| `SHA256SUMS` \| Checksums \|
	\| `config.json` \| Machine-readable build descriptor used by Hub download tracking \|

	## Coverage

	\| Platform \| Target \| Backend \| Status \|
	\|---\|---\|---\|---\|
	\| Linux x86_64 \| Any x86-64 CPU \| CPU, `GGML_NATIVE=OFF` \| shipped \|
	\| Linux x86_64 \| NVIDIA sm_80 / 86 / 89 / 90 (A100, A10G, L40S, RTX 4090, H100/H200) \| CUDA 12.0 \| shipped \|
	\| Linux x86_64 \| NVIDIA Blackwell sm_100/120, AMD, Intel \| none \| not in this build \|
	\| Windows x86_64 \| NVIDIA sm_50 through sm_120 (Maxwell to Blackwell) \| CUDA 12.8 \| shipped \|
	\| Windows x86_64 \| AMD and Intel GPUs \| Vulkan 1.4.363 \| shipped \|
	\| Windows x86_64 \| Any x86-64 CPU \| per-CPU DLLs, AVX-512 off \| shipped \|
	\| macOS, ARM64, ROCm on any OS \| - \| none \| not published \|

	Windows is native. The ZIP carries its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver; extract it and run `set PATH=%~dp0bin;%PATH%` from the archive root. It ships the same tool set as the Linux archives, including `llama-speculative-simple`, `llama-quantize` and `llama-imatrix`.

	The Linux x86_64 archives require `GLIBC_2.38` or newer, so Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build. Anything outside the table above, including macOS and ARM64, needs a build from the same branch and commit.

	## Compatibility

	- Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors.
	- Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream.

	## Usage

	All runtime archives are dynamically linked and ship `llama-cli`, `llama-server`, `llama-diffusion-cli`, `llama-quantize`, `llama-imatrix`, and `llama-perplexity`. Point the loader at the bundled libraries before running. The archive names changed in this build; the extract directories kept their old names.

	CPU:

	```bash
	tar -xzf llama-clover-4838ffb-linux-x86_64-cpu.tar.gz
	cd llama-clover-b4838ffb-linux-x64-cpu
	export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
	./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello"
	```

	CUDA 12:

	```bash
	tar -xzf llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz
	cd llama-clover-b4838ffb-linux-x64-cuda12
	export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
	./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello"
	./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
	```

	Windows x86_64 (universal ZIP, no separate runtime install):

	```bat
	set PATH=%~dp0bin;%PATH%
	llama-completion.exe -m Clover-35B-A3B-Preview.APEX-I-NanoPlus-V2.1.gguf -p "hi" -n 1 -c 128 -b 16 -ub 16 -ngl 8 -no-cnv
	```

	Run both lines from the extracted archive root. Drop `-ngl 8` for CPU or Vulkan-only use; raise it to offload more layers on a large NVIDIA card.

	## Conversion

	```bash
	tar -xzf llama-clover-4838ffb-converter-python.tar.gz
	python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
	```

	## Quantization and importance matrix

	```bash
	./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
	./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
	```

	The APEX-I V2.1 releases were produced with this build: `llama-quantize --imatrix Clover-VAL-1.25M-I-Matrix.gguf --tensor-type-file <recipe> clover-bf16.gguf <output> Q4_0 64`. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in `F32`.

	## Notes

	The CUDA packages require a compatible NVIDIA driver and CUDA 12.x runtime; the Windows package bundles its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.