IsValorum commited on
Commit
ccdaff2
·
verified ·
1 Parent(s): 992e5a2

Card: align metadata with the Clover quantization family

Browse files
Files changed (1) hide show
  1. README.md +64 -13
README.md CHANGED
@@ -2,8 +2,28 @@
2
  license: mit
3
  library_name: llama.cpp
4
  base_model: Diffuspace/Clover-35B-A3B-Preview
5
- tags: [llama.cpp, clover, clover_moe, prebuilt, runtime, linux, cuda]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  ---
 
7
  # llama.cpp with Clover support (clover_moe)
8
 
9
  **Prebuilt Clover runtime by IsValorum**
@@ -21,24 +41,51 @@ Prebuilt Linux x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://h
21
 
22
  Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.
23
 
24
- ## Compatibility
25
 
26
- - Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors.
27
- - Linux x86_64: the CPU package runs on any Linux; the CUDA package requires an NVIDIA GPU and CUDA 12.x runtime, with archs 80/86/89/90 (A100, H100/H200, L40S, RTX 4090, A10G). Blackwell 100/120 is not in this build.
28
- - Windows: no native binary here. Use WSL2 with the Linux CPU or CUDA package; a native Windows build must be compiled from the PR with CMake and MSVC. The GGUF files themselves are OS-agnostic.
29
- - macOS: no binary here; use a Metal build of the PR or the third-party IQ4_XS Mac build.
30
- - Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream.
 
 
 
 
31
 
32
  | File | Description |
33
  |---|---|
34
- | `llama-clover-b4838ffb-linux-x64-cpu.tar.gz` | Portable CPU build |
35
- | `llama-clover-b4838ffb-linux-x64-cuda12.tar.gz` | CUDA build, archs `80;86;89;90` |
36
  | `llama-clover-b4838ffb-converter.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py |
37
- | `BUILD_INFO.json` | Compiler, flags and SHA256 |
38
  | `SHA256SUMS` | Checksums |
 
 
 
 
 
 
 
 
 
 
39
 
40
  ## Usage
41
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  ```bash
43
  tar -xzf llama-clover-b4838ffb-linux-x64-cuda12.tar.gz
44
  cd llama-clover-b4838ffb-linux-x64-cuda12
@@ -47,18 +94,22 @@ export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
47
  ./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
48
  ```
49
 
50
- Conversion:
51
 
52
  ```bash
53
  tar -xzf llama-clover-b4838ffb-converter.tar.gz
54
  python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
55
  ```
56
 
57
- Quantization and importance matrix:
58
 
59
  ```bash
60
  ./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
61
  ./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
62
  ```
63
 
64
- The CUDA package requires a compatible NVIDIA driver and CUDA 12.x runtime. The CPU package is built with `GGML_NATIVE=OFF`. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.
 
 
 
 
 
2
  license: mit
3
  library_name: llama.cpp
4
  base_model: Diffuspace/Clover-35B-A3B-Preview
5
+ language:
6
+ - en
7
+ tags:
8
+ - llama.cpp
9
+ - gguf
10
+ - clover
11
+ - clover_moe
12
+ - prebuilt
13
+ - runtime
14
+ - linux
15
+ - x86-64
16
+ - cpu
17
+ - cuda
18
+ - cuda12
19
+ - converter
20
+ - quantize
21
+ - imatrix
22
+ - diffusion
23
+ - block-diffusion
24
+ - moe
25
  ---
26
+
27
  # llama.cpp with Clover support (clover_moe)
28
 
29
  **Prebuilt Clover runtime by IsValorum**
 
41
 
42
  Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.
43
 
44
+ ## Clover artifacts for this runtime
45
 
46
+ | Artifact | Size | Link |
47
+ |---|---|---|
48
+ | BF16 source | 69.38 GB | [Clover-35B-A3B-Preview-BF16-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-BF16-GGUF) |
49
+ | Q8_0 | 36.91 GB | [Clover-35B-A3B-Preview-Q8_0-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF) |
50
+ | 1.25M I-Matrix | 383,689,280 bytes | [Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF) |
51
+ | APEX-I NanoPlus V2.1 | 12.56 GB / 11.70 GiB | [Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF) |
52
+ | APEX-I MiniPlus V2.1 | 14.75 GB / 13.74 GiB | [Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF) |
53
+
54
+ ## Files
55
 
56
  | File | Description |
57
  |---|---|
58
+ | `llama-clover-b4838ffb-linux-x64-cpu.tar.gz` | Portable CPU build, `GGML_NATIVE=OFF`, runs on any x86-64 machine |
59
+ | `llama-clover-b4838ffb-linux-x64-cuda12.tar.gz` | CUDA 12 build, archs `80;86;89;90` |
60
  | `llama-clover-b4838ffb-converter.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py |
61
+ | `BUILD_INFO.json` | Compiler, flags and SHA256 per archive |
62
  | `SHA256SUMS` | Checksums |
63
+ | `config.json` | Machine-readable build descriptor used by Hub download tracking |
64
+
65
+ ## Compatibility
66
+
67
+ - Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors.
68
+ - Linux x86_64: the CPU package runs on any Linux; the CUDA package requires an NVIDIA GPU and a CUDA 12.x runtime, with archs 80/86/89/90 (A100, H100/H200, L40S, RTX 4090, A10G). Blackwell 100/120 is not in this build.
69
+ - glibc: the Linux x64 archives require `GLIBC_2.38` or newer. Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build from the branch.
70
+ - Windows: no native binary here. Use WSL2 with the Linux CPU or CUDA package; a native Windows build must be compiled from the PR with CMake and MSVC. The GGUF files themselves are OS-agnostic.
71
+ - macOS: no binary here; use a Metal build of the PR or the third-party IQ4_XS Mac build.
72
+ - Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream.
73
 
74
  ## Usage
75
 
76
+ Both runtime archives are dynamically linked and ship `llama-cli`, `llama-server`, `llama-diffusion-cli`, `llama-quantize`, `llama-imatrix`, and `llama-perplexity`. Point the loader at the bundled libraries before running.
77
+
78
+ CPU:
79
+
80
+ ```bash
81
+ tar -xzf llama-clover-b4838ffb-linux-x64-cpu.tar.gz
82
+ cd llama-clover-b4838ffb-linux-x64-cpu
83
+ export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
84
+ ./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello"
85
+ ```
86
+
87
+ CUDA 12:
88
+
89
  ```bash
90
  tar -xzf llama-clover-b4838ffb-linux-x64-cuda12.tar.gz
91
  cd llama-clover-b4838ffb-linux-x64-cuda12
 
94
  ./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
95
  ```
96
 
97
+ ## Conversion
98
 
99
  ```bash
100
  tar -xzf llama-clover-b4838ffb-converter.tar.gz
101
  python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
102
  ```
103
 
104
+ ## Quantization and importance matrix
105
 
106
  ```bash
107
  ./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
108
  ./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
109
  ```
110
 
111
+ The APEX-I V2.1 releases were produced with this build: `llama-quantize --imatrix imatrix-1.25M.gguf --tensor-type-file <recipe> clover-bf16.gguf <output> Q4_0 64`. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in `F32`.
112
+
113
+ ## Notes
114
+
115
+ The CUDA package requires a compatible NVIDIA driver and CUDA 12.x runtime. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.