Correct platform note, Windows compatibility and usage, archive names

Browse files

Files changed (1) hide show

README.md +132 -120

README.md CHANGED Viewed

@@ -1,120 +1,132 @@
----
-license: mit
-library_name: llama.cpp
-base_model: Diffuspace/Clover-35B-A3B-Preview
-base_model_relation: quantized
-language:
-- en
-tags:
-- llama.cpp
-- gguf
-- clover
-- clover_moe
-- prebuilt
-- runtime
-- linux
-- x86-64
-- cpu
-- cuda
-- cuda12
-- converter
-- quantize
-- imatrix
-- diffusion
-- block-diffusion
-- moe
----
-# llama.cpp with Clover support (clover_moe)
-**Prebuilt Clover runtime by IsValorum**
-<p align="left">
-  <a href="https://ko-fi.com/isvalorum"><img src="https://ko-fi.com/img/githubbutton_sm.svg" alt="Support on Ko-fi" height="38"></a>
-</p>
-> [!IMPORTANT]
-> **Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.**
->
-> If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider [supporting on Ko-fi](https://ko-fi.com/isvalorum). Every contribution directly sustains future cloud compute hours and helps support my university studies.
-Prebuilt Linux and Windows x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview), built from llama.cpp PR [#31306](https://github.com/ggml-org/llama.cpp/pull/31306) at commit `4838ffb0c8ade92f34d9157deea38d92758ab824`.
-Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.
-## Clover artifacts for this runtime
-| Artifact | Size | Link |
-|---|---|---|
-| BF16 source | 69.38 GB | [Clover-35B-A3B-Preview-BF16-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-BF16-GGUF) |
-| Q8_0 | 36.91 GB | [Clover-35B-A3B-Preview-Q8_0-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF) |
-| 1.25M I-Matrix | 383,689,280 bytes | [Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF) |
-| APEX-I NanoPlus V2.1 | 12.56 GB / 11.70 GiB | [Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF) |
-| APEX-I MiniPlus V2.1 | 14.75 GB / 13.74 GiB | [Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF) |
-## Files
-| File | Description |
-|---|---|
-| `llama-clover-4838ffb-linux-x86_64-cpu.tar.gz` | Linux CPU build, `GGML_NATIVE=OFF` |
-| `llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz` | CUDA 12 build, archs `80;86;89;90` |
-| `llama-clover-4838ffb-converter-python.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py |
-| `llama-clover-4838ffb-windows-x86_64-nvidia-amd-universal-cuda12.8-vulkan-cpu.zip` | Windows x86_64 universal: Nvidia CUDA 12.8 sm_50-120, AMD/Intel Vulkan, any CPU. Tested with NanoPlus: CPU, CUDA -ngl 8, Vulkan Intel |
-| `BUILD_INFO.json` | Compiler, flags and SHA256 per archive |
-| `SHA256SUMS` | Checksums |
-| `config.json` | Machine-readable build descriptor used by Hub download tracking |
-## Compatibility
-> [!NOTE]
-> **Platform availability.** The precompiled builds published today run on Linux only. Universal builds are in progress for Windows, AMD GPUs, NVIDIA GPUs and any CPU; until they ship, any other platform needs a build from the same branch.
-- Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors.
-- Linux x86_64: the CPU package runs on any Linux; the CUDA package requires an NVIDIA GPU and a CUDA 12.x runtime, with archs 80/86/89/90 (A100, H100/H200, L40S, RTX 4090, A10G). Blackwell 100/120 is not in this build.
-- glibc: the Linux x64 archives require `GLIBC_2.38` or newer. Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build from the branch.
-- Windows: no native binary here. Use WSL2 with the Linux CPU or CUDA package; a native Windows build must be compiled from the PR with CMake and MSVC. The GGUF files themselves are OS-agnostic.
-- macOS: no binary here; use a Metal build of the PR or the third-party IQ4_XS Mac build.
-- Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream.
-## Usage
-Both runtime archives are dynamically linked and ship `llama-cli`, `llama-server`, `llama-diffusion-cli`, `llama-quantize`, `llama-imatrix`, and `llama-perplexity`. Point the loader at the bundled libraries before running.
-CPU:
-```bash
-tar -xzf llama-clover-b4838ffb-linux-x64-cpu.tar.gz
-cd llama-clover-b4838ffb-linux-x64-cpu
-export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
-./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello"
-```
-CUDA 12:
-```bash
-tar -xzf llama-clover-b4838ffb-linux-x64-cuda12.tar.gz
-cd llama-clover-b4838ffb-linux-x64-cuda12
-export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
-./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello"
-./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
-```
-## Conversion
-```bash
-tar -xzf llama-clover-b4838ffb-converter.tar.gz
-python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
-```
-## Quantization and importance matrix
-```bash
-./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
-./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
-```
-The APEX-I V2.1 releases were produced with this build: `llama-quantize --imatrix imatrix-1.25M.gguf --tensor-type-file <recipe> clover-bf16.gguf <output> Q4_0 64`. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in `F32`.
-## Notes
-The CUDA package requires a compatible NVIDIA driver and CUDA 12.x runtime. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.

+---
+license: mit
+library_name: llama.cpp
+base_model: Diffuspace/Clover-35B-A3B-Preview
+base_model_relation: quantized
+language:
+- en
+tags:
+- llama.cpp
+- gguf
+- clover
+- clover_moe
+- prebuilt
+- runtime
+- linux
+- x86-64
+- windows
+- universal
+- vulkan
+- cpu
+- cuda
+- cuda12
+- converter
+- quantize
+- imatrix
+- diffusion
+- block-diffusion
+- moe
+---
+# llama.cpp with Clover support (clover_moe)
+**Prebuilt Clover runtime by IsValorum**
+<p align="left">
+  <a href="https://ko-fi.com/isvalorum"><img src="https://ko-fi.com/img/githubbutton_sm.svg" alt="Support on Ko-fi" height="38"></a>
+</p>
+> [!IMPORTANT]
+> **Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.**
+>
+> If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider [supporting on Ko-fi](https://ko-fi.com/isvalorum). Every contribution directly sustains future cloud compute hours and helps support my university studies.
+Prebuilt Linux and Windows x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview), built from llama.cpp PR [#31306](https://github.com/ggml-org/llama.cpp/pull/31306) at commit `4838ffb0c8ade92f34d9157deea38d92758ab824`.
+Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.
+## Clover artifacts for this runtime
+| Artifact | Size | Link |
+|---|---|---|
+| BF16 source | 69.38 GB | [Clover-35B-A3B-Preview-BF16-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-BF16-GGUF) |
+| Q8_0 | 36.91 GB | [Clover-35B-A3B-Preview-Q8_0-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF) |
+| 1.25M I-Matrix | 383,689,280 bytes | [Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF) |
+| APEX-I NanoPlus V2.1 | 12.56 GB / 11.70 GiB | [Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF) |
+| APEX-I MiniPlus V2.1 | 14.75 GB / 13.74 GiB | [Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF](https://huggingface.co/IsValorum/Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF) |
+## Files
+| File | Description |
+|---|---|
+| `llama-clover-4838ffb-linux-x86_64-cpu.tar.gz` | Linux CPU build, `GGML_NATIVE=OFF` |
+| `llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz` | CUDA 12 build, archs `80;86;89;90` |
+| `llama-clover-4838ffb-converter-python.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py |
+| `llama-clover-4838ffb-windows-x86_64-nvidia-amd-universal-cuda12.8-vulkan-cpu.zip` | Windows x86_64 universal: Nvidia CUDA 12.8 sm_50-120, AMD/Intel Vulkan, any CPU. Tested with NanoPlus: CPU, CUDA -ngl 8, Vulkan Intel |
+| `BUILD_INFO.json` | Compiler, flags and SHA256 per archive |
+| `SHA256SUMS` | Checksums |
+| `config.json` | Machine-readable build descriptor used by Hub download tracking |
+## Compatibility
+> [!NOTE]
+> **Platform availability.** Prebuilt binaries cover Linux x86_64 (CPU, and NVIDIA CUDA 12 for sm_80/86/89/90) and Windows x86_64 (one universal package: NVIDIA CUDA 12.8 for sm_50-120, AMD and Intel GPUs through Vulkan 1.4.363, and any CPU, with AVX-512 disabled and per-CPU backends loaded at runtime). macOS is not published yet; build from the same branch and commit there.
+- Model: `Diffuspace/Clover-35B-A3B-Preview` (`clover_moe`). Works with the BF16 and Q8_0 GGUFs and with third-party `clover_moe` GGUFs such as the IQ4_XS. It does not load the original safetensors.
+- Linux x86_64: the CPU package runs on any Linux; the CUDA package requires an NVIDIA GPU and a CUDA 12.x runtime, with archs 80/86/89/90 (A100, H100/H200, L40S, RTX 4090, A10G). Blackwell 100/120 is not in this build.
+- glibc: the Linux x86_64 archives require `GLIBC_2.38` or newer. Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build from the branch.
+- Windows x86_64: native, no WSL needed. The universal ZIP carries CUDA 12.8 for sm_50-120, Vulkan 1.4.363 for AMD and Intel, and per-CPU backends picked at runtime (AVX-512 disabled in this build). Extract it and run `set PATH=%~dp0bin;%PATH%` from the archive root. It ships the same tool set as the Linux archives, including `llama-speculative-simple`, `llama-quantize` and `llama-imatrix`.
+- macOS: not published yet; use a Metal build of the PR or the third-party IQ4_XS Mac build.
+- Frontends: not LM Studio or Ollama. Clover needs this `clover_moe` runtime until the PR merges upstream.
+## Usage
+All runtime archives are dynamically linked and ship `llama-cli`, `llama-server`, `llama-diffusion-cli`, `llama-quantize`, `llama-imatrix`, and `llama-perplexity`. Point the loader at the bundled libraries before running. The archive names changed in this build; the extract directories kept their old names.
+CPU:
+```bash
+tar -xzf llama-clover-4838ffb-linux-x86_64-cpu.tar.gz
+cd llama-clover-b4838ffb-linux-x64-cpu
+export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
+./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello"
+```
+CUDA 12:
+```bash
+tar -xzf llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz
+cd llama-clover-b4838ffb-linux-x64-cuda12
+export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
+./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello"
+./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
+```
+Windows x86_64 (universal ZIP, no separate runtime install):
+```bat
+set PATH=%~dp0bin;%PATH%
+llama-completion.exe -m Clover-35B-A3B-Preview.APEX-I-NanoPlus-V2.1.gguf -p "hi" -n 1 -c 128 -b 16 -ub 16 -ngl 8 -no-cnv
+```
+Run both lines from the extracted archive root. Drop `-ngl 8` for CPU or Vulkan-only use; raise it to offload more layers on a large NVIDIA card.
+## Conversion
+```bash
+tar -xzf llama-clover-4838ffb-converter-python.tar.gz
+python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
+```
+## Quantization and importance matrix
+```bash
+./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
+./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
+```
+The APEX-I V2.1 releases were produced with this build: `llama-quantize --imatrix imatrix-1.25M.gguf --tensor-type-file <recipe> clover-bf16.gguf <output> Q4_0 64`. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in `F32`.
+## Notes
+The CUDA packages require a compatible NVIDIA driver and CUDA 12.x runtime; the Windows package bundles its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.