Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,43 @@
+---
+license: mit
+tags: [llama.cpp, clover, clover_moe, prebuilt, cuda]
+---
+# llama.cpp with Clover support (clover_moe)
+
+Prebuilt Linux x86_64 binaries for [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview), built from llama.cpp PR [#31306](https://github.com/ggml-org/llama.cpp/pull/31306) at commit `4838ffb0c8ade92f34d9157deea38d92758ab824`.
+
+Mainline llama.cpp does not load `clover_moe`. This build adds the converter, the runtime graph and the `draft-self` speculative mode. The PR is not merged upstream; pin this repository for reproducibility.
+
+| File | Description |
+|---|---|
+| `llama-clover-b4838ffb-linux-x64-cpu.tar.gz` | Portable CPU build |
+| `llama-clover-b4838ffb-linux-x64-cuda12.tar.gz` | CUDA build, archs `80;86;89;90` |
+| `llama-clover-b4838ffb-converter.tar.gz` | HF-to-GGUF converter, conversion code and gguf-py |
+| `BUILD_INFO.json` | Compiler, flags and SHA256 |
+| `SHA256SUMS` | Checksums |
+
+## Usage
+
+```bash
+tar -xzf llama-clover-b4838ffb-linux-x64-cuda12.tar.gz
+cd llama-clover-b4838ffb-linux-x64-cuda12
+export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
+./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello"
+./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99
+```
+
+Conversion:
+
+```bash
+tar -xzf llama-clover-b4838ffb-converter.tar.gz
+python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf
+```
+
+Quantization and importance matrix:
+
+```bash
+./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
+./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99
+```
+
+The CUDA package requires a compatible NVIDIA driver and CUDA 12.x runtime. The CPU package is built with `GGML_NATIVE=OFF`. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.