llama.cpp/README.md

# Patched llama.cpp runtime

The GGUF in this repository is not compatible with stock llama.cpp. Apply
the seven patches in numeric order to the exact upstream base:

- repository: `https://github.com/ggml-org/llama.cpp`
- base commit: `277a105dc8f8643dab54331926a9830860a03292`
- tested fork commit: `8c099e4eb6c79e5d2587c8205ee9971564c740cc`
- expected patched tree: `253695d8b0ca0723742c0109806a831a968cdffd`
- scoring executable: `llama-kalm-reranker`

```bash
git clone https://github.com/ggml-org/llama.cpp llama.cpp-src
git -C llama.cpp-src checkout 277a105dc8f8643dab54331926a9830860a03292
bash ./llama.cpp/apply-patches.sh "$PWD/llama.cpp-src"
```

CUDA build:

```bash
cmake -S llama.cpp-src -B llama.cpp-src/build-cuda -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_CUDA=ON
cmake --build llama.cpp-src/build-cuda \
  --target llama-kalm-reranker test-t5gemma2-load test-llama-archs -j
```

CPU build:

```bash
cmake -S llama.cpp-src -B llama.cpp-src/build-cpu -G Ninja \
  -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=OFF
cmake --build llama.cpp-src/build-cpu \
  --target llama-kalm-reranker test-t5gemma2-load test-llama-archs -j
```

The complete patch series is intentionally distributed, including the
converter patch, so users reproduce the exact tested source tree. There is
no second squashed patch to keep in sync.

The custom CLI requires a local model path:

```bash
llama.cpp-src/build-cuda/bin/llama-kalm-reranker \
  -m kalm-reranker-v1-small-q8_0.gguf -ngl 99 \
  --query "What is the capital of China?" \
  --passage "The capital of China is Beijing."
```