# Patched llama.cpp runtime

Stock llama.cpp does not support the `t5gemma2` reranker architecture in
this release. Apply all seven patches in lexical order to the exact base.

- upstream: `https://github.com/ggml-org/llama.cpp`
- base commit: `277a105dc8f8643dab54331926a9830860a03292`
- tested fork commit: `8c099e4eb6c79e5d2587c8205ee9971564c740cc`
- expected patched tree: `253695d8b0ca0723742c0109806a831a968cdffd`
- executable: `llama-kalm-reranker`

```bash
git clone https://github.com/ggml-org/llama.cpp llama.cpp-src
git -C llama.cpp-src checkout 277a105dc8f8643dab54331926a9830860a03292
bash ./llama.cpp/apply-patches.sh "$PWD/llama.cpp-src"
cmake -S llama.cpp-src -B llama.cpp-src/build-cuda -G Ninja \
  -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build llama.cpp-src/build-cuda \
  --target llama-kalm-reranker test-t5gemma2-load test-llama-archs -j
```

For CPU, configure with `-DGGML_CUDA=OFF`. The custom CLI requires a local
model path:

```bash
llama.cpp-src/build-cuda/bin/llama-kalm-reranker \
  -m kalm-reranker-v1-small-q4_k_m.gguf -ngl 99 --require-gpu \
  --query "What is the capital of China?" \
  --passage "The capital of China is Beijing."
```