# llama.cpp (fork) — Gemma 4 + audio (mtmd) Source snapshot of the private llama.cpp fork used by AGIWS inference workers. ## What is inside - `src/models/gemma4.cpp`, `gemma4-assistant.cpp` — text models `gemma4`, `gemma4-assistant`. - `tools/mtmd/clip.cpp` — graphs `clip_graph_gemma4a` / `clip_graph_gemma4ua` (audio encoders). - `tools/mtmd/mtmd-audio.cpp` — preprocessors `gemma4a` / `gemma4ua`, mel filters, streaming. - `tools/mtmd/mtmd.h` — `mtmd_support_audio`, `mtmd_bitmap_init_from_audio`. - `tools/server` — OpenAI-compatible server accepting `input_audio` content parts. - `conversion/gemma.py` — `Gemma4Model` GGUF converter (`Gemma4ForConditionalGeneration`). ## Files - `src/llama-cpp-src.tar.gz` — full source tree (no `.git`, no build artifacts). - `lib/*.so` — prebuilt CUDA libraries and `llama-server` for T4 (sm_75), uploaded by the build job. - `lib/BUILD_INFO.txt` — compiler, CUDA version, git-less snapshot hash, sha256 of every artifact. ## Build (Colab T4) ```bash tar -xzf src/llama-cpp-src.tar.gz -C /content/llama-cpp-fork cmake -S /content/llama-cpp-fork -B /content/llama-cpp-fork/build \ -DCMAKE_BUILD_TYPE=Release \ -DGGML_CUDA=ON \ -DGGML_NATIVE=OFF \ -DCMAKE_CUDA_ARCHITECTURES=75 \ -DBUILD_SHARED_LIBS=ON \ -DLLAMA_CURL=OFF \ -DLLAMA_BUILD_TESTS=OFF \ -DLLAMA_BUILD_EXAMPLES=OFF cmake --build /content/llama-cpp-fork/build -j"$(nproc)" --target llama-server ``` Artifacts needed at runtime: `llama-server` plus `libllama.so*`, `libmtmd.so*`, `libggml*.so*` (`libggml-base`, `libggml-cpu`, `libggml-cuda`, `libggml`). Runtime flags for Gemma audio: ```bash LD_LIBRARY_PATH=/content/llama/lib \ /content/llama/lib/llama-server -m .../gemma-4-12b-it-Q4_K_M.gguf \ --mmproj .../mmproj-F16.gguf -ngl 99 -c 4096 --host 127.0.0.1 --port 8091 ```