| --- |
| license: mit |
| tags: |
| - llama.cpp |
| - llama-server |
| - macos |
| - apple-silicon |
| - multimodal |
| - batiai |
| library_name: llama.cpp |
| --- |
| |
| # llama.cpp `llama-server` — macOS (Apple Silicon) static builds, by BatiAI |
|
|
| Self-contained **`llama-server`** binaries for macOS, built so that [BatiFlow](https://flow.bati.ai) can run **on-device multimodal** (image + audio) models like [Gemma 4 12B](https://huggingface.co/batiai/gemma-4-12B-it-GGUF) that **Ollama can't serve yet**. |
|
|
| > Ollama 0.20 doesn't know the Gemma 4 `gemma4uv` / `gemma4ua` vision/audio projectors. `llama-server` from recent llama.cpp master does. These are the macOS binaries that make that work locally. |
|
|
| ## Status |
| ✅ **Live** (2026-06-06) — built on Apple Silicon, verified (`/props` → vision+audio, image OCR), zero external deps. **arm64 only** (universal2 not needed — all targets are Apple Silicon). |
|
|
| ## Files |
| | File | Size | What | |
| |------|------|------| |
| | `llama-server-macos-arm64` | 20 MB | static `llama-server` (Apple Silicon, Metal) — BatiFlow sidecar | |
| | `llama-mtmd-cli-macos-arm64` | 9.8 MB | static multimodal CLI (fallback) | |
| | `BUILD-INFO.txt` | — | commit hash + sha256 + cmake flags | |
|
|
| - **llama.cpp commit**: `308f61c31f083251ce8150f10b9ef97679b500b5` |
| - **sha256 (server)**: `75cb1b97dff5de3ed3dbc3b79bbbda3cc33cb20f2e9287c3d0b685c46ad4c7db` |
| - **deps**: macOS base frameworks only (Metal/Accelerate/Foundation/libSystem/libc++/libobjc) — no homebrew/openssl/dylib |
|
|
| ## Build pin (reproducible) |
| ```bash |
| git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp |
| git checkout 308f61c # gemma4 projector merged (gemma4v/uv/a/ua clip graphs) |
| cmake -B build -DGGML_METAL=ON -DLLAMA_CURL=OFF \ |
| -DBUILD_SHARED_LIBS=OFF \ # static single file (no dylib deps) |
| -DLLAMA_OPENSSL=OFF \ # drop homebrew openssl@3 dep (crashes on Macs without homebrew) |
| -DGGML_METAL_EMBED_LIBRARY=ON \ # embed Metal shaders (no separate .metal file → true single binary) |
| -DCMAKE_BUILD_TYPE=Release |
| cmake --build build --target llama-server llama-mtmd-cli -j |
| # static check: `otool -L build/bin/llama-server | grep -iE 'libllama|libggml|libmtmd|openssl|ssl'` → empty |
| ``` |
|
|
| ## Usage (Gemma 4 12B multimodal) |
| ```bash |
| llama-server-macos-arm64 \ |
| -m gemma-4-12B-it-Q4_K_M.gguf \ |
| --mmproj mmproj-google-gemma-4-12B-it-BF16.gguf \ |
| --host 127.0.0.1 --port 8899 -ngl 99 -c 8192 --jinja |
| # GET /props → {"vision": true, "audio": true} |
| # POST /v1/chat/completions with image_url / input_audio (OpenAI-compatible) |
| ``` |
| GGUF + mmproj: [batiai/gemma-4-12B-it-GGUF](https://huggingface.co/batiai/gemma-4-12B-it-GGUF) (filenames are pinned/stable). |
|
|
| ## Scope |
| - ✅ Image (OCR, visual reasoning) · ✅ Audio (speech, experimental) · ❌ Video (no gemma4 video projector in llama.cpp yet) |
| - `--jinja` required. Gemma 4 is a reasoning model → give generous `max_tokens` (answer in `reasoning_content` + `content`). |
| - Model-agnostic: this `llama-server` works with any GGUF, not only Gemma 4. |
|
|
| Built and hosted by [BatiAI](https://bati.ai) for [BatiFlow](https://flow.bati.ai). |
|
|
