LLamaOptimizerAuto/README.md

---
license: mit
---

# LLamaOptimizerAuto

LLama.cpp optimizer toolkit with CUDA support for Windows.

## Contents

This repository contains pre-built llama.cpp binaries with GPU (CUDA 12) and multi-CPU architecture support.

### Core Binaries

| File | Description |
|------|-------------|
| `llama-server.exe` | OpenAI-compatible API server |
| `llama-cli.exe` | Command-line inference |
| `llama-bench.exe` | Benchmarking tool |
| `llama-quantize.exe` | Model quantization |
| `llama-imatrix.exe` | Importance matrix computation |
| `llama-gguf-split.exe` | GGUF file splitting/joining |
| `llama-perplexity.exe` | Perplexity calculation |
| `llama-tokenize.exe` | Tokenization utility |
| `llama-completion.exe` | Text completion |
| `llama-batched-bench.exe` | Batched benchmarking |
| `llama-llava-cli.exe` | LLaVA multimodal CLI |
| `llama-minicpmv-cli.exe` | MiniCPM-V multimodal CLI |
| `llama-mtmd-cli.exe` | Multimodal CLI |
| `llama-qwen2vl-cli.exe` | Qwen2-VL CLI |
| `llama-gemma3-cli.exe` | Gemma 3 CLI |
| `llama-tts.exe` | Text-to-speech |
| `llama-results.exe` | Results utility |
| `llama-template-analysis.exe` | Template analysis |
| `llama-fit-params.exe` | Parameter fitting |
| `llama-mtmd-debug.exe` | Multimodal debug |
| `rpc-server.exe` | RPC server for distributed inference |

### GPU Support

- `ggml-cuda.dll` — CUDA 12 backend
- `cublas64_12.dll`, `cublasLt64_12.dll`, `cudart64_12.dll` — CUDA 12 libraries

### CPU Architecture Support

| DLL | Architecture |
|-----|-------------|
| `ggml-cpu-x64.dll` | Generic x86-64 |
| `ggml-cpu-sse42.dll` | SSE 4.2 |
| `ggml-cpu-ivybridge.dll` | Ivy Bridge |
| `ggml-cpu-sandybridge.dll` | Sandy Bridge |
| `ggml-cpu-haswell.dll` | Haswell |
| `ggml-cpu-skylakex.dll` | Skylake-X |
| `ggml-cpu-cascadelake.dll` | Cascade Lake |
| `ggml-cpu-cooperlake.dll` | Cooper Lake |
| `ggml-cpu-icelake.dll` | Ice Lake |
| `ggml-cpu-cannonlake.dll` | Cannon Lake |
| `ggml-cpu-alderlake.dll` | Alder Lake |
| `ggml-cpu-sapphirerapids.dll` | Sapphire Rapids |
| `ggml-cpu-piledriver.dll` | AMD Piledriver |
| `ggml-cpu-zen4.dll` | AMD Zen 4 |

### Optimizer Tools

- `LLamaOptimizer.exe` — Auto optimizer for llama.cpp configurations
- `LlamaOptGUI(1).exe` — GUI version of the optimizer

## Quick Start

1. Download the files
2. Run `llama-server.exe` with your GGUF model:

```bash
llama-server.exe -m your_model.gguf -ngl 99 --host 0.0.0.0 --port 8080
```

3. Access the API at `http://localhost:8080`

## Requirements

- Windows x64
- NVIDIA GPU with CUDA 12 support (for GPU acceleration)
- Sufficient RAM/VRAM for your model