LLamaOptimizerAuto/README.md

metadata

license: mit

LLamaOptimizerAuto

LLama.cpp optimizer toolkit with CUDA support for Windows.

Contents

This repository contains pre-built llama.cpp binaries with GPU (CUDA 12) and multi-CPU architecture support.

Core Binaries

File	Description
llama-server.exe	OpenAI-compatible API server
llama-cli.exe	Command-line inference
llama-bench.exe	Benchmarking tool
llama-quantize.exe	Model quantization
llama-imatrix.exe	Importance matrix computation
llama-gguf-split.exe	GGUF file splitting/joining
llama-perplexity.exe	Perplexity calculation
llama-tokenize.exe	Tokenization utility
llama-completion.exe	Text completion
llama-batched-bench.exe	Batched benchmarking
llama-llava-cli.exe	LLaVA multimodal CLI
llama-minicpmv-cli.exe	MiniCPM-V multimodal CLI
llama-mtmd-cli.exe	Multimodal CLI
llama-qwen2vl-cli.exe	Qwen2-VL CLI
llama-gemma3-cli.exe	Gemma 3 CLI
llama-tts.exe	Text-to-speech
llama-results.exe	Results utility
llama-template-analysis.exe	Template analysis
llama-fit-params.exe	Parameter fitting
llama-mtmd-debug.exe	Multimodal debug
rpc-server.exe	RPC server for distributed inference

GPU Support

• ggml-cuda.dll — CUDA 12 backend
• cublas64_12.dll, cublasLt64_12.dll, cudart64_12.dll — CUDA 12 libraries

CPU Architecture Support

DLL	Architecture
ggml-cpu-x64.dll	Generic x86-64
ggml-cpu-sse42.dll	SSE 4.2
ggml-cpu-ivybridge.dll	Ivy Bridge
ggml-cpu-sandybridge.dll	Sandy Bridge
ggml-cpu-haswell.dll	Haswell
ggml-cpu-skylakex.dll	Skylake-X
ggml-cpu-cascadelake.dll	Cascade Lake
ggml-cpu-cooperlake.dll	Cooper Lake
ggml-cpu-icelake.dll	Ice Lake
ggml-cpu-cannonlake.dll	Cannon Lake
ggml-cpu-alderlake.dll	Alder Lake
ggml-cpu-sapphirerapids.dll	Sapphire Rapids
ggml-cpu-piledriver.dll	AMD Piledriver
ggml-cpu-zen4.dll	AMD Zen 4

Optimizer Tools

• LLamaOptimizer.exe — Auto optimizer for llama.cpp configurations
• LlamaOptGUI(1).exe — GUI version of the optimizer

Quick Start

1. Download the files
2. Run llama-server.exe with your GGUF model:

llama-server.exe -m your_model.gguf -ngl 99 --host 0.0.0.0 --port 8080

3. Access the API at http://localhost:8080

Requirements

• Windows x64
• NVIDIA GPU with CUDA 12 support (for GPU acceleration)
• Sufficient RAM/VRAM for your model