---
license: mit
---
# gguf-cpp

One package for working with GGUF models locally: an OpenAI-compatible LLM
server, a diffusion image/video generator and a GGUF metadata/tensor editor
with a built-in quantizer — three panels on one GUI, powered by one unified
**gguf.cpp** engine compiled in a single build with a shared set of ggml
kernels.

## Install

```bash
pip install gguf-cpp
```

The build compiles the bundled engine (CPU by default, Metal on macOS).
GPU backends are opt-in at install time:

```bash
GGUF_CPP_CUDA=1 pip install gguf-cpp     # NVIDIA
GGUF_CPP_HIP=1 pip install gguf-cpp      # AMD ROCm
GGUF_CPP_VULKAN=1 pip install gguf-cpp   # Vulkan
```

## Run

```bash
gguf-cpp                 # unified GUI — Server / Diffuser / Editor panels
python -m gguf_cpp       # same thing
```

![screenshot](https://raw.githubusercontent.com/gguf-org/gguf-desktop/master/demo15.gif)

Each panel also runs on its own, exactly like the standalone
gguf-server / gguf-diffusion / gguf-editor packages did:

```bash
gguf-cpp server          # LLM server GUI
gguf-cpp diffuser        # image/video generation GUI
gguf-cpp editor          # GGUF editor GUI
```

And the engines are directly scriptable from the CLI:

```bash
gguf-cpp server engine -- --model model.gguf --port 8888
gguf-cpp diffuser engine -- -m sd.gguf -p "a lighthouse at dusk" -o out.png
gguf-cpp editor quantize -m in.gguf -o out-q4_k.gguf --type q4_k
gguf-cpp editor devices
```

or run it with `gguf-connector`
```
ggc gp
```

![screenshot](https://raw.githubusercontent.com/gguf-org/gguf-desktop/master/pizza.jpg)