README.md

---
title: LlamaCpp WebUI
emoji: "\U0001F999"
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 8080
pinned: false
hardware: cpu-basic
models:
  - empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF
---

# llama.cpp WebUI (llama-server)

Runs the official **llama.cpp** `llama serve` web UI on a Hugging Face Space (Docker SDK, NVIDIA A10G).

## Model
[`empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF`](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF)
Quantization: `Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf` (MTP variant for speculative decoding).

## How it works
- The prebuilt `llama.cpp` binary is installed at **runtime** via the official one-liner
  `curl -LsSf https://llama.app/install.sh | sh`. The installer probes the hardware and
  downloads the CUDA build (an A10G is present at runtime).
- The GGUF model is baked into the image at **build time** for instant cold starts.
- `llama serve` is launched with Multi-Token-Prediction speculative decoding:

```bash
llama serve \
  -m Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 6 \
  -c 16384 \
  --host 0.0.0.0 \
  --port 8080 \
  -ngl 99
```

The web UI is served on the Space app port `8080`.