README.md

metadata

title: LlamaCpp WebUI
emoji: 🦙
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 8080
pinned: false
hardware: cpu-basic
models:
  - empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF

llama.cpp WebUI (llama-server)

Runs the official llama.cpp llama serve web UI on a Hugging Face Space (Docker SDK, NVIDIA A10G).

Model

empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF Quantization: Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf (MTP variant for speculative decoding).

How it works

• The prebuilt llama.cpp binary is installed at runtime via the official one-liner curl -LsSf https://llama.app/install.sh | sh. The installer probes the hardware and downloads the CUDA build (an A10G is present at runtime).
• The GGUF model is baked into the image at build time for instant cold starts.
• llama serve is launched with Multi-Token-Prediction speculative decoding:

llama serve \
  -m Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 6 \
  -c 16384 \
  --host 0.0.0.0 \
  --port 8080 \
  -ngl 99

The web UI is served on the Space app port 8080.