ggk

One package for working with GGUF models locally: an OpenAI-compatible LLM server, a diffusion image/video/audio generator and a GGUF metadata/tensor editor with a built-in quantizer — three panels on one GUI, powered by one unified engine compiled in a single build on top of gk, an independent tensor library. There is no ggml anywhere in the tree.

Install

pip install ggk

The build compiles the bundled engine (CPU by default, Metal on macOS). GPU backends are opt-in at install time:

GGK_CUDA=1 pip install ggk     # NVIDIA
GGK_HIP=1 pip install ggk      # AMD ROCm
GGK_VULKAN=1 pip install ggk   # Vulkan

Each switch drives the whole engine — the server, the diffusion runtime and the multimodal projectors all evaluate their graphs on the one gk build.

Run

ggk                 # unified GUI — Server / Diffuser / Editor panels
python -m ggk       # same thing

Each panel also runs on its own, exactly like the standalone gguf-server / gguf-diffusion / gguf-editor packages did:

ggk server          # LLM server GUI
ggk diffuser        # image generation GUI
ggk editor          # GGUF editor GUI

And the engines are directly scriptable from the CLI:

ggk server engine -- --model model.gguf
ggk diffuser engine -- -m sd.gguf -p "a lighthouse at dusk" -o out.png
ggk editor quantize -m in.gguf -o out-q4_k.gguf --type q4_k
ggk editor devices

For examples, test the diffusion model in this repo:

ggk diffuser engine -- -m test-nvfp4.gguf -p "fox" -o fox.png
ggk diffuser engine -- -m test-nvfp4.gguf -p "cow" -o cow.png
ggk diffuser engine -- -m test-nvfp4.gguf -p "dog" -o dog.png
ggk editor test-nvfp4.gguf

add --diffusion-fa tag (turn on flash attention for diffusion model) to diffuser engine will get significantly faster process

Prompt

fox

Prompt

cow

Prompt

dog

*gk is our own experimental kernel, recently support multiGPU tensor split, new features are coming very soon, please see reference for details

Layout

vendor/engine/           the unified ggk engine (one CMake build)
  gk/                    the gk compute kernels (CPU + optional GPU backends)
  gk/compat/             the historical ggml C API, implemented on gk
  src/ common/ mtmd/     GGUF LLM runtime
  app/                   the gguf-server HTTP server
  diffusion/             diffusion runtime + CLI
  quantizer/             quantizer shared library (its own quant kernels)
src/ggk/                 the Python package
  server/ diffuser/ editor/   the three panels (backend + web frontend each)
  gui.py static/         the unified 3-panel GUI shell

Nothing above gk/compat/ knows gk exists: the runtimes include the same ggml.h / ggml-backend.h / gguf.h headers and call the same functions they always did, while graph building, allocation, scheduling and the kernels themselves are gk's. See vendor/engine/README.md for the engine's own build options.

The editor's quantizer stays independent — its qz_* codec is compiled both into the quantizer library the editor drives and into gk itself, so the encoder and the runtimes' decoder can never disagree about a GGUF block.

Reference

pig engine - the new gguf compute kernels (gk)