---

license: mit

widget:

• text: >-

fox

output:

url: fox.png

• text: >-

cow

output:

url: cow.png

• text: >-

dog

output:

url: dog.png

---

ggk

One package for working with GGUF models locally: an OpenAI-compatible LLM

server, a diffusion image/video/audio generator and a GGUF metadata/tensor

editor with a built-in quantizer — three panels on one GUI, powered by one

unified engine compiled in a single build on top of gk, an independent

tensor library. There is no ggml anywhere in the tree.

Install

pip install ggk

The build compiles the bundled engine (CPU by default, Metal on macOS).

GPU backends are opt-in at install time:

GGK_CUDA=1 pip install ggk     # NVIDIA
GGK_HIP=1 pip install ggk      # AMD ROCm
GGK_VULKAN=1 pip install ggk   # Vulkan

Each switch drives the whole engine — the server, the diffusion runtime and

the multimodal projectors all evaluate their graphs on the one gk build.

Run

ggk                 # unified GUI — Server / Diffuser / Editor panels
python -m ggk       # same thing

Each panel also runs on its own, exactly like the standalone

gguf-server / gguf-diffusion / gguf-editor packages did:

ggk server          # LLM server GUI
ggk diffuser        # image generation GUI
ggk editor          # GGUF editor GUI

And the engines are directly scriptable from the CLI:

ggk server engine -- --model model.gguf
ggk diffuser engine -- -m sd.gguf -p "a lighthouse at dusk" -o out.png
ggk editor quantize -m in.gguf -o out-q4_k.gguf --type q4_k
ggk editor devices

For examples, test the diffusion model in this repo:

ggk diffuser engine -- -m test-nvfp4.gguf -p "fox" -o fox.png
ggk diffuser engine -- -m test-nvfp4.gguf -p "cow" -o cow.png
ggk diffuser engine -- -m test-nvfp4.gguf -p "dog" -o dog.png
ggk editor test-nvfp4.gguf

add --diffusion-fa tag (turn on flash attention for diffusion model) to diffuser engine will get significantly faster process

<Gallery />

*gk is our own experimental kernel, recently support multiGPU tensor split,

new features are coming very soon, please see reference for details

Layout

vendor/engine/           the unified ggk engine (one CMake build)
  gk/                    the gk compute kernels (CPU + optional GPU backends)
  gk/compat/             the historical ggml C API, implemented on gk
  src/ common/ mtmd/     GGUF LLM runtime
  app/                   the gguf-server HTTP server
  diffusion/             diffusion runtime + CLI
  quantizer/             quantizer shared library (its own quant kernels)
src/ggk/                 the Python package
  server/ diffuser/ editor/   the three panels (backend + web frontend each)
  gui.py static/         the unified 3-panel GUI shell

Nothing above gk/compat/ knows gk exists: the runtimes include the same

ggml.h / ggml-backend.h / gguf.h headers and call the same functions

they always did, while graph building, allocation, scheduling and the kernels

themselves are gk's. See vendor/engine/README.md for the engine's own build

options.

The editor's quantizer stays independent — its qz_* codec is compiled both

into the quantizer library the editor drives and into gk itself, so the

encoder and the runtimes' decoder can never disagree about a GGUF block.

!screenshot

Reference

pig engine - the new gguf compute kernels (gk)