title: README
emoji: 🧊
colorFrom: gray
colorTo: green
sdk: static
pinned: false
Glyd
Lossless AI compression: 33% less GPU memory, bit for bit.
Glyd keeps a bf16 model's weights and KV cache in fewer bits in GPU memory and rebuilds the exact values as the model runs. It is the same model in about a third less memory, so it fits a smaller GPU or serves more requests on the same one.
Levels
Each model here comes at a level, the tag after its name in glyd run, and its repo is glyd/<model>-<level>:
| Level | What it keeps |
|---|---|
| penguin | lossless: every weight decodes to its bf16 value (the default) |
| kestrel | Glyd's own quantization, about 6.5 bits per weight |
| swift | Glyd's own quantization, about 5.5 bits per weight |
curl -LsSf https://getglyd.com/install.sh | sh
glyd run Qwen/Qwen3.8-27B:swift
pip install "glyd[vllm]"
vllm serve Qwen/Qwen3.5-9B --quantization glyd
import glyd
model = glyd.from_pretrained("Qwen/Qwen3.5-9B")
Linux (x86_64 or aarch64) with an NVIDIA GPU: Ampere, Ada, Hopper or Blackwell, driver 580 or newer.
| Measured | bf16 | Glyd |
|---|---|---|
| Qwen3.5-9B under vLLM on an RTX 4090: weights | 17.66 GiB | 12.68 GiB (−28.2%) |
| Qwen3.5-9B on an RTX 4090 (24 GB): context | out of memory at start | 189,440 tokens |
| Qwen3.8-27B under vLLM on an H100: weights | 50.22 GiB | 38.77 GiB (−22.8%) |
| Qwen3.8-27B: its layers' matrices, read back bit for bit | 49.52 GB | 33.28 GB (−32.8%) |
Every result and its logs: getglyd.com/benchmarks.
License: BUSL-1.1. Free for personal and non-commercial use on your own computers; commercial use needs a license; each version becomes Apache-2.0 four years after release. Contact: suryakoritala@getglyd.com.