---
license: other
license_name: fraqtl-verification
license_link: LICENSE
tags:
  - kv-cache
  - kv-cache-compression
  - vllm
  - a100
  - fraqtl
inference: false
---

# fraQtl SM80 runtime — prebuilt verification wheel

Prebuilt binary wheel of the `fraqtl` vLLM runtime, published so anyone can independently reproduce the fraQtl KV-cache receipts on a rented A100 for ≈$15 — **without the kernel or calibration source being distributed**.

## What's inside

- The fraQtl CUDA kernel as a compiled **SM80 (A100) binary** (`fraqtl/_native/libggml-cuda.so`). No `.cu`/`.c`/header source ships; the build hard-fails if any leaks in (see `wheel_build_receipt.json`).
- The vLLM integration layer (attention backend, model adapters, sidecar readers) as Python source — it must stay `torch.compile`-traceable or the engine provisions a smaller KV pool than the receipts. It is orchestration glue: how eigenbases are computed is not in the wheel in any form (no calibration or sidecar-factory code; remaining runtime modules are Cython-compiled binaries).
- vLLM plugin auto-registration via the `vllm.general_plugins` entry point — `pip install` the wheel and vLLM's plugin loader finds the backend in every worker process.

## Scope

- **Hardware: A100 (SM80) only.** The binary will not run on H100, L4, or consumer GPUs.
- **vLLM 0.20.2, CUDA 12.4, Python 3.11** — the exact receipt environment. Other versions are unsupported.
- Purpose: verification of the published receipts. It is not a general-purpose release.

## Use it

Don't drive this by hand — use the one-command repro script (`fraqtl_repro_receipts.py`, in this repo and in the sidecar repos):

```bash
pip install modal && modal setup
modal secret create huggingface HF_TOKEN=hf_...
modal run fraqtl_repro_receipts.py
```

It installs this wheel, pulls the calibrated sidecars from
[`fraQtl/mistral-7b-instruct-v0.3-kv-sidecars`](https://huggingface.co/fraQtl/mistral-7b-instruct-v0.3-kv-sidecars),
runs the three-arm receipt (fraQtl D2 vs fp16 vs fp8-KV) at 8K and 32K with a
needle-in-a-haystack gate per arm, and prints the table.

### Honesty footnotes

- **NIAH-verified, not "lossless."**
- **vs-fp8 always visible** — fp8 KV is the real competitor.
- **Batch-1 decode** for the published speed rows.

## Files

| File | What |
|---|---|
| `fraqtl_runtime-*.whl` | The prebuilt wheel (binary-only) |
| `wheel_build_receipt.json` | Build provenance: sha256, source-leak audit result, module list |
| `fraqtl_repro_receipts.py` | The one-command repro script |

## License

Proprietary; free to install and run for verification and evaluation. No reverse engineering. Sidecars and receipts in the companion repos are Apache 2.0.