---
title: README
emoji: 🔧
colorFrom: indigo
colorTo: purple
sdk: static
pinned: false
---

# SlyBase

**Fast LLM inference on consumer AMD GPUs.**

We run large models on a single AMD Radeon AI PRO R9700 (RDNA4, 32 GB) and publish what it takes to make
them fast: kernels, serving configs and ready-to-run quantizations — measured, not guessed.

### 🚀 [vllm-sly-radiance](https://github.com/SlyBase/vllm-sly-radiance)

A vLLM fork for RDNA4 (gfx1201). Hand-written W4A8 MXFP4 kernels on the fp8 matrix cores, fused
norm/quant, tuned decode and drafter attention, DFlash2 speculative decoding and int4 heads — Qwen3.8-27B at
~130 tokens/s decode on one card, 262k context. One Docker image, every change gated on a benchmark and a
quality check.

### 📦 Models

Quantizations built for hardware with native MXFP4 (RDNA4, AMD Instinct MI350), each with a model card
that says what it costs in quality and speed — including when you are better off with another build.

- [ThinkingCap-Qwen3.8-27B-MXFP4](https://huggingface.co/slybase/ThinkingCap-Qwen3.8-27B-MXFP4) —
  BottleCap AI's thinking-efficient Qwen3.8-27B finetune, GPTQ-quantized to MXFP4.

### 📏 How we measure

Decode by step time and speculative acceptance (not single noisy runs), quality by deterministic
prompt-logprob comparison plus GSM8K, all on the same card and power cap. Numbers and scripts live in the
fork's [docs](https://github.com/SlyBase/vllm-sly-radiance/tree/main/docs).

---

Credit where it's due: this work stands on [vLLM](https://github.com/vllm-project/vllm),
[StillDeadcode/vllm-radiance](https://github.com/StillDeadcode/vllm-radiance) and ggz14's RDNA4 kernels.
Model weights belong to their authors and keep their licenses.