---
license: mit
base_model: unsloth/DeepSeek-R1-Distill-Qwen-14B
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
  - the-rock8
  - rock8
  - rdna4
  - gfx1201
  - radeon-ai-pro-r9700
  - rx-9070
  - fp8
  - f8e4m3
  - quark
  - gguf
  - amd
  - rocm
  - llama.cpp
  - quantized
  - reasoning
  - deepseek-r1
model_name: Quacken-R1-14B-FP8
quantized_by: The Rock8
---

# Quacken-R1-14B-FP8
### The Rock8 - Got any weights? 💪🦆

Native **fp8 E4M3** GGUF of **DeepSeek-R1-Distill-Qwen-14B** (a reasoning model) for
**AMD RDNA4** (gfx1201 - Radeon AI PRO R9700 / RX 9070 / 9070 XT / W-series),
quantized with **AMD Quark** from the full-precision BF16 weights by
[The Rock8](https://github.com/The-Monk/The-Rock8).

The Rock8's llama.cpp fork runs this fp8 on RDNA4's **native WMMA fp8 tensor cores**
(prefill) and `v_dot4_f32_fp8_fp8` (decode) - not a dequant-to-f16 fallback.

## What it is
- **Format:** fp8 E4M3 (`F8E4M3`), block-scaled, produced by AMD Quark from BF16.
- **Target:** AMD RDNA4 / gfx1201 (Radeon AI PRO R9700, RX 9070 / 9070 XT).
- **Runtime:** The Rock8 (llama.cpp fork with native RDNA4 fp8 kernels) on TheRock ROCm 7.13.
- **File:** `DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf` (16 GB).

## Reasoning model - usage note
This is an R1-distill reasoning model. It emits a chain-of-thought inside
`<think>...</think>` before the final answer; llama.cpp / OpenAI-compatible servers
surface that as `reasoning_content`. To **disable** thinking for a turn, append
`/no_think` to the prompt (or set the chat template's thinking flag off). Expect
longer generations by default because of the reasoning trace.

## Source model + license
- **Source:** [unsloth/DeepSeek-R1-Distill-Qwen-14B](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B)
  (a mirror of deepseek-ai/DeepSeek-R1-Distill-Qwen-14B).
- **License:** **MIT** (inherited from the source model; redistribution of this
  quantized derivative is permitted with attribution). This is a derivative work.

## Validation (real gfx1201 hardware)
| Metric | Value |
|---|---:|
| Perplexity (wikitext, 20 chunks, n_ctx=512) | **8.97** |
| Prefill `pp512` | **2499 t/s** |
| Decode `tg128` | **33.4 t/s** |

Benched on a single R9700 (gfx1201).

## Run it

### llama.cpp (The Rock8 fork)
```bash
# reasoning chat (keeps <think>)
llama-cli   -m DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -ngl 99 \
  -p "Solve step by step: a train travels 60 km in 40 minutes. What is its speed in km/h?"
# fast, no reasoning trace
llama-cli   -m DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -ngl 99 -p "What do you call a dried grape? Answer in one word. /no_think"
# bench
llama-bench -m DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -ngl 99 -p 512 -n 128
```

### Lemonade appliance (container)
```bash
podman run -d --rm --runtime crun --name lemonade \
  --device /dev/kfd --device /dev/dri \
  --group-add keep-groups --security-opt seccomp=unconfined \
  -v /path/to/quacken-r1-14b:/models:ro \
  -e MODEL=/models/DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -e MODEL_NAME=Quacken-R1-14B-FP8 \
  -e HIP_VISIBLE_DEVICES=0 -p 13305:13305 \
  ghcr.io/the-monk/the-rock8:rdna4-tr713 serve
```
Container (same image on each registry; `--runtime crun` is required for GPU):
`ghcr.io/the-monk/the-rock8:rdna4-tr713` - `docker.io/gorilla4x/the-rock8:rdna4-tr713` - `quay.io/the-monk/the-rock8:rdna4-tr713`
(images may not be pushed to every registry yet).

## The Rock8 - RDNA4 fp8 (links)
- **GitHub:** [The-Rock8](https://github.com/The-Monk/The-Rock8) - kernels, patch series, appliance recipe, full feature doc.
- **Collection:** [The Rock8 - RDNA4 fp8](https://huggingface.co/collections/Gorilla4X/the-rock8-rdna4-fp8-6a547070f667cb41db0bc2ed).
- **Sibling model:**
  [Quacken-8B-FP8](https://huggingface.co/Gorilla4X/Quacken-8B-FP8).
- **3 more coming:** Quacken-27B-FP8 (Qwen3.6-27B), Qwen3.6-35B-A3B (MoE) fp8, and Ornith-1.0-35B fp8 (in final validation).

*Every artifact links to the others - land on any one, reach them all.*