model card

README.md

@@ -0,0 +1,101 @@
+---
+license: mit
+base_model: unsloth/DeepSeek-R1-Distill-Qwen-14B
+base_model_relation: quantized
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+  - the-rock8
+  - rock8
+  - rdna4
+  - gfx1201
+  - radeon-ai-pro-r9700
+  - rx-9070
+  - fp8
+  - f8e4m3
+  - quark
+  - gguf
+  - amd
+  - rocm
+  - llama.cpp
+  - quantized
+  - reasoning
+  - deepseek-r1
+model_name: Quacken-R1-14B-FP8
+quantized_by: The Rock8
+---
+
+# Quacken-R1-14B-FP8
+### The Rock8 - Got any weights? 💪🦆
+
+Native **fp8 E4M3** GGUF of **DeepSeek-R1-Distill-Qwen-14B** (a reasoning model) for
+**AMD RDNA4** (gfx1201 - Radeon AI PRO R9700 / RX 9070 / 9070 XT / W-series),
+quantized with **AMD Quark** from the full-precision BF16 weights by
+[The Rock8](https://github.com/The-Monk/The-Rock8).
+
+The Rock8's llama.cpp fork runs this fp8 on RDNA4's **native WMMA fp8 tensor cores**
+(prefill) and `v_dot4_f32_fp8_fp8` (decode) - not a dequant-to-f16 fallback.
+
+## What it is
+- **Format:** fp8 E4M3 (`F8E4M3`), block-scaled, produced by AMD Quark from BF16.
+- **Target:** AMD RDNA4 / gfx1201 (Radeon AI PRO R9700, RX 9070 / 9070 XT).
+- **Runtime:** The Rock8 (llama.cpp fork with native RDNA4 fp8 kernels) on TheRock ROCm 7.13.
+- **File:** `DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf` (16 GB).
+
+## Reasoning model - usage note
+This is an R1-distill reasoning model. It emits a chain-of-thought inside
+`<think>...</think>` before the final answer; llama.cpp / OpenAI-compatible servers
+surface that as `reasoning_content`. To **disable** thinking for a turn, append
+`/no_think` to the prompt (or set the chat template's thinking flag off). Expect
+longer generations by default because of the reasoning trace.
+
+## Source model + license
+- **Source:** [unsloth/DeepSeek-R1-Distill-Qwen-14B](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B)
+  (a mirror of deepseek-ai/DeepSeek-R1-Distill-Qwen-14B).
+- **License:** **MIT** (inherited from the source model; redistribution of this
+  quantized derivative is permitted with attribution). This is a derivative work.
+
+## Validation (real gfx1201 hardware)
+| Metric | Value |
+|---|---:|
+| Perplexity (wikitext, 20 chunks, n_ctx=512) | **8.97** |
+| Prefill `pp512` | **2499 t/s** |
+| Decode `tg128` | **33.4 t/s** |
+
+Benched on a single R9700 (gfx1201).
+
+## Run it
+
+### llama.cpp (The Rock8 fork)
+```bash
+# reasoning chat (keeps <think>)
+llama-cli   -m DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -ngl 99 \
+  -p "Solve step by step: a train travels 60 km in 40 minutes. What is its speed in km/h?"
+# fast, no reasoning trace
+llama-cli   -m DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -ngl 99 -p "What do you call a dried grape? Answer in one word. /no_think"
+# bench
+llama-bench -m DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -ngl 99 -p 512 -n 128
+```
+
+### Lemonade appliance (container)
+```bash
+podman run -d --rm --runtime crun --name lemonade \
+  --device /dev/kfd --device /dev/dri \
+  --group-add keep-groups --security-opt seccomp=unconfined \
+  -v /path/to/quacken-r1-14b:/models:ro \
+  -e MODEL=/models/DeepSeek-R1-Distill-Qwen-14B-Quark-F8E4M3.gguf -e MODEL_NAME=Quacken-R1-14B-FP8 \
+  -e HIP_VISIBLE_DEVICES=0 -p 13305:13305 \
+  ghcr.io/the-monk/the-rock8:rdna4-tr713 serve
+```
+Container (same image on each registry; `--runtime crun` is required for GPU):
+`ghcr.io/the-monk/the-rock8:rdna4-tr713` - `docker.io/gorilla4x/the-rock8:rdna4-tr713` - `quay.io/the-monk/the-rock8:rdna4-tr713`
+(images may not be pushed to every registry yet).
+
+## The Rock8 - RDNA4 fp8 (links)
+- **GitHub:** [The-Rock8](https://github.com/The-Monk/The-Rock8) - kernels, patch series, appliance recipe, full feature doc.
+- **Collection:** [The Rock8 - RDNA4 fp8](https://huggingface.co/Gorilla4X).
+- **Sibling model:**
+  [Quacken-8B-FP8](https://huggingface.co/Gorilla4X/Quacken-8B-FP8).
+- **3 more coming:** Quacken-27B-FP8 (Qwen3.6-27B), Qwen3.6-35B-A3B (MoE) fp8, and Ornith-1.0-35B fp8 (in final validation).
+
+*Every artifact links to the others - land on any one, reach them all.*