File size: 2,113 Bytes
97785d7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
tags:
- rocmfpx
- gguf
- quantization
- llama-cpp
- playbook
license: mit
---
# ROCmFPX Quantize Playbook
A field-tested, agent-ready playbook for producing **ROCmFPX hybrid GGUF quantizations**
(`q4_0_rocmfp4_fast` / `q8_0_rocmfpx`) with the
[ROCmFPX fork of llama.cpp](https://github.com/charlie12345/ROCmFPX) — fully CPU-only,
from either pre-quantized GGUF repos (e.g. Unsloth BF16) or raw HF safetensors.
The complete guide is in [`rocmfpx-quantization-guide.md`](./rocmfpx-quantization-guide.md).
Hand this repo (or just that file) to a coding agent together with a Hugging Face link and a
recipe, and it can reproduce every step below.
## What it covers
- One-time CPU-only build of `llama-quantize` / `llama-gguf-split` + Python conversion deps
- **Workflow A:** source repo already has BF16 GGUF shards (e.g. `unsloth/*-GGUF`)
- **Workflow B:** raw safetensors → `convert_hf_to_gguf.py` → BF16 GGUF
- How to discover routed-expert tensor names per architecture (regex lookup table)
- Dry-run verification, shard merging, and output validation
- VRAM-fit hybrid recipe (q4 bulk + q8 sensitive) derived from Unsloth Dynamic tiers
- Cheatsheets, timings, and gotchas (regex anchoring, MTP auto-protection, gguf-py version trap…)
## Validated runs
| Model | Recipe | Result |
|---|---|---|
| [Laguna-S-2.1 118B-A10B](https://huggingface.co/JackBinary/Laguna-S-2.1-GGUF-ROCMFPX) | q4 experts / q8 rest | 224 GB → 61.6 GB (4.39 bpw) |
| [Qwen3.8-27B](https://huggingface.co/JackBinary/Qwen3.8-27B-GGUF-ROCMFPX) | pure q8 **and** 16 GB hybrid from UD-Q4_K_XL tiers | 26.9 GB (8.25 bpw) / 16.4 GB (5.15 bpw) |
| [G4-MeroMero-26B-A4B](https://huggingface.co/JackBinary/G4-MeroMero-26B-A4B-GGUF-ROCMFPX) | q4 experts / q8 rest + mmproj | 50.5 GB → 14.0 GB (4.64 bpw) |
Timings on a 64-core CPU box: 118B MoE ≈ 8 min, 27B dense ≈ 1 min, 26B MoE ≈ 3 min.
## Related
- Quant format project: https://github.com/charlie12345/ROCmFPX
- Companion model collection: [ROCmFPX GGUF Quants](https://huggingface.co/collections/JackBinary/rocmfpx-gguf-quants-6a7f51b4114f2897ea046100)
|