---
tags:
  - rocmfpx
  - gguf
  - quantization
  - llama-cpp
  - playbook
license: mit
---

# ROCmFPX Quantize Playbook

A field-tested, agent-ready playbook for producing **ROCmFPX hybrid GGUF quantizations**
(`q4_0_rocmfp4_fast` / `q8_0_rocmfpx`) with the
[ROCmFPX fork of llama.cpp](https://github.com/charlie12345/ROCmFPX) — fully CPU-only,
from either pre-quantized GGUF repos (e.g. Unsloth BF16) or raw HF safetensors.

The complete guide is in [`rocmfpx-quantization-guide.md`](./rocmfpx-quantization-guide.md).
Hand this repo (or just that file) to a coding agent together with a Hugging Face link and a
recipe, and it can reproduce every step below.

## What it covers

- One-time CPU-only build of `llama-quantize` / `llama-gguf-split` + Python conversion deps
- **Workflow A:** source repo already has BF16 GGUF shards (e.g. `unsloth/*-GGUF`)
- **Workflow B:** raw safetensors → `convert_hf_to_gguf.py` → BF16 GGUF
- How to discover routed-expert tensor names per architecture (regex lookup table)
- Dry-run verification, shard merging, and output validation
- VRAM-fit hybrid recipe (q4 bulk + q8 sensitive) derived from Unsloth Dynamic tiers
- Cheatsheets, timings, and gotchas (regex anchoring, MTP auto-protection, gguf-py version trap…)

## Validated runs

| Model | Recipe | Result |
|---|---|---|
| [Laguna-S-2.1 118B-A10B](https://huggingface.co/JackBinary/Laguna-S-2.1-GGUF-ROCMFPX) | q4 experts / q8 rest | 224 GB → 61.6 GB (4.39 bpw) |
| [Qwen3.8-27B](https://huggingface.co/JackBinary/Qwen3.8-27B-GGUF-ROCMFPX) | pure q8 **and** 16 GB hybrid from UD-Q4_K_XL tiers | 26.9 GB (8.25 bpw) / 16.4 GB (5.15 bpw) |
| [G4-MeroMero-26B-A4B](https://huggingface.co/JackBinary/G4-MeroMero-26B-A4B-GGUF-ROCMFPX) | q4 experts / q8 rest + mmproj | 50.5 GB → 14.0 GB (4.64 bpw) |

Timings on a 64-core CPU box: 118B MoE ≈ 8 min, 27B dense ≈ 1 min, 26B MoE ≈ 3 min.

## Related

- Quant format project: https://github.com/charlie12345/ROCmFPX
- Companion model collection: [ROCmFPX GGUF Quants](https://huggingface.co/collections/JackBinary/rocmfpx-gguf-quants-6a7f51b4114f2897ea046100)