| --- |
| license: apache-2.0 |
| base_model: jialinyyzz/humanizer |
| base_model_relation: quantized |
| pipeline_tag: text-generation |
| library_name: gguf |
| language: [en, zh] |
| tags: [gguf, rocmfpx, rocmfp4, amd, rocm, strix-halo, quantized, humanizer, text-rewriting, gemma4] |
| --- |
| |
| # humanizer 12B — ROCmFPX GGUF |
|
|
| [jialinyyzz/humanizer](https://huggingface.co/jialinyyzz/humanizer) (v2) rewrites AI-written drafts in English and |
| Chinese so they read like a person wrote them, and keeps every number, date, name and quote. It is a fine-tune of |
| Gemma 4 12B. This repo has it in the [ROCmFPX](https://github.com/charlie12345/ROCmFPX) formats, a llama.cpp fork |
| with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm). |
|
|
| **Start with `humanizer-12b-Q8_0_ROCMFPX.gguf`** if it fits in your GPU memory: it is almost identical to the full |
| model. Take the 6-bit file next, and a 4-bit file when memory is tight. |
|
|
| > These files need [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX), built from |
| > 23 October 2026 or later (they carry per-tensor scales, see below). Stock llama.cpp, LM Studio, Ollama and other |
| > ROCmFPX builds may not load them. **These files have not been tested on AMD hardware yet.** |
| > |
| > Not an official release of humanizer. All credit for the model goes to its authors. Their own files in stock |
| > llama.cpp formats, including quantization-aware trained 2-, 3- and 4-bit files, are at |
| > [jialinyyzz/humanizer-GGUF](https://huggingface.co/jialinyyzz/humanizer-GGUF). |
|
|
| ## Files |
|
|
| | File | Size | KL to BF16 | Same top token | | |
| |---|---:|---:|---:|---| |
| | `humanizer-12b-Q8_0_ROCMFPX.gguf` | 12.3 GB | 0.00024 | 99.3% | 8-bit, closest to the full model | |
| | `humanizer-12b-Q6_K_ROCMFPX.gguf` | 9.8 GB | 0.0023 | 97.8% | 6-bit (ROCmFPX `Q6_0_ROCMFPX` weights, see below) | |
| | `humanizer-12b-Q4_0_ROCMFP4_STRIX.gguf` | 6.7 GB | 0.040 | 91.9% | recommended 4-bit (Strix Halo recipe) | |
| | `humanizer-12b-Q4_0_ROCMFP4_FAST.gguf` | 6.3 GB | 0.047 | 90.8% | smallest, speed-first 4-bit | |
|
|
| Stock llama.cpp builds of this model, measured the same way: |
|
|
| | Stock file | Size | KL to BF16 | Same top token | |
| |---|---:|---:|---:| |
| | Q8_0 | 12.7 GB | 0.00064 | 98.8% | |
| | Q6_K | 10.0 GB | 0.0026 | 97.8% | |
| | Q4_K_M | 7.4 GB | 0.0186 | 94.1% | |
|
|
| The 8-bit file here is closer to BF16 than a stock Q8_0, and the 6-bit file matches a stock Q6_K; both are a |
| little smaller. At 4-bit, a stock Q4_K_M (or the authors' quantization-aware Q4_K_M) stays closer to the full model than the ROCmFP4 |
| files here. The ROCmFP4 files are smaller and are meant for speed on AMD GPUs. |
|
|
| **How this was measured:** llama.cpp `llama-perplexity --kl-divergence` against the authors' BF16 file, 40 chunks of |
| 512 tokens of English Wikipedia text (wikitext-2), run on a GPU after expanding each file to F16 (the expansion is |
| exact). KL is how far the file's next-token probabilities drift from BF16, averaged over every token; lower is better. |
| "Same top token" is how often the file's most likely next token is the same as BF16's. This is general text, not |
| rewriting text; the authors measured their files on held-out rewriting data, and their stock Q4_K_M (KL 0.021) is |
| close to what this test gives (0.0186). |
|
|
| Every file also passed a rewrite check: a test draft with five facts (a date, a count, a name, a year and an amount), |
| greedy decoding on the CPU. All five facts came through in every file. |
|
|
| The 6-bit file is named `humanizer-12b-Q6_K_ROCMFPX.gguf` so that Hugging Face lists it as a 6-bit quant. Its weights |
| are the ROCmFPX `Q6_0_ROCMFPX` type, not a stock llama.cpp `Q6_K`, and it needs the same fork as the other files. |
|
|
| ## Why these files carry tensor scales |
|
|
| ROCmFPX stores one scale for each block of weights as an 8-bit float (UE4M3) whose smallest step is 2^-10. This |
| model's weights are small (the largest value in a typical block of 32 is about 0.02 to 0.04), so an 8-bit ROCmFPX |
| file cannot pick small enough scales: almost every block gets stuck at the smallest scale, and the file is about as |
| precise as a 6-bit one. These files multiply each weight matrix by a power of two (2^7 to 2^10) before quantizing and store |
| the inverse as `<name>.scale`, which the fork multiplies back after the matrix product. The shared embedding and |
| output matrix cannot take such a scale, so it uses a stock llama.cpp type: Q8_0 in the 8-bit file, Q6_K in the 6-bit |
| and STRIX files (the FAST file keeps it in ROCmFP4 for speed). |
|
|
| | KL to BF16 | Plain ROCmFPX | With tensor scales | |
| |---|---:|---:| |
| | 8-bit | 0.0112 | 0.00024 | |
| | 6-bit | 0.0080 | 0.0023 | |
| | 4-bit STRIX | 0.053 | 0.040 | |
| | 4-bit FAST | 0.064 | 0.047 | |
|
|
| ## How these were made |
|
|
| 1. Source: the authors' BF16 GGUF (`humanizer-12b-bf16.gguf` from |
| [jialinyyzz/humanizer-GGUF](https://huggingface.co/jialinyyzz/humanizer-GGUF)), sha256 checked against their card. |
| 2. Importance matrix from the BF16 file over a general calibration text (`calibration_datav3`), 512-token chunks. |
| The authors used their own rewriting data for theirs. |
| 3. `llama-quantize --imatrix` from the BF16 file with `LLAMA_ROCMFPX_TENSOR_SCALE=1`, one run per type, from |
| [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX); `--token-embedding-type q8_0` for the |
| 8-bit file and `q6_K` for the 6-bit file. |
|
|
| The only change from the original is the weight quantization. The tokenizer, the chat template and the stored sampling |
| defaults come unchanged from the authors' GGUF. |
|
|
| ## Run |
|
|
| ```bash |
| git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX |
| cmake -B build -DGGML_VULKAN=ON # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900) |
| cmake --build build --target llama-server -j |
| build/bin/llama-server -m humanizer-12b-Q8_0_ROCMFPX.gguf -ngl 999 -c 8192 -np 1 --jinja |
| ``` |
|
|
| The model does text completion, not chat. The prompt is the instruction from `prompt_format.json`, a blank line, your |
| draft, then `\n\n### Rewritten:\n\n`. Sample with temperature 1.0, top-p 0.95, top-k 0, min-p 0 and repeat penalty 1.0, |
| and stop at the end-of-sequence token only. |
|
|
| ```python |
| import json, urllib.request |
| |
| pf = json.load(open("prompt_format.json", encoding="utf-8")) |
| |
| def humanize(draft, url="http://127.0.0.1:8080"): |
| body = {"prompt": pf["instr"] + "\n\n" + draft.strip() + pf["sep"], |
| "temperature": 1.0, "top_p": 0.95, "top_k": 0, "min_p": 0, |
| "repeat_penalty": 1.0, "n_predict": 2048} |
| req = urllib.request.Request(url + "/completion", json.dumps(body).encode("utf-8"), |
| {"Content-Type": "application/json"}) |
| with urllib.request.urlopen(req, timeout=900) as r: |
| return json.load(r)["content"].strip() |
| |
| print(humanize(open("draft.txt", encoding="utf-8").read())) |
| ``` |
|
|
| The chat template in each file builds the same prompt from the last user message, so `/v1/chat/completions` with one |
| draft per message also works. Keep instruction, draft and rewrite within 8,192 tokens and split long documents at |
| paragraph breaks. The authors note that a short, informal English draft occasionally comes back in Chinese; sample |
| again when that happens. Their [usage guide](https://github.com/sgaofen/humanizer-local-model/blob/main/docs/USAGE.md) |
| has more. Their desktop app runs stock llama.cpp with their own files, not these. |
|
|
| ## License |
|
|
| Apache-2.0, same as the original (see `LICENSE`). The authors' `NOTICE` file is included and must stay with any copy. |
| humanizer is fine-tuned from [google/gemma-4-12B](https://huggingface.co/google/gemma-4-12B), which Google releases |
| under Apache 2.0. This repo is not affiliated with or endorsed by Google or the humanizer authors. |
|
|
