sudoingx/Offrail-27B-Uncensored

Offrail 27B: the uncensored Bonsai 2 27B, MTP on

27B for 8GB and 12GB cards. OS-Software's uncensored Heretic build of Ternary Bonsai 2 27B (the 1.75 bpw PTQ1_0 file, 5.95 GB) with the Qwen 3.8 27B multi-token-prediction head grafted back on as block 64, so the PrismML llama.cpp fork runs --spec-type draft-mtp on it.

The body is not ours: OS-Software baked a Heretic LoRA (rank 128, layers 27-44) into the official PQ2_0 ternary codes and kept the block scales, and their card calls it an approximate merge. Their test: 0/100 refusals against the original's 95/100 at KL divergence 0.0135, WikiText-2 perplexity 10.1435. Their test, not ours: we have not run a refusal baseline of our own (HarmBench needs a second GPU for its judge model).

What is ours: the graft, the two merged files and the graft tooling, plus the PTQ1_0 mat-vec kernel (PrismML-Eng/llama.cpp#218) on top of PrismML's fork. What is not ours: the head's weights are Qwen's, the MTP graph and the GGML_CUDA_BATCH_INVARIANT switch are PrismML's fork, the uncensoring is OS-Software's, the compression is PrismML's.

uncensored + MTP, one file, 12GB	Offrail-27B-12G.gguf 7.0 GB, 50.8 tok/s on an RTX 3060 12GB
the same, for 8GB	Offrail-27B-8G.gguf 6.3 GB, 49.6 tok/s at 131K, 49.8 at 48K
lossless	greedy output identical head on vs off, GGML_CUDA_BATCH_INVARIANT=1

Install

One line on Linux or Windows: it picks the build and the file for your card (Offrail-27B-8G.gguf on 8 GB, Offrail-27B-12G.gguf on 12 GB and up), checks every SHA256 and opens the chat page at http://127.0.0.1:8899.

Windows, in PowerShell:

$env:BONSAI_MODEL='offrail'; irm https://raw.githubusercontent.com/sudoingX/local-ai-on-your-gaming-gpu/main/install.ps1 | iex

Linux or WSL2:

curl -fsSL https://raw.githubusercontent.com/sudoingX/local-ai-on-your-gaming-gpu/main/install.sh | bash -s -- --model offrail

NVIDIA RTX 20 to 50 (RTX 20 on Windows in beta), and AMD Radeon in beta on the same two lines. The installers live in local-ai-on-your-gaming-gpu; the manual llama-server line is under Files below.

Numbers

RTX 3060 12GB, one slot, thinking off, q4_0 K/V, flash attention on, probe.py medians (3 prompts x 3 runs, 400 tokens), on the kernel branch (bonsai2 @ ff41412) with GGML_CUDA_BATCH_INVARIANT=1.

file	window	draft head	overall	code	prose	bash	VRAM after load
12G	131,072	off	41.4	41.4	41.5	41.2	8,734 MiB
12G	131,072	draft-mtp, n-max 1	50.8	55.4	45.5	50.2	10,618 MiB
8G	131,072	off	41.4	41.4	41.6	41.4	8,734 MiB
8G	131,072	draft-mtp, n-max 1	49.6	54.6	44.0	49.6	9,936 MiB
8G	65,536	draft-mtp, n-max 2	48.4	56.0	35.4	48.4	8,294 MiB
8G	49,152, ubatch 256	draft-mtp, n-max 2	49.8	57.4	35.9	49.8	7,732 MiB
8G	32,768, ubatch 256	draft-mtp, n-max 2	not probed				7,300 MiB

The read:

• Uncensoring costs no speed. The base graft runs 41.3 tok/s head off and 50.2 with the head; here it is 41.4 and 50.8, on the same kernel and the same shapes.
• The grafted head adds +23% on the 12G file and +20% on the 8G file. By prompt on 8G: code +32%, bash +20%, prose +6%.
• The 8G file costs about 1.2 tok/s against the 12G one (49.6 vs 50.8) and saves 682 MiB, because it carries no second copy of the token embedding.
• On 8GB, 48K fits (7,732 MiB) and 64K does not (8,294 MiB). 32K is 7,300 MiB, the pick when a desktop is also on the card. n-max 2 buys code and costs prose: at 48K it is 57.4 code against 54.6 at 131K n-max 1, and 35.9 prose against 44.0. The overalls land within a point of each other (49.8 against 49.6), so pick by window: n-max 1 at a long window, n-max 2 when the window is small.

Draft acceptance, greedy runs (temperature 0, top_k 1, 300 tokens): code 0.91 (142/156), bash 0.76 (129/170), prose 0.55 (68/123) on the 8G file; on the 12G file 0.92 / 0.79 / 0.61. Prose is the weak spot: the head was trained by Qwen against fp16 hidden states and here it reads a ternary trunk.

Losslessness

With GGML_CUDA_BATCH_INVARIANT=1, greedy output (temperature 0, top_k 1, 300 tokens, 3 prompts) with the head on is byte-identical to greedy output with the head off, on both files. On the 8G file all three setups agree: head off at 131K, head on at 131K n-max 1, and head on at 64K n-max 2 produce the same nine texts (code 48b24993…, prose bfc74792…, bash 195b4b12…, 1,007 / 1,017 / 1,125 bytes). The 12G pair was verified the same way on Oct 14. Without the switch, batched verification changes the floating-point reduction order and the text can differ at near-ties. That is true of every graft in this family, and on Turing (RTX 20) the switch does not cover every kernel, so a near tie can flip a token late. Sampling at temperature 1.0, top_p 0.95, top_k 20 is unaffected either way: every drafted token is verified against the target model.

Files

file	bytes	sha256
Offrail-27B-12G.gguf	7,012,820,544	3142487fa092ba27f90df0a54220996f82e9635c1a2f5ac28ad45971d2a06ac3
Offrail-27B-8G.gguf	6,297,658,848	be2b5ce27bd4bf6125f571e994489a325b711b092315c4b874837e4649fc04ed

Both need the PrismML llama.cpp fork (stock llama.cpp cannot read PTQ1_0). For the numbers above, the bonsai2 branch of github.com/sudoingX/llama.cpp, which is the kernel plus the MTP fixes; the 8G file additionally needs the 15-line qwen35 MTP Hadamard fix (PrismML-Eng/llama.cpp#205, merged into PrismML's fork on Sep 21) that is already in that branch. The prebuilt bundles in sudoingX/Ternary-Bonsai-2-27B-PTQ1_0-MTP-GGUF carry both and serve these files unchanged.

hf download sudoingx/Offrail-27B-Uncensored Offrail-27B-12G.gguf --local-dir .
GGML_CUDA_BATCH_INVARIANT=1 llama-server -m Offrail-27B-12G.gguf \
  -ngl 99 -fa on -c 131072 -np 1 -ctk q4_0 -ctv q4_0 --jinja \
  --reasoning-effort medium --spec-type draft-mtp --spec-draft-n-max 1

On 8GB take Offrail-27B-8G.gguf with -c 49152 -ub 256 --spec-draft-n-max 2 (7,732 MiB), or -c 32768 if your desktop is on the card. Vision: add --mmproj with the mmproj from OS-Software's repo, which is byte-identical to PrismML's (sha256 6807ede61d570bb86ba34b756a0fa109edc33668604de867c6ea6d8f1d631903). The tower loading is not the same thing as the answers being good: we have not run the vision check on this uncensored body, and the 16GB configuration (the head, the tower and the full 262,144 window in 15.1 GB) is the base file's row, not a measurement of this one.

Set --reasoning-effort medium. The chat template defaults it to xhigh, which injects a "think carefully through the task" system line; on an RTX 3060 at a 4,096-token client cap that returned nothing at all on three build tasks. The measurement is in kernel/reasoning_effort.md.

How it was made

The body is OS-Software's Ternary-Bonsai-2-27B-Uncensored-Heretic-PTQ1_0.gguf (5,946,648,928 B, sha256 a18c3e17da397838305522b3ee877f9d2cbf4facd04ba01a394b22f3f3a477be) with 15 tensors appended: one block's worth of attention (blk.64.attn_norm, blk.64.attn_q, blk.64.attn_k, blk.64.attn_v, blk.64.attn_q_norm, blk.64.attn_k_norm, blk.64.attn_output, blk.64.post_attention_norm), the FFN (blk.64.ffn_gate, blk.64.ffn_up, blk.64.ffn_down) and the four nextn projections blk.64.nextn.eh_proj, blk.64.nextn.enorm, blk.64.nextn.hnorm, blk.64.nextn.shared_head_norm (Q6_K, Q8_0, F32, 335 MiB). That is Qwen 3.8 27B's own MTP block, copied byte for byte from the graft in sudoingX/Ternary-Bonsai-2-27B-PTQ1_0-MTP-GGUF. The header gains qwen35.block_count 64 → 65, qwen35.nextn_predict_layers = 1 and four graft.* provenance keys; every one of the body's 851 tensors keeps its bytes and its offset. merge.py --strip on Offrail-27B-8G.gguf reproduces the body's own sha256 above, so the uncensored weights are untouched.

Offrail-27B-12G.gguf carries its own plain copy of the token embedding for the head (682 MiB, the full variant). Offrail-27B-8G.gguf does not: the head reads the trunk's table through the Hadamard inverse fix. Both were built on an RTX 3060 12GB in about a minute from the two public parents; the tools are graft/ in the repo, and tools/merge.py --strip proves the round trip. The 15 head tensors in this file hash identically, tensor for tensor, to the ones in the base repo's -lean.gguf file.

Not run yet on this body: the vision check with the mmproj, the depth curve, concurrency, and our own refusal baseline. The last of those is unrun for the base file too, so the only refusal figure in this card is OS-Software's. Vision, depth and concurrency do exist as rows for the base file.

Credit

PrismML for Bonsai 2, the ternary compression and the fork; OS-Software for the uncensored Heretic build and the Heretic LoRA (made with p-e-w's tool); Qwen for the 27B trunk and the MTP head; unsloth for the donor GGUF the head was exported from. Qwen, PrismML, OS-Software and unsloth are the four links in the chain, and none of them made this file.

For the two pieces this file leans on that we did not write: decent-jawfish, ProCreations and BoldingBuilds for the Hadamard patch, which is what lets the 8G file read the trunk's own embedding table instead of carrying a 682 MiB copy; the graft recipe traces back to sudoingX/qwen38-mtp, and the PTQ1_0 kernel is PrismML-Eng/llama.cpp#218. No other credit is claimed and none is implied: the uncensoring, the compression and the head belong to OS-Software, PrismML and Qwen, and everything this repo adds is the graft and the file it produced.

Responsible use

This is an uncensored model: it will answer what the base refuses. The uncensoring is OS-Software's approximate merge of a Heretic LoRA into the ternary weights, and it comes with their notice. Use it where you are allowed to use it, and you own the output. We host the weights; we did not bake the LoRA and we do not moderate what it says.

Licence

Apache 2.0 throughout the chain: Qwen/Qwen3.8-27B (Alibaba Cloud), prism-ml/Ternary-Bonsai-2-27B-gguf (PrismML), OS-Software/Ternary-Bonsai-2-27B-Uncensored-Heretic-GGUF (OS-Software), and the graft in sudoingX/Ternary-Bonsai-2-27B-PTQ1_0-MTP-GGUF (ours). This repo redistributes their weights unchanged, under the same licence, with credit. Not an official PrismML, Qwen or OS-Software release, and not affiliated with any of them.