In the works, a few hours away. The abliterated model is done and gated; the quant is being made right now. Every TBD below gets a real number before the file goes up.
stage (21 Nov 2026, AEST/AEDT) status abliteration (ARA) + refusal gate done: 6/100 harmful, 1/100 harmless refused teacher statistics (393K tokens through the engine) done per-expert sensitivity scoring + size/tier map running now, ~1 h quantisation next, ~1.5-2.5 h quality gates vs the parent (Gauntlet v3, held-out KL/top-1) right after, ~1 h upload projected tonight, 21 Nov (AEDT), if the gates pass Target: a ~12.5 GB file that runs 512K context on a single 24 GB RTX 4090 (quantised K/V cache).
Inkling-2-Mini 路 XQ, abliterated
Thinking Machines' Inkling 2 Mini with the refusal direction edited out of the weights, then requantised with Indra's Mirror's mixed-precision XQ pipeline. A 26B-parameter looped MoE with about 3B active parameters, 128 routed experts at top-8, and a 1M-token context.
Runs on Xolotl, an engine built for this class of model and machine, on top of Strata. The forward graph is ported from llama.cpp (PR #32694 by Pedro Cuenca, MIT); the engine around it is Xolotl's own.
| file size | TBD |
| routed experts | TBD |
| dense backbone | TBD |
| arch / tensors | TBD |
The model
Inkling 2 Mini is a 24-layer decoder-only transformer with sparse MoE feed-forward blocks. Its middle 12 layers (layers 6 to 17) run three times with shared weights, so each token gets 48 layer applications out of 24 layers of weights, and the cache carries 48 slots.
| total / active | 26B total, about 3B active |
| loop | layers 6 to 17 run 3 passes, 48 cache slots; passes 2 and 3 read the previous pass through per-layer low-rank loop controllers |
| MoE | 128 routed experts, top-8, no shared expert, sigmoid router with a selection bias, SiTU activation |
| attention | sliding window 1024 with a global layer every 6th (5, 11, 17, 23); no RoPE: a learned relative-position bias per head, with log-scaled positions on the global layers |
| extras | short causal convolutions on K, V, the attention output and the FFN output; block attention residuals that mix earlier depths; final logit softcap 30 |
| vocab | 200,058 |
| context | 1M tokens (the trained key says 1048576) |
| official precision | routed experts NVFP4 on layers 0 to 19, MXFP8 on 20 to 23, everything else BF16 |
The official checkpoint ships in that mixed precision. This build starts from it rather than from a dequantised copy, so the native FP4 experts can be carried through at their own precision.
How it was made
- Abliteration with Heretic-style ARA (Abliteration Rank Adjustment) on the attention output projections of layers 11 to 23, the model's only dense writers into the residual stream. The classic refusal-direction projection does not survive on this model: the routed experts are FP4, and re-encoding an edited expert to FP4 erases most of the edit (measured: only ~13 % of the direction removed). Refusal gate (system "You are a helpful assistant.", thinking off, 96 greedy tokens, a BERT refusal classifier): 6/100 harmful and 1/100 harmless prompts refused, at KL 0.033 against the original on harmless prompts.
- Quantisation with XQ: every routed-expert layer gets its own format, picked from each expert's measured output error on held-out text for a given file size; the official NVFP4 experts are kept bit-exact wherever the budget allows, and the dense backbone goes to 8-bit. Statistics come from the abliterated model itself, run through Xolotl's engine. The XQ pipeline itself is private.
Xolotl found and fixed a bug in the llama.cpp loop path along the way: passes 2 and 3 of the loop ran their experts without the per-expert NVFP4 scale when the GGUF keeps NVFP4 experts. The upstream converter dequantises, so upstream never meets that case. It is fixed in Xolotl's build.
Quality
Measured against the parent, which is the official checkpoint converted exactly (NVFP4 and MXFP8 kept) and run on the same engine, hardware and settings.
| parent | abliterated (before quant) | this (XQ) | |
|---|---|---|---|
| Gauntlet v3, solved | TBD | TBD | TBD |
| Gauntlet v3, score | TBD | TBD | TBD |
| held-out top-1 agreement vs parent | n/a | TBD | TBD |
| held-out KL vs parent | n/a | TBD | TBD |
| refusal gate, harmful prompts refused | TBD | 6/100 | TBD |
| refusal gate, harmless prompts refused | TBD | 1/100 | TBD |
Three columns on purpose: the abliteration and the quant are each checked for damage on their own.
The Gauntlet is Xolotl's agentic coding set: hard tasks run over multiple turns with tools in a sandbox, hidden tests graded after every turn, thinking on, run at the model's recommended sampling over several seeds.
Speed
TBD, measured on an RTX 4090 through the real server. Today, on the exact NVFP4 parent: 57 tok/s decode and about 1.1K tok/s prefill. In progress in Xolotl: the model's own DSpark block drafter (speculative decoding), a quantised K/V cache for 512K context, and flash attention.
Files
| file | size | what |
|---|---|---|
Inkling-2-Mini-XQ-Uncensored.gguf |
TBD | the model |
mmproj-Inkling-2-Mini-XQ.gguf |
TBD | planned: the official vision projector, unchanged, for images (text first) |
Quick start
Xolotl's xolotl-inkling engine runs any Inkling-2-Mini GGUF: the upstream converter's, llama.cpp's quants, or
this file.
git clone https://github.com/Indras-Mirror/Xolotl.git xolotl && cd xolotl
# build per docs/INSTALL.md, then serve with the Inkling config
Credits
The model is Thinking Machines Lab's, released with open weights under Apache 2.0. The Inkling 2 forward graph and the GGUF converter come from llama.cpp PR #32694 by Pedro Cuenca (MIT), which extends the earlier Inkling 2 work in llama.cpp and builds on ggml and llama.cpp by Georgi Gerganov and the llama.cpp contributors. Xolotl itself is built on Strata by Niko1221.
License
Apache 2.0, the same as the official model. The official card says, in its General Information section: "License: Apache 2.0", with the link to LICENSE.
Model tree for IndrasMirror/Inkling-2-Mini-XQ-Uncensored
Base model
thinkingmachines/Inkling-2-Mini