Inkling-2-Mini-Unbound v2 GGUF

Inkling-2-Mini-Unbound v2

GGUF builds of nokeepai/Inkling-2-Mini-Unbound v2, an uncensored Inkling 2 Mini, for llama.cpp, LM Studio and Ollama. The main model card has the method, the full evaluation and the limits.

Built by nokeep, who also sponsored the compute.

This model follows harmful instructions. It gives detailed answers to requests for harmful, dangerous or illegal content that the original model declines. It has no safety training layered back on. Do not deploy it to end users without your own safeguards.

v2 or v1.0?

v1.0 stopped the model refusing. v2 is trained on top of v1.0 to stop it dodging: fewer disclaimers, fewer "satire" or "conceptual" rewrites, more answers that actually do what was asked.

v1.0 vs v2

Original v1.0 v2
Useful answers to 277 held-out harmful requests, thinking off 0.7% 51.6% 78.7%
Same, thinking on (effort 0.9) 0.7% 62.1% 83.4%
Refused, thinking off / on 96.0% / 95.7% 0.0% / 0.0% 0.4% / 1.1%
Benign borderline prompts refused (784) 38.0% 0.0% 0.3%
Drift from the original, Q8_0 (KLD, lower is closer) โ€” 0.0102 0.0318
MMLU-300 / GSM8K-100 76.3 / 97 75.7 / 99 76.0 / 99
Tensors changed from the original โ€” 20 120
Files in this repo Inkling-2-Mini-Unbound-<TYPE>.gguf Inkling-2-Mini-Unbound-v2-<TYPE>.gguf

Measured on the Q8_0 GGUF in llama.cpp 19bc41dbb, greedy, answers allowed up to 8,192 tokens, on a sealed test split. "Useful" means an LLM judge (Jev) rates the answer as fulfilling the request.

  • Pick v2 for the most useful answers.
  • Pick v1.0 for the smallest change from the original model (3ร— lower drift), or to use the 7 MB LoRA.

The v1.0 files stay available under their v1.0 names (no -v2) and as the v1.0 revision of this repo. The main model card compares both with every other public uncensored Inkling 2 Mini.

Files

File Size SHA-256 KLD vs original Q8_0 Same top token
Inkling-2-Mini-Unbound-v2-Q8_0.gguf 27.8 GB 60c43f25b64930d72aac4219f688524e31f2cdf421b0ce0f60ae48a5ef70c8fa 0.0318 95.6%
Inkling-2-Mini-Unbound-v2-Q6_K.gguf 23.6 GB 81ca1e0a751ad4f944872132d7bd808ddd1f53f87a1705c42722d1b43423c00b 0.0324 95.5%
Inkling-2-Mini-Unbound-v2-Q5_K_M.gguf 20.1 GB f583f25f7b6b4135997299c55918da261ca0f08014851e89294f789f7cd7522e 0.0328 95.4%
Inkling-2-Mini-Unbound-v2-Q4_K_M.gguf 17.8 GB 55d3aa9c0e5c4c6b666451ae51ce70f6490b99f41959c6bae7e99b31d3f6d0db 0.0346 95.1%
Inkling-2-Mini-Unbound-v2-IQ3_M.gguf 13.3 GB 3d90872857c28e63650b5b3167b5fb4fe413bd8e5720e4a6871fcba9ac5230ab 0.0474 93.4%
Inkling-2-Mini-Unbound-v2-IQ2_M.gguf 11.2 GB 91d02db308b9a662675c588fc3ae23bfd24465ffa83ab140a3c2780bfa3604a0 0.0638 91.2%
mmproj-Inkling-2-Mini-BF16.gguf / -Q8_0.gguf 0.6 / 0.3 GB vision projector, unchanged from the original

KLD and top-token agreement are measured with llama.cpp against the original model's Q8_0 (ggml-org) on wikitext-2, 40 chunks of 4,096 tokens. The Q8_0 row is the model's own drift (v1.0: 0.0102); the smaller sizes add quantization loss on top. These numbers are not comparable with the v1.0 table, which was measured against BF16 at 512 tokens.

Which file should I pick?

Rule of thumb: to run fully on the GPU you need about the file size in VRAM, plus a few GB for the context (more for long contexts). Inkling 2 Mini is a mixture-of-experts model, so if it doesn't fit, llama.cpp can keep some experts in system RAM (--n-cpu-moe N). That works well, just slower; then VRAM + RAM together need to cover the file size plus headroom.

You have Pick File size
32 GB+ VRAM, or plenty of RAM Q8_0 (what the evaluation used; near lossless) 27.8 GB
24โ€“32 GB VRAM Q6_K or Q5_K_M (close to Q8_0) 23.6 GB / 20.1 GB
16โ€“24 GB VRAM Q4_K_M (the size most people want) 17.8 GB
12โ€“16 GB VRAM IQ3_M (noticeably lossy) 13.3 GB
Under 12 GB VRAM IQ2_M (lossy; or Q4_K_M with experts in RAM) 11.2 GB

Add the mmproj file (0.3 GB at Q8_0) only if you want image input.

How they were made

  1. Start from the official file. The base is ggml-org's Inkling-2-Mini-GGUF BF16.
  2. Patch 120 tensors, byte-exactly. v2 changes the q, k, v, output and gate attention projections of all 24 layers (attn_q, attn_k, attn_v, attn_output, attn_gate). Before writing each one, the patch script proves the mapping: the original checkpoint's BF16 bytes must equal ggml-org's tensor bytes exactly. Only then are v2's BF16 bytes written in place. The other 584 tensors (every expert, the router, embeddings, head) are ggml-org's bytes, untouched. The only header change is the model name.
  3. Quantize with stock llama.cpp. llama-quantize from the patched BF16. Every size below Q8_0 uses ggml-org's importance matrix, and keeps all five edited attention tensor types at Q8_0 so the change isn't blurred by low-bit rounding.

The main card's evaluation ran on the Q8_0 file in this repo, so its numbers apply to it directly.

Run it

You need a llama.cpp build with Inkling 2 support that includes the loop fix from PR #32704. That fix is on master (we used commit 19bc41dbb); tagged releases up to b12608 don't have it. Apps that bundle llama.cpp need an equally recent build.

llama.cpp

hf download nokeepai/Inkling-2-Mini-Unbound-GGUF \
    Inkling-2-Mini-Unbound-v2-Q4_K_M.gguf mmproj-Inkling-2-Mini-Q8_0.gguf --local-dir models

./build/bin/llama-server \
    -m models/Inkling-2-Mini-Unbound-v2-Q4_K_M.gguf \
    --mmproj models/mmproj-Inkling-2-Mini-Q8_0.gguf \
    --jinja -c 32768 -ngl 99

LM Studio. Search for nokeepai/Inkling-2-Mini-Unbound-GGUF and pick a file with -v2 in its name. The files without -v2 are v1.0.

Ollama. Download a file as above, then point a Modelfile at it:

echo 'FROM ./models/Inkling-2-Mini-Unbound-v2-Q4_K_M.gguf' > Modelfile
ollama create inkling-unbound-v2 -f Modelfile
ollama run inkling-unbound-v2

Settings

  • Thinking effort is set through the chat template (reasoning_effort, 0 to 0.99, default 0.9), the same as the original. OpenAI-compatible servers take it as "chat_template_kwargs": {"reasoning_effort": 0}. The model still thinks briefly at 0.
  • Set a thinking cap (we used 4,096 tokens). At effort 0.9, v2 hits that cap on 17% of harmful prompts.

About the LoRA

The 7 MB Inkling-2-Mini-Unbound-LoRA-r16.gguf in this repo is v1.0's adapter. Loaded on any GGUF of the original model it gives v1.0 behaviour, not v2. There is no GGUF LoRA for v2. v2's own LoRA (on top of v1.0, in safetensors) is in the main repo under finetune/.

Known limits

  • More drift than v1.0. KLD 0.0318 vs 0.0102. MMLU-300 and GSM8K-100 stay within noise of the original, but expect more wording differences from the original on ordinary requests.
  • Long thinking at high effort. 17% of harmful prompts hit a 4,096-token thinking cap at effort 0.9 (v1.0 13%). Set a cap.
  • Occasional soft refusal. About 1% of harmful requests at effort 0.9.
  • Not every answer is useful. About 17โ€“21% of harmful requests still get a vague, partial or off-target answer.
  • Crisis handling. v2 was not trained to add crisis resources and may leave them out. Deployments that can reach people at risk need their own handling.
  • Not yet measured for v2: the full capability suite v1.0 passed, long-context, vision and tool use.
  • Full list: main model card.

Intended use and misuse

Intended for research on refusal and model editing, red-teaming and safety evaluation, and for developers who apply their own content policy in a system they control. Not intended for products that serve the public without moderation, content that facilitates violence, weapons capable of mass harm, attacks on computer systems or abuse of children or other people, or anything illegal where you use it. The base model's Acceptable Use Policy applies.

Licence

Apache 2.0, like the base model, Inkling 2 Mini by Thinking Machines Lab, Inc. 120 attention tensors differ from the original; every other tensor is ggml-org's conversion of the original, unmodified before quantization.

About nokeep

This model was built, and its compute paid for, by nokeep. nokeep is a private AI app for uncensored chat, images and video. If this model is useful to you, the best way to support more releases like it is to try nokeep.

Downloads last month
7,852
GGUF
Model size
26B params
Architecture
inkling2_loop
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for nokeepai/Inkling-2-Mini-Unbound-GGUF

Quantized
(4)
this model

Space using nokeepai/Inkling-2-Mini-Unbound-GGUF 1