Instructions to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M # Run inference directly in the terminal: llama cli -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M # Run inference directly in the terminal: llama cli -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M # Run inference directly in the terminal: ./build/bin/llama cli -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Use Docker
docker model run hf.co/AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AtomicChat/Inkling-2-Mini-Uncensored-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomicChat/Inkling-2-Mini-Uncensored-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
- Ollama
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with Ollama:
ollama run hf.co/AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
- Unsloth Desktop
- Pi
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with Docker Model Runner:
docker model run hf.co/AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
- Lemonade
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Run and chat with the model
lemonade run user.Inkling-2-Mini-Uncensored-GGUF-IQ2_M
List all available models
lemonade list
- Hermes Agent
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use AtomicChat/Inkling-2-Mini-Uncensored-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "AtomicChat/Inkling-2-Mini-Uncensored-GGUF:IQ2_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Inkling-2-Mini Uncensored GGUF
- Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
An uncensored build of thinkingmachines/Inkling-2-Mini by AtomicChat (26.4B MoE, 3.4B active, looped core; text and image input; Apache-2.0, LICENSE in this repo). It comes as a LoRA adapter for any GGUF of AtomicChat/Inkling-2-Mini-GGUF and as three ready-made GGUF files with the edit baked in. Every gate we hold it to is measured below, and the two it misses are shown.
Files
| file | GB on disk | KLD to the original BF16 | top-1 | |
|---|---|---|---|---|
Inkling-2-Mini-Uncensored-AD-Q4_K.gguf |
16.53 | 0.0115 | 95.6% | AD-Q4_K in the base repo: 0.0080 / 96.3% |
Inkling-2-Mini-Uncensored-AD-IQ3_S.gguf |
13.34 | 0.0235 | 93.8% | AD-IQ3_S in the base repo: 0.0205 / 94.2% |
Inkling-2-Mini-Uncensored-AD-IQ2_M.gguf |
8.67 | 0.132 | 85.7% | AD-IQ2_M in the base repo: 0.130 / 85.7% |
Inkling-2-Mini-Uncensored-LoRA.gguf |
0.04 | the adapter, for any GGUF of the base repo, BF16 to 1-bit | ||
mmproj-Inkling-2-Mini-Uncensored-F16.gguf / -BF16.gguf |
0.64 | vision projector (the original's, unchanged) |
KLD and top-1 are against the original BF16 on our neutral corpus (87 chunks of 4096 tokens), the same stand as the base repo.
Run it
It needs llama.cpp built from ggml-org/llama.cpp#32695; build steps are in the GGUF card. llama.cpp merged a different implementation (#32694, release b12600) with its own GGUF layout; b12600 and later do not load these files yet. We are converting them to the merged layout under the same names.
./build/bin/llama-server -m Inkling-2-Mini-Uncensored-AD-Q4_K.gguf --mmproj mmproj-Inkling-2-Mini-Uncensored-F16.gguf \
--jinja -c 32768 -ngl 99
Or put the adapter on any file of the base repo:
./build/bin/llama-server -m Inkling-2-Mini-AD-Q4_K.gguf --lora Inkling-2-Mini-Uncensored-LoRA.gguf \
--mmproj mmproj-Inkling-2-Mini-F16.gguf --jinja -c 32768 -ngl 99
The adapter was measured at strength 1.0: --lora FILE and --lora-scaled FILE:1.0 are the same. Reasoning is set by reasoning_effort as in the base model.
What the edit is
A rank-1 LoRA on the 49 places that write into the residual stream: the attention output and the routed experts of all 24 layers, plus the token embedding. The 12 looped layers share their weights across the three passes, so one edit covers all 48 passes.
In each layer it removes that layer's own refusal direction: the difference of mean activations on harmful and harmless prompts, English and Russian, with the harmless mean projected out. The direction is pooled over prompts rendered with and without reasoning. Inkling 2 Mini reasons even at effort none and decides to refuse inside its reasoning, and a direction taken without reasoning alone still left soft refusals in chat.
What we measured
The BF16 model with the adapter against the same model without it, greedy decoding. Harmful prompts: mlabonne/harmful_behaviors and its Russian translation; the adapter was built from the train split, the numbers below come from the validation and test splits. Over-refusal: XSTest. MMLU: 2,000 questions.
| check | original | with the adapter | gate | |
|---|---|---|---|---|
| harmful test prompts refused, English (81), reasoning off | 81 (100%) | 0 | at most 3% | pass |
| harmful test prompts refused, Russian (100), reasoning off | 91 (91%) | 0 | at most 3% | pass |
| harmful validation prompts refused, English (104) / Russian (100) | 100% / 89% | 0 / 0 | at most 3% | pass |
harmful prompts refused with reasoning, effort low, 3,072-token budget (30) |
28 | 0 | at most 1 | pass |
the same at effort high (30) |
26 | 0 | at most 1 | pass |
8 harmful chat prompts, effort none / low (the baked AD-Q4_K too) |
8 / 8 | 0 / 0 | ||
| XSTest safe prompts refused (250) | 10 (4.0%) | 0 | at most the original's + 1 pt | pass |
| empty answers | 0 | 0 | at most 1% | pass |
harmless prompts whose reasoning did not close in 3,072 tokens, effort high (30) |
2 | 2 | at most the original's | pass |
| valid tool calls (20) | 18 | 20 | within 1 | pass |
| needle in a long context (3) | 3 | 3 | as the original | pass |
| KLD to the original, neutral text / code (24 chunks; random edit 0.0067 / 0.0052) | โ | 0.0078 / 0.0076 | 0.05 | pass |
| MMLU, 2,000 questions | 80.65% | 79.15% (โ1.50 pt, McNemar p = 0.006) | no significant drop | miss |
| first-token KL on harmless test prompts | โ | 0.94 (random edit: 0.014) | 0.10 | miss |
Some answers still open with a caution line, and then answer.
The two gates it misses
- MMLU: 79.15% against 80.65% (โ1.5 points, p = 0.006 on 2,000 questions). A random direction applied the same way costs 0.75 points (p = 0.08), so about half of the loss comes from editing every writer at all. Stronger edits cost more: โ2.2 points at strength 1.25.
- First-token KL on harmless prompts: 0.94 (gate 0.10; 0.375 on the validation prompts). The largest single value is about 22. Most of the mean likely comes from the harmless prompts the original refused (1โ6% of them) and the adapter answers, where the first token changes completely. KLD on running text, where refusals play no part, stays at 0.008.
- The leak probe reads 0.11 against its gate of 0.001. It measures one direction (row 29's) in every writer, while this adapter removes each layer's own direction. It also cannot be exact here: the attention and expert outputs pass a short per-channel convolution before the residual. Removing the convolved directions as well (the exact projection) broke the model in our tests (first-token KL 16โ21), so the adapter removes the plain direction. It is no gate here.
This model will answer requests that the original refuses. You are responsible for how you use it.
- Downloads last month
- 9,540
Model tree for AtomicChat/Inkling-2-Mini-Uncensored-GGUF
Base model
thinkingmachines/Inkling-2-Mini


