Instructions to use netprojects/Inkling-2-Mini-MLX-4-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("netprojects/Inkling-2-Mini-MLX-4-8bit") config = load_config("netprojects/Inkling-2-Mini-MLX-4-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "netprojects/Inkling-2-Mini-MLX-4-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default netprojects/Inkling-2-Mini-MLX-4-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "netprojects/Inkling-2-Mini-MLX-4-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Inkling 2 Mini for Apple Silicon (MLX, 4/8-bit, lossless)
thinkingmachines/Inkling-2-Mini (revision 6e7c824) for MLX on
Apple Silicon, with its DSpark speculative-decoding draft bundled (mtp.safetensors) and its vision encoder included.
The routed experts are kept exactly as Thinking Machines released them: 4-bit NVFP4 in layers 0–19 and 8-bit MXFP8 in layers 20–23, repacked into MLX's native nvfp4 / mxfp8 formats with every value unchanged. Attention, embeddings, the output layer and the vision encoder stay in the original bf16. This is the release itself, in MLX. Size: 20.2 GB plus the 0.64 GB draft.
Inkling 2 Mini is Thinking Machines' looped mixture-of-experts model: 26.4B parameters, about 3.4B active per token, 24 layers of which the middle twelve run three times (48 layer passes per token), text and image input, up to 1M tokens of context.
Builds
| Build | Size | Same next token as Thinking Machines' code | KL vs their code | GSM8K (50 questions) | Peak memory, 8K-token prompt |
|---|---|---|---|---|---|
| netprojects/Inkling-2-Mini-MLX-4-8bit (recommended) | 20.2 GB + 0.64 GB draft | 98.6% | 0.0018 | 47/50 | 27.8 GB |
| netprojects/Inkling-2-Mini-MLX-3bit | 14.7 GB + 0.64 GB draft | 95.7% | 0.0421 | 48/50 | 21.2 GB |
| netprojects/Inkling-2-Mini-MLX-2.5bit | 11.5 GB + 0.64 GB draft | 87.3% | 0.1904 | 45/50 | 18.2 GB |
"Thinking Machines' code" is their Transformers implementation of Inkling 2 Mini, run on the released weights; the columns are explained under "What was checked".
Install
Inkling 2 Mini support is not in a released MLX-VLM yet. Install MLX-VLM with it from internet-projects/inkling2-mlx (MLX-VLM plus Inkling 2 Mini support; Python 3.10 or newer):
pip install "git+https://github.com/internet-projects/inkling2-mlx@v0.1.0" torch torchvision
PyTorch and torchvision are only used on the CPU by the image processor.
Use
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("netprojects/Inkling-2-Mini-MLX-4-8bit")
prompt = apply_chat_template(processor, model.config, "Explain in two sentences why the sky is blue.",
reasoning_effort="low")
print(generate(model, processor, prompt, max_tokens=2048, temperature=0.0).text)
reasoning_effort is part of Inkling's chat template: none, minimal, low, medium, high (the default), xhigh,
max, or a number from 0.0 to 0.99. Thinking Machines notes that the model can think at length even at none and can get
stuck in long reasoning loops, so set max_tokens.
With an image:
prompt = apply_chat_template(processor, model.config, "What error does this terminal show?", num_images=1, reasoning_effort="low")
print(generate(model, processor, prompt, image=["screenshot.png"], max_tokens=2048).text)
With the DSpark draft (greedy output stays the model's own, up to near-ties):
from mlx_vlm.speculative.drafters import load_drafter
draft, kind = load_drafter("netprojects/Inkling-2-Mini-MLX-4-8bit")
print(generate(model, processor, prompt, max_tokens=2048, draft_model=draft, draft_kind=kind).text)
OpenAI-compatible server, with tool calls and the thinking returned separately (reasoning_content):
mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-4-8bit --port 8000
Thinking Machines' results
From the Inkling 2 Mini card, for the original model (their harnesses and settings; a selection of their table):
| Benchmark (Thinking Machines' figures) | Inkling 2 Mini | Qwen3.8-27B | Gemma 4 31B |
|---|---|---|---|
| Artificial Analysis Intelligence Index v5.0 | 20.1 | 20.1 | 4.4 |
| SWE-bench Verified (bash only) | 79.2% | 77.8% | 46.1% |
| SWE-bench Pro v2 Hard (mini-swe-agent) | 77.9% | 73.8% | 14.2% |
| Terminal-Bench 2.1 (pi) | 71.6% | 73.0% | 43.4% |
| BrowseComp (with compaction) | 79.0% | 61.6% | 37.0% |
| MCP Atlas | 82.7% | 77.3% | 54.2% |
| MMMU Pro | 74.9% | 76.3% | 73.4% |
| OmniDocBench v1.6 | 92.8% | 91.5% | 82.5% |
| LongBench v2 | 66.9% | 52.3% | 52.9% |
| ARC-AGI v1 / v2 | 84.5 / 49.3% | 87.5 / 42.4% | – |
What was checked
On a MacBook Pro M5 Max (128 GB), 21 November 2026, with MLX 0.32.4 and inkling2-mlx v0.1.0.
- Against Thinking Machines' code. Their Transformers implementation (pull request #51050) ran on the CPU with the released weights, on 12 short chat texts (488 answer positions). This build gave the same next token at 98.6% of positions (mean KL 0.0018), and its greedy answers to 3 prompts matched theirs token for token (24 tokens each). Every tensor of this build is bit-identical to the release as MLX reads it.
- GSM8K, 50 fixed test questions (reasoning
none, greedy): 47/50. - Tool calling (reasoning
none): an agent prompt withbashandreadtools produced a well-formed call in Inkling's format, whichmlx_vlm.serverreturns as OpenAItool_calls. - Coding: 3/3 short Python tasks passed their asserts. Thinking (reasoning
low): 2/2 questions answered correctly after one closed thinking block. Vision: named the red square and blue circle of a test image. - Memory: peak 26.9 GB with a 4K-token prompt and 27.8 GB with an 8K-token prompt (mlx-vlm defaults). With a smaller prefill step the peak drops: 22.3 / 22.6 GB at
prefill_step_size=512and 21.4 / 21.6 GB atprefill_step_size=256(4K / 8K-token prompts, plus 0.64 GB if the DSpark draft is loaded), which fits the default GPU memory limit of a 36 GB Mac: passprefill_step_size=256togenerate, or--prefill-step-size 256tomlx_vlm.server. - DSpark: the bundled draft loads with this build and drafts about 7 tokens per round.
Precision note: Thinking Machines released the routed experts in 4-bit NVFP4 (8-bit MXFP8 in the last four layers) and no higher-precision weights; MLX builds of this model keep the experts at that precision, so this 4/8-bit build already carries the full released precision.
Known limitation of the current server: when the model calls a tool, content can also carry a short sentence of its
reasoning and the tool name.
License and credits
Apache-2.0, as Inkling 2 Mini. Use of the model is also governed by Thinking Machines'
Model Acceptable Use Policy. LICENSE is the Apache-2.0
text and NOTICE describes the conversion. Inkling 2 Mini is by Thinking Machines Lab; please cite their work.
Published by NetProjects (Internet Projects Ltd). Not affiliated with Thinking Machines.
More MLX builds from NetProjects: huggingface.co/netprojects · netprojects.com
- Downloads last month
- -
4-bit
Model tree for netprojects/Inkling-2-Mini-MLX-4-8bit
Base model
thinkingmachines/Inkling-2-Mini