How to use from the
Use from the
MLX library
# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("netprojects/Inkling-2-Mini-MLX-3bit")
config = load_config("netprojects/Inkling-2-Mini-MLX-3bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Inkling 2 Mini for Apple Silicon (MLX, 3-bit, for 24 GB Macs)

thinkingmachines/Inkling-2-Mini (revision 6e7c824) for MLX on Apple Silicon, with its DSpark speculative-decoding draft bundled (mtp.safetensors) and its vision encoder included. The routed experts are re-quantized from their released NVFP4 / MXFP8 values to 3-bit (affine, group 64), with 4-bit in layers 0–1 and in layers 20–23 (the layers Thinking Machines kept at MXFP8). Attention is 6-bit; embeddings, the output layer and the vision encoder are 4-bit. Size: 14.7 GB plus the 0.64 GB draft.

Inkling 2 Mini is Thinking Machines' looped mixture-of-experts model: 26.4B parameters, about 3.4B active per token, 24 layers of which the middle twelve run three times (48 layer passes per token), text and image input, up to 1M tokens of context.

Builds

Build Size Same next token as Thinking Machines' code KL vs their code GSM8K (50 questions) Peak memory, 8K-token prompt
netprojects/Inkling-2-Mini-MLX-4-8bit (recommended) 20.2 GB + 0.64 GB draft 98.6% 0.0018 47/50 27.8 GB
netprojects/Inkling-2-Mini-MLX-3bit 14.7 GB + 0.64 GB draft 95.7% 0.0421 48/50 21.2 GB
netprojects/Inkling-2-Mini-MLX-2.5bit 11.5 GB + 0.64 GB draft 87.3% 0.1904 45/50 18.2 GB

"Thinking Machines' code" is their Transformers implementation of Inkling 2 Mini, run on the released weights; the columns are explained under "What was checked".

Install

Inkling 2 Mini support is not in a released MLX-VLM yet. Install MLX-VLM with it from internet-projects/inkling2-mlx (MLX-VLM plus Inkling 2 Mini support; Python 3.10 or newer):

pip install "git+https://github.com/internet-projects/inkling2-mlx@v0.1.0" torch torchvision

PyTorch and torchvision are only used on the CPU by the image processor.

Use

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("netprojects/Inkling-2-Mini-MLX-3bit")
prompt = apply_chat_template(processor, model.config, "Explain in two sentences why the sky is blue.",
                             reasoning_effort="low")
print(generate(model, processor, prompt, max_tokens=2048, temperature=0.0).text)

reasoning_effort is part of Inkling's chat template: none, minimal, low, medium, high (the default), xhigh, max, or a number from 0.0 to 0.99. Thinking Machines notes that the model can think at length even at none and can get stuck in long reasoning loops, so set max_tokens.

With an image:

prompt = apply_chat_template(processor, model.config, "What error does this terminal show?", num_images=1, reasoning_effort="low")
print(generate(model, processor, prompt, image=["screenshot.png"], max_tokens=2048).text)

With the DSpark draft (greedy output stays the model's own, up to near-ties):

from mlx_vlm.speculative.drafters import load_drafter
draft, kind = load_drafter("netprojects/Inkling-2-Mini-MLX-3bit")
print(generate(model, processor, prompt, max_tokens=2048, draft_model=draft, draft_kind=kind).text)

OpenAI-compatible server, with tool calls and the thinking returned separately (reasoning_content):

mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-3bit --port 8000

Thinking Machines' results

From the Inkling 2 Mini card, for the original model (their harnesses and settings; a selection of their table):

Benchmark (Thinking Machines' figures) Inkling 2 Mini Qwen3.8-27B Gemma 4 31B
Artificial Analysis Intelligence Index v5.0 20.1 20.1 4.4
SWE-bench Verified (bash only) 79.2% 77.8% 46.1%
SWE-bench Pro v2 Hard (mini-swe-agent) 77.9% 73.8% 14.2%
Terminal-Bench 2.1 (pi) 71.6% 73.0% 43.4%
BrowseComp (with compaction) 79.0% 61.6% 37.0%
MCP Atlas 82.7% 77.3% 54.2%
MMMU Pro 74.9% 76.3% 73.4%
OmniDocBench v1.6 92.8% 91.5% 82.5%
LongBench v2 66.9% 52.3% 52.9%
ARC-AGI v1 / v2 84.5 / 49.3% 87.5 / 42.4% –

What was checked

On a MacBook Pro M5 Max (128 GB), 21 November 2026, with MLX 0.32.4 and inkling2-mlx v0.1.0.

  • Against Thinking Machines' code. Their Transformers implementation (pull request #51050) ran on the CPU with the released weights, on 12 short chat texts (488 answer positions). This build gave the same next token at 95.7% of positions (mean KL 0.0421), and its greedy answers to 3 prompts matched theirs for the first 24, 17, 24 of 24 tokens. Against our 4/8-bit build: 95.7% same next token, mean KL 0.0417.
  • GSM8K, 50 fixed test questions (reasoning none, greedy): 48/50.
  • Tool calling (reasoning none): an agent prompt with bash and read tools produced a well-formed call in Inkling's format, which mlx_vlm.server returns as OpenAI tool_calls.
  • Coding: 3/3 short Python tasks passed their asserts. Thinking (reasoning low): 2/2 questions answered correctly after one closed thinking block. Vision: named the red square and blue circle of a test image.
  • Memory: peak 20.0 GB with a 4K-token prompt and 21.2 GB with an 8K-token prompt (mlx-vlm defaults). With a smaller prefill step the peak drops: 15.6 / 15.9 GB at prefill_step_size=512 and 14.7 / 14.8 GB at prefill_step_size=256 (4K / 8K-token prompts, plus 0.64 GB if the DSpark draft is loaded), which fits the default GPU memory limit of a 24 GB Mac: pass prefill_step_size=256 to generate, or --prefill-step-size 256 to mlx_vlm.server.
  • DSpark: the bundled draft loads with this build and drafts about 7 tokens per round.

Known limitation of the current server: when the model calls a tool, content can also carry a short sentence of its reasoning and the tool name.

License and credits

Apache-2.0, as Inkling 2 Mini. Use of the model is also governed by Thinking Machines' Model Acceptable Use Policy. LICENSE is the Apache-2.0 text and NOTICE describes the conversion. Inkling 2 Mini is by Thinking Machines Lab; please cite their work.

Published by NetProjects (Internet Projects Ltd). Not affiliated with Thinking Machines.

More MLX builds from NetProjects: huggingface.co/netprojects · netprojects.com

Downloads last month
-
Safetensors
Model size
26B params
Tensor type
U32
·
F32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

3-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for netprojects/Inkling-2-Mini-MLX-3bit

Quantized
(22)
this model

Collection including netprojects/Inkling-2-Mini-MLX-3bit