Image-Text-to-Text
MLX
Safetensors
inkling2_loop
apple-silicon
Mixture of Experts
dspark
speculative-decoding
conversational
4-bit precision
Instructions to use netprojects/Inkling-2-Mini-MLX-4-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("netprojects/Inkling-2-Mini-MLX-4-8bit") config = load_config("netprojects/Inkling-2-Mini-MLX-4-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "netprojects/Inkling-2-Mini-MLX-4-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default netprojects/Inkling-2-Mini-MLX-4-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use netprojects/Inkling-2-Mini-MLX-4-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "netprojects/Inkling-2-Mini-MLX-4-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from netprojects/Inkling-2-Mini-MLX-4-8bit: direct link, hf CLI and curl.
- Browser
- Download file 7.47 kB
-
https://huggingface.co/netprojects/Inkling-2-Mini-MLX-4-8bit/resolve/main/README.md
- Command line
-
hf download hf://netprojects/Inkling-2-Mini-MLX-4-8bit/README.md
-
curl -L -o README.md https://huggingface.co/netprojects/Inkling-2-Mini-MLX-4-8bit/resolve/main/README.md
7.47 kB
| license: apache-2.0 | |
| license_link: https://www.apache.org/licenses/LICENSE-2.0 | |
| base_model: thinkingmachines/Inkling-2-Mini | |
| base_model_relation: quantized | |
| pipeline_tag: image-text-to-text | |
| library_name: mlx | |
| tags: | |
| - mlx | |
| - apple-silicon | |
| - inkling2_loop | |
| - moe | |
| - dspark | |
| - speculative-decoding | |
| # Inkling 2 Mini for Apple Silicon (MLX, 4/8-bit, lossless) | |
| [thinkingmachines/Inkling-2-Mini](https://huggingface.co/thinkingmachines/Inkling-2-Mini) (revision `6e7c824`) for MLX on | |
| Apple Silicon, with its DSpark speculative-decoding draft bundled (`mtp.safetensors`) and its vision encoder included. | |
| The routed experts are kept exactly as Thinking Machines released them: 4-bit NVFP4 in layers 0–19 and 8-bit MXFP8 in layers 20–23, repacked into MLX's native nvfp4 / mxfp8 formats with every value unchanged. Attention, embeddings, the output layer and the vision encoder stay in the original bf16. This is the release itself, in MLX. Size: 20.2 GB plus the 0.64 GB draft. | |
| Inkling 2 Mini is Thinking Machines' looped mixture-of-experts model: 26.4B parameters, about 3.4B active per token, | |
| 24 layers of which the middle twelve run three times (48 layer passes per token), text and image input, up to 1M tokens of context. | |
| ## Builds | |
| | Build | Size | Same next token as Thinking Machines' code | KL vs their code | GSM8K (50 questions) | Peak memory, 8K-token prompt | | |
| |---|---|---|---|---|---| | |
| | [netprojects/Inkling-2-Mini-MLX-4-8bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-4-8bit) (recommended) | 20.2 GB + 0.64 GB draft | 98.6% | 0.0018 | 47/50 | 27.8 GB | | |
| | [netprojects/Inkling-2-Mini-MLX-3bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-3bit) | 14.7 GB + 0.64 GB draft | 95.7% | 0.0421 | 48/50 | 21.2 GB | | |
| | [netprojects/Inkling-2-Mini-MLX-2.5bit](https://huggingface.co/netprojects/Inkling-2-Mini-MLX-2.5bit) | 11.5 GB + 0.64 GB draft | 87.3% | 0.1904 | 45/50 | 18.2 GB | | |
| "Thinking Machines' code" is their Transformers implementation of Inkling 2 Mini, run on the released weights; the | |
| columns are explained under "What was checked". | |
| ## Install | |
| Inkling 2 Mini support is not in a released MLX-VLM yet. Install MLX-VLM with it from [internet-projects/inkling2-mlx](https://github.com/internet-projects/inkling2-mlx) | |
| (MLX-VLM plus Inkling 2 Mini support; Python 3.10 or newer): | |
| ```bash | |
| pip install "git+https://github.com/internet-projects/inkling2-mlx@v0.1.0" torch torchvision | |
| ``` | |
| PyTorch and torchvision are only used on the CPU by the image processor. | |
| ## Use | |
| ```python | |
| from mlx_vlm import load, generate | |
| from mlx_vlm.prompt_utils import apply_chat_template | |
| model, processor = load("netprojects/Inkling-2-Mini-MLX-4-8bit") | |
| prompt = apply_chat_template(processor, model.config, "Explain in two sentences why the sky is blue.", | |
| reasoning_effort="low") | |
| print(generate(model, processor, prompt, max_tokens=2048, temperature=0.0).text) | |
| ``` | |
| `reasoning_effort` is part of Inkling's chat template: `none`, `minimal`, `low`, `medium`, `high` (the default), `xhigh`, | |
| `max`, or a number from 0.0 to 0.99. Thinking Machines notes that the model can think at length even at `none` and can get | |
| stuck in long reasoning loops, so set `max_tokens`. | |
| With an image: | |
| ```python | |
| prompt = apply_chat_template(processor, model.config, "What error does this terminal show?", num_images=1, reasoning_effort="low") | |
| print(generate(model, processor, prompt, image=["screenshot.png"], max_tokens=2048).text) | |
| ``` | |
| With the DSpark draft (greedy output stays the model's own, up to near-ties): | |
| ```python | |
| from mlx_vlm.speculative.drafters import load_drafter | |
| draft, kind = load_drafter("netprojects/Inkling-2-Mini-MLX-4-8bit") | |
| print(generate(model, processor, prompt, max_tokens=2048, draft_model=draft, draft_kind=kind).text) | |
| ``` | |
| OpenAI-compatible server, with tool calls and the thinking returned separately (`reasoning_content`): | |
| ```bash | |
| mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-4-8bit --port 8000 | |
| ``` | |
| ## Thinking Machines' results | |
| From the [Inkling 2 Mini card](https://huggingface.co/thinkingmachines/Inkling-2-Mini), for the original model (their | |
| harnesses and settings; a selection of their table): | |
| | Benchmark (Thinking Machines' figures) | Inkling 2 Mini | Qwen3.8-27B | Gemma 4 31B | | |
| |---|---|---|---| | |
| | Artificial Analysis Intelligence Index v5.0 | 20.1 | 20.1 | 4.4 | | |
| | SWE-bench Verified (bash only) | 79.2% | 77.8% | 46.1% | | |
| | SWE-bench Pro v2 Hard (mini-swe-agent) | 77.9% | 73.8% | 14.2% | | |
| | Terminal-Bench 2.1 (pi) | 71.6% | 73.0% | 43.4% | | |
| | BrowseComp (with compaction) | 79.0% | 61.6% | 37.0% | | |
| | MCP Atlas | 82.7% | 77.3% | 54.2% | | |
| | MMMU Pro | 74.9% | 76.3% | 73.4% | | |
| | OmniDocBench v1.6 | 92.8% | 91.5% | 82.5% | | |
| | LongBench v2 | 66.9% | 52.3% | 52.9% | | |
| | ARC-AGI v1 / v2 | 84.5 / 49.3% | 87.5 / 42.4% | – | | |
| ## What was checked | |
| On a MacBook Pro M5 Max (128 GB), 21 November 2026, with MLX 0.32.4 and [inkling2-mlx](https://github.com/internet-projects/inkling2-mlx) v0.1.0. | |
| - **Against Thinking Machines' code.** Their Transformers implementation (pull request #51050) ran on the CPU with the | |
| released weights, on 12 short chat texts (488 answer positions). This build gave the same next token at | |
| 98.6% of positions (mean KL 0.0018), and its greedy answers to 3 prompts matched theirs token for token (24 tokens each). Every tensor of this build is bit-identical to the release as MLX reads it. | |
| - **GSM8K, 50 fixed test questions** (reasoning `none`, greedy): 47/50. | |
| - **Tool calling** (reasoning `none`): an agent prompt with `bash` and `read` tools produced a well-formed call in Inkling's | |
| format, which `mlx_vlm.server` returns as OpenAI `tool_calls`. | |
| - **Coding:** 3/3 short Python tasks passed their asserts. **Thinking** (reasoning `low`): 2/2 | |
| questions answered correctly after one closed thinking block. **Vision:** named the red square and blue circle of a test image. | |
| - **Memory:** peak 26.9 GB with a 4K-token prompt and 27.8 GB with an 8K-token prompt (mlx-vlm defaults). With a smaller prefill step the peak drops: 22.3 / 22.6 GB at `prefill_step_size=512` and 21.4 / 21.6 GB at `prefill_step_size=256` (4K / 8K-token prompts, plus 0.64 GB if the DSpark draft is loaded), which fits the default GPU memory limit of a 36 GB Mac: pass `prefill_step_size=256` to `generate`, or `--prefill-step-size 256` to `mlx_vlm.server`. | |
| - **DSpark:** the bundled draft loads with this build and drafts about 7 tokens per round. | |
| Precision note: Thinking Machines released the routed experts in 4-bit NVFP4 (8-bit MXFP8 in the last four layers) and no higher-precision weights; MLX builds of this model keep the experts at that precision, so this 4/8-bit build already carries the full released precision. | |
| Known limitation of the current server: when the model calls a tool, `content` can also carry a short sentence of its | |
| reasoning and the tool name. | |
| ## License and credits | |
| Apache-2.0, as Inkling 2 Mini. Use of the model is also governed by Thinking Machines' | |
| [Model Acceptable Use Policy](https://thinkingmachines.ai/model-acceptable-use-policy/). `LICENSE` is the Apache-2.0 | |
| text and `NOTICE` describes the conversion. Inkling 2 Mini is by Thinking Machines Lab; please cite their work. | |
| Published by [NetProjects](https://netprojects.com) (Internet Projects Ltd). Not affiliated with Thinking Machines. | |
| More MLX builds from NetProjects: [huggingface.co/netprojects](https://huggingface.co/netprojects) · [netprojects.com](https://netprojects.com) | |