Instructions to use ToPo-ToPo/Inkling-Small-mlx-2bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ToPo-ToPo/Inkling-Small-mlx-2bit with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("ToPo-ToPo/Inkling-Small-mlx-2bit")
config = load_config("ToPo-ToPo/Inkling-Small-mlx-2bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use ToPo-ToPo/Inkling-Small-mlx-2bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "ToPo-ToPo/Inkling-Small-mlx-2bit"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ToPo-ToPo/Inkling-Small-mlx-2bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use ToPo-ToPo/Inkling-Small-mlx-2bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "ToPo-ToPo/Inkling-Small-mlx-2bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ToPo-ToPo/Inkling-Small-mlx-2bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Hermes Agent

How to use ToPo-ToPo/Inkling-Small-mlx-2bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "ToPo-ToPo/Inkling-Small-mlx-2bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ToPo-ToPo/Inkling-Small-mlx-2bit

Run Hermes

hermes

Atomic Chat

Inkling-Small-mlx-2bit / README.md

ToPo-ToPo

Reconvert with mlx-vlm 0.6.9 (official config schema; restores MoE global_scale/gate.bias dropped by the 0.6.7 conversion)

f07cd4c verified 10 days ago

preview code

Raw

History Blame Contribute Delete

3.02 kB

	---
	license: apache-2.0
	base_model: thinkingmachines/Inkling-Small
	library_name: mlx
	tags:
	- mlx
	- vision
	- moe
	pipeline_tag: image-text-to-text
	---

	# ToPo-ToPo/Inkling-Small-mlx-2bit

	MLX 2bit conversion of [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small)
	for Apple Silicon (mlx-vlm). 276B total / 12B active sparse MoE (42 layers, 256 routed experts top-6 + 2 shared),
	text + image + audio in, text out.

	See also: [`ToPo-ToPo/Inkling-Small-mlx-4bit`](https://huggingface.co/ToPo-ToPo/Inkling-Small-mlx-4bit).

	## Requires mlx-vlm >= 0.6.9

	0.6.9 is the first release whose `models/inkling` can load an official Inkling checkpoint through the
	public loader, and the first that implements the MoE `global_scale` / `gate.bias` tensors. On 0.6.7 / 0.6.8
	this repo will not load.

	```python
	from mlx_vlm import load, generate
	model, processor = load("ToPo-ToPo/Inkling-Small-mlx-2bit")
	```

	The config is the official schema, unmodified — no key translation and no loader patches are needed.

	## Provenance (self-converted from official weights)
	- Source: [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small) (license: apache-2.0, bf16, 531.9 GB)
	- Tool: `mlx-vlm 0.6.9` — `mlx_vlm.convert --hf-path thinkingmachines/Inkling-Small --mlx-path . -q --q-bits 2 --q-group-size 64`
	- Effective: 2.506 bits/weight (77 GiB on disk)
	- Only edit on top of the conversion: `pad_token` / `eos_token` added to `tokenizer_config.json`
	(the official `TokenizersBackend` config sets neither, so transformers raises on any padded call).
	Both point at existing ids — the vocabulary is unchanged.

	## Reasoning effort

	The chat template always injects a `Thinking effort level:` system message (default 0.9). Control it
	with the OpenAI-compatible `reasoning_effort` — `"none"` / `"minimal"` / `"low"` / `"medium"` / `"high"` /
	`"max"`, or a float in `[0.0, 0.99]`. `"none"` disables thinking entirely.

	When serving over `mlx_vlm.server`, note that Inkling wraps its answer in structural tokens
	(`<\|message_model\|>`, `<\|content_text\|>`, `<\|end_message\|>`) which the server's fixed
	`_CONTENT_MARKERS` list does not strip, and that its reasoning channel is
	`<\|content_thinking\|>` … `<\|end_message\|><\|message_model\|>` rather than one of the built-in marker pairs.
	Set `MLX_VLM_THINKING_START_TOKEN` / `MLX_VLM_THINKING_END_TOKEN` accordingly and strip the structural
	tokens, or the reasoning and those markers end up in `content`.

	## Revision history

	- 2026-08-04 — reconverted with mlx-vlm 0.6.9. The previous upload had been converted with 0.6.7,
	whose `models/inkling` did not implement the MoE `mlp.global_scale` (50 keys) and `mlp.gate.bias`
	(40 keys) present in the official checkpoint, so those tensors were silently dropped. It also
	shipped a translated config (renamed `intermediate_size` / `dense_intermediate_size`, etc.) that 0.6.9
	rejects. If you pulled this repo before this date, re-download it.