Instructions to use Hygaard/Inkling-2-Small-oQ3e with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Hygaard/Inkling-2-Small-oQ3e with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("Hygaard/Inkling-2-Small-oQ3e")
config = load_config("Hygaard/Inkling-2-Small-oQ3e")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Hygaard/Inkling-2-Small-oQ3e with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Hygaard/Inkling-2-Small-oQ3e"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Hygaard/Inkling-2-Small-oQ3e"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use Hygaard/Inkling-2-Small-oQ3e with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Hygaard/Inkling-2-Small-oQ3e"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Hygaard/Inkling-2-Small-oQ3e

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Hygaard/Inkling-2-Small-oQ3e with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Hygaard/Inkling-2-Small-oQ3e"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Hygaard/Inkling-2-Small-oQ3e" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Inkling-2-Small oQ3e

oQ3e enhanced-calibrated MLX conversion of Thinking Machines’ Inkling-2-Small-MXFP8 for Apple Silicon.

Effective weight storage: 3.60 BPW · 268.10 GB · group size 64.

The full expert set, vision encoder/projector, original tokenizer and chat template are retained, along with the original BF16 multi-token-prediction (MTP) sidecar. All 48 text layers retain 384 routed experts and top-8 routing.

Other precision: oQ2e.

Conversion

Source: published MXFP8 checkpoint, thinkingmachines/Inkling-2-Small-MXFP8@086a047e8b7ff2ac902dd52355328ddb66b16914; converted directly, not requantized from oQ2e.
Format: MLX safetensors; enhanced oQ3e mixed precision, affine packing, group size 64.
Tooling: oMLX native Inkling 2/oQe converter, PR #5229.
Calibration: 384 samples of 512 tokens with authentic text/image inputs; full expert coverage and fresh level-3 sensitivity measurements.
Weights: 49 main shards plus the MTP sidecar, 268.10 GB / 249.69 GiB.

Conversion records are in CONVERSION.json.

Mixed precision

The packed quantized matrices average 3.055 bits per weight. The 3.60 effective BPW includes quantization metadata, unquantized tensors and safetensors headers.

Component	Weight precision	Original scalar parameters	Stored GB
Attention-residual controls	FP32	516,096	0.002064
MTP / draft	BF16	1,711,710,384	3.423421
Output head	8-bit	926,318,592	0.984214
Text attention	8-bit, BF16	4,713,811,968	4.862202
Text convolutions	BF16	2,162,688	0.004325
Text normalization	BF16	525,312	0.001051
Text routed experts	3-bit	587,068,342,272	256.842400
Text routers	FP32	84,953,088	0.339812
Token embeddings	8-bit	926,318,592	0.984214
Vision encoder	BF16	292,041,728	0.584083
Vision merger / projector	BF16	35,651,584	0.071303
Safetensors headers	—	—	0.000201
Total	3.60 effective BPW	595,762,352,304	268.099290

Stored GB uses decimal units. MTP is included; no tensors are counted twice.

Calculation: 8 × 268,099,290,020 stored bytes ÷ 595,762,352,304 original scalar parameters = 3.600084 BPW. Tokenizer/config files, KV cache and runtime workspace are excluded.

Per-block precision map

Block	Routed expert matrices	Other quantized matrices	Unquantized tensors	Stored GB
Text global	—	8-bit	BF16, FP32	1.968464
Text 0	3-bit	8-bit	BF16, FP32	5.459335
Text 1	3-bit	8-bit	BF16, FP32	5.459353
Text 2	3-bit	8-bit	BF16, FP32	5.459353
Text 3	3-bit	8-bit	BF16, FP32	5.459353
Text 4	3-bit	8-bit	BF16, FP32	5.459353
Text 5	3-bit	8-bit	BF16, FP32	5.459713
Text 6	3-bit	8-bit	BF16, FP32	5.459353
Text 7	3-bit	8-bit	BF16, FP32	5.459353
Text 8	3-bit	8-bit	BF16, FP32	5.459353
Text 9	3-bit	8-bit	BF16, FP32	5.459353
Text 10	3-bit	8-bit	BF16, FP32	5.459353
Text 11	3-bit	8-bit	BF16, FP32	5.459713
Text 12	3-bit	8-bit	BF16, FP32	5.459353
Text 13	3-bit	8-bit	BF16, FP32	5.459353
Text 14	3-bit	8-bit	BF16, FP32	5.459353
Text 15	3-bit	8-bit	BF16, FP32	5.459353
Text 16	3-bit	8-bit	BF16, FP32	5.459353
Text 17	3-bit	8-bit	BF16, FP32	5.459713
Text 18	3-bit	8-bit	BF16, FP32	5.459353
Text 19	3-bit	8-bit	BF16, FP32	5.459353
Text 20	3-bit	8-bit	BF16, FP32	5.459353
Text 21	3-bit	8-bit	BF16, FP32	5.459353
Text 22	3-bit	8-bit	BF16, FP32	5.459353
Text 23	3-bit	8-bit	BF16, FP32	5.459713
Text 24	3-bit	8-bit	BF16, FP32	5.459353
Text 25	3-bit	8-bit	BF16, FP32	5.459353
Text 26	3-bit	8-bit	BF16, FP32	5.459353
Text 27	3-bit	8-bit	BF16, FP32	5.459353
Text 28	3-bit	8-bit	BF16, FP32	5.459353
Text 29	3-bit	8-bit	BF16, FP32	5.459713
Text 30	3-bit	8-bit	BF16, FP32	5.459353
Text 31	3-bit	8-bit	BF16, FP32	5.459353
Text 32	3-bit	8-bit	BF16, FP32	5.459353
Text 33	3-bit	8-bit	BF16, FP32	5.459353
Text 34	3-bit	8-bit	BF16, FP32	5.459353
Text 35	3-bit	8-bit	BF16, FP32	5.459713
Text 36	3-bit	8-bit	BF16, FP32	5.459353
Text 37	3-bit	8-bit	BF16, FP32	5.459353
Text 38	3-bit	8-bit	BF16, FP32	5.459353
Text 39	3-bit	8-bit	BF16, FP32	5.459353
Text 40	3-bit	8-bit	BF16, FP32	5.459353
Text 41	3-bit	8-bit	BF16, FP32	5.459713
Text 42	3-bit	8-bit	BF16, FP32	5.459353
Text 43	3-bit	8-bit	BF16, FP32	5.459353
Text 44	3-bit	8-bit	BF16, FP32	5.459353
Text 45	3-bit	8-bit	BF16, FP32	5.459353
Text 46	3-bit	8-bit	BF16, FP32	5.459353
Text 47	3-bit	8-bit	BF16, FP32	5.459713
Vision global	—	—	BF16	0.075866
Vision 0	—	—	BF16	0.025197
Vision 1	—	—	BF16	0.025197
Vision 2	—	—	BF16	0.025197
Vision 3	—	—	BF16	0.025197
Vision 4	—	—	BF16	0.025197
Vision 5	—	—	BF16	0.025197
Vision 6	—	—	BF16	0.025197
Vision 7	—	—	BF16	0.025197
Vision 8	—	—	BF16	0.025197
Vision 9	—	—	BF16	0.025197
Vision 10	—	—	BF16	0.025197
Vision 11	—	—	BF16	0.025197
Vision 12	—	—	BF16	0.025197
Vision 13	—	—	BF16	0.025197
Vision 14	—	—	BF16	0.025197
Vision 15	—	—	BF16	0.025197
Vision 16	—	—	BF16	0.025197
Vision 17	—	—	BF16	0.025197
Vision 18	—	—	BF16	0.025197
Vision 19	—	—	BF16	0.025197
Vision 20	—	—	BF16	0.025197
Vision 21	—	—	BF16	0.025197
Vision 22	—	—	BF16	0.025197
MTP global	—	—	BF16	0.120264
MTP 0	—	—	BF16	0.660631
MTP 1	—	—	BF16	0.660631
MTP 2	—	—	BF16	0.660631
MTP 3	—	—	BF16	0.660631
MTP 4	—	—	BF16	0.660631

Complete per-module/tensor accounting: CSV and JSON.

Running

Requires oMLX’s native Inkling 2 implementation (PR #5229, tested at 2a18263b91e14ee92ed984f8c5a56d0053178cea). Native loader: omlx.patches.inkling2.vlm.load. Keep the included processor, tokenizer, chat template and complete tiktoken/ directory; generic mlx-lm loading is not supported.

Tested successfully on a 256 GB M5 Ultra Mac with paged MLX allocations; fully memory-resident loading did not succeed.

Credits and license

Model: Thinking Machines Lab. Conversion tooling: oMLX / oQe. Community conversion: Hygaard.

Licensed under Apache-2.0. See NOTICE for attribution and modification notes.

Downloads last month: -

Safetensors

Model size

594B params

Tensor type

U32

BF16

F32

MLX

Hardware compatibility

3-bit

Model tree for Hygaard/Inkling-2-Small-oQ3e

Base model

thinkingmachines/Inkling-2-Small-MXFP8

Quantized

(3)

this model