Instructions to use Hygaard/Inkling-2-Small-oQ2e with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Hygaard/Inkling-2-Small-oQ2e with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("Hygaard/Inkling-2-Small-oQ2e") config = load_config("Hygaard/Inkling-2-Small-oQ2e") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use Hygaard/Inkling-2-Small-oQ2e with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Hygaard/Inkling-2-Small-oQ2e"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Hygaard/Inkling-2-Small-oQ2e" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use Hygaard/Inkling-2-Small-oQ2e with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Hygaard/Inkling-2-Small-oQ2e"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Hygaard/Inkling-2-Small-oQ2e
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Hygaard/Inkling-2-Small-oQ2e with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Hygaard/Inkling-2-Small-oQ2e"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Hygaard/Inkling-2-Small-oQ2e" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download README.md from Hygaard/Inkling-2-Small-oQ2e: direct link, hf CLI and curl.
- Browser
- Download file 7.7 kB
-
https://huggingface.co/Hygaard/Inkling-2-Small-oQ2e/resolve/main/README.md
- Command line
-
hf download hf://Hygaard/Inkling-2-Small-oQ2e/README.md
-
curl -L -o README.md https://huggingface.co/Hygaard/Inkling-2-Small-oQ2e/resolve/main/README.md
license: apache-2.0
base_model: thinkingmachines/Inkling-2-Small-MXFP8
base_model_relation: quantized
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- omlx
- inkling2
- oq2e
- quantized
- image-text-to-text
Inkling-2-Small oQ2e
oQ2e mixed-precision MLX conversion of Thinking Machinesβ Inkling-2-Small-MXFP8 for Apple Silicon.
Effective weight storage: 2.61 BPW Β· 194.72 GB Β· group size 64.
The full expert set, vision encoder/projector, original tokenizer and chat template are retained, along with the original BF16 multi-token-prediction (MTP) sidecar. All 48 text layers retain 384 routed experts and top-8 routing.
Other precision: mixed 2/3-bit oQe (3.07 BPW), preferred for 256 GB Apple Silicon hardware.
Conversion
- Source: published MXFP8 checkpoint,
thinkingmachines/Inkling-2-Small-MXFP8@086a047e8b7ff2ac902dd52355328ddb66b16914. - Format: MLX safetensors; oQ2e mixed precision, affine packing, group size 64.
- Tooling: oMLX native Inkling 2/oQe converter, PR #5229.
- Calibration: 384 samples of 512 tokens with text/image inputs; full expert coverage.
- Weights: 33 main shards plus the MTP sidecar, 194.72 GB / 181.34 GiB.
Conversion records are in CONVERSION.json.
Mixed precision
The packed quantized matrices average 2.066 bits per weight. The 2.61 effective BPW includes quantization metadata, unquantized tensors and safetensors headers.
| Component | Weight precision | Original scalar parameters | Stored GB |
|---|---|---|---|
| Attention-residual controls | FP32 | 516,096 | 0.002064 |
| MTP / draft | BF16 | 1,711,710,384 | 3.423421 |
| Output head | 8-bit | 926,318,592 | 0.984214 |
| Text attention | 8-bit, BF16 | 4,713,811,968 | 4.862202 |
| Text convolutions | BF16 | 2,162,688 | 0.004325 |
| Text normalization | BF16 | 525,312 | 0.001051 |
| Text routed experts | 2-bit | 587,068,342,272 | 183.458857 |
| Text routers | FP32 | 84,953,088 | 0.339812 |
| Token embeddings | 8-bit | 926,318,592 | 0.984214 |
| Vision encoder | BF16 | 292,041,728 | 0.584083 |
| Vision merger / projector | BF16 | 35,651,584 | 0.071303 |
| Safetensors headers | β | β | 0.000201 |
| Total | 2.61 effective BPW | 595,762,352,304 | 194.715747 |
Stored GB uses decimal units. MTP is included; no tensors are counted twice.
Calculation: 8 Γ 194,715,747,050 stored bytes Γ· 595,762,352,304 original scalar parameters = 2.614677 BPW. Tokenizer/config files, KV cache and runtime workspace are excluded.
Per-block precision map
| Block | Routed expert matrices | Other quantized matrices | Unquantized tensors | Stored GB |
|---|---|---|---|---|
| Text global | β | 8-bit | BF16, FP32 | 1.968464 |
| Text 0 | 2-bit | 8-bit | BF16, FP32 | 3.930511 |
| Text 1 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 2 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 3 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 4 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 5 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Text 6 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 7 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 8 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 9 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 10 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 11 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Text 12 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 13 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 14 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 15 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 16 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 17 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Text 18 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 19 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 20 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 21 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 22 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 23 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Text 24 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 25 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 26 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 27 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 28 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 29 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Text 30 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 31 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 32 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 33 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 34 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 35 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Text 36 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 37 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 38 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 39 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 40 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 41 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Text 42 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 43 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 44 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 45 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 46 | 2-bit | 8-bit | BF16, FP32 | 3.930530 |
| Text 47 | 2-bit | 8-bit | BF16, FP32 | 3.930889 |
| Vision global | β | β | BF16 | 0.075866 |
| Vision 0 | β | β | BF16 | 0.025197 |
| Vision 1 | β | β | BF16 | 0.025197 |
| Vision 2 | β | β | BF16 | 0.025197 |
| Vision 3 | β | β | BF16 | 0.025197 |
| Vision 4 | β | β | BF16 | 0.025197 |
| Vision 5 | β | β | BF16 | 0.025197 |
| Vision 6 | β | β | BF16 | 0.025197 |
| Vision 7 | β | β | BF16 | 0.025197 |
| Vision 8 | β | β | BF16 | 0.025197 |
| Vision 9 | β | β | BF16 | 0.025197 |
| Vision 10 | β | β | BF16 | 0.025197 |
| Vision 11 | β | β | BF16 | 0.025197 |
| Vision 12 | β | β | BF16 | 0.025197 |
| Vision 13 | β | β | BF16 | 0.025197 |
| Vision 14 | β | β | BF16 | 0.025197 |
| Vision 15 | β | β | BF16 | 0.025197 |
| Vision 16 | β | β | BF16 | 0.025197 |
| Vision 17 | β | β | BF16 | 0.025197 |
| Vision 18 | β | β | BF16 | 0.025197 |
| Vision 19 | β | β | BF16 | 0.025197 |
| Vision 20 | β | β | BF16 | 0.025197 |
| Vision 21 | β | β | BF16 | 0.025197 |
| Vision 22 | β | β | BF16 | 0.025197 |
| MTP global | β | β | BF16 | 0.120264 |
| MTP 0 | β | β | BF16 | 0.660631 |
| MTP 1 | β | β | BF16 | 0.660631 |
| MTP 2 | β | β | BF16 | 0.660631 |
| MTP 3 | β | β | BF16 | 0.660631 |
| MTP 4 | β | β | BF16 | 0.660631 |
Complete per-module/tensor accounting: CSV and JSON.
Running
Requires oMLXβs native Inkling 2 implementation (PR #5229, tested at 2a18263b91e14ee92ed984f8c5a56d0053178cea). Native loader: omlx.patches.inkling2.vlm.load. Keep the included processor, tokenizer, chat template and complete tiktoken/ directory; generic mlx-lm loading is not supported.
Cold loading, text and image generation tested on a 256 GB M5 Ultra. Main-model MLX allocator peak: 179.32 GiB. Runtime and context state need additional memory. MTP decoding, broader quality and full-context capacity remain untested.
Credits and license
Model: Thinking Machines Lab. Conversion tooling: oMLX / oQe. Community conversion: Hygaard.
Licensed under Apache-2.0. See NOTICE for attribution and modification notes.