File size: 3,017 Bytes
1276f64 f07cd4c 1276f64 f07cd4c 1276f64 f07cd4c 1276f64 f07cd4c 1276f64 f07cd4c 1276f64 f07cd4c 1276f64 f07cd4c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | ---
license: apache-2.0
base_model: thinkingmachines/Inkling-Small
library_name: mlx
tags:
- mlx
- vision
- moe
pipeline_tag: image-text-to-text
---
# ToPo-ToPo/Inkling-Small-mlx-2bit
MLX **2bit** conversion of [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small)
for Apple Silicon (mlx-vlm). 276B total / 12B active sparse MoE (42 layers, 256 routed experts top-6 + 2 shared),
text + image + audio in, text out.
See also: [`ToPo-ToPo/Inkling-Small-mlx-4bit`](https://huggingface.co/ToPo-ToPo/Inkling-Small-mlx-4bit).
## Requires mlx-vlm >= 0.6.9
0.6.9 is the first release whose `models/inkling` can load an official Inkling checkpoint through the
public loader, and the first that implements the MoE `global_scale` / `gate.bias` tensors. On 0.6.7 / 0.6.8
this repo will not load.
```python
from mlx_vlm import load, generate
model, processor = load("ToPo-ToPo/Inkling-Small-mlx-2bit")
```
The config is the **official schema, unmodified** — no key translation and no loader patches are needed.
## Provenance (self-converted from official weights)
- Source: [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small) (license: apache-2.0, bf16, 531.9 GB)
- Tool: `mlx-vlm 0.6.9` — `mlx_vlm.convert --hf-path thinkingmachines/Inkling-Small --mlx-path . -q --q-bits 2 --q-group-size 64`
- Effective: **2.506 bits/weight** (77 GiB on disk)
- Only edit on top of the conversion: `pad_token` / `eos_token` added to `tokenizer_config.json`
(the official `TokenizersBackend` config sets neither, so transformers raises on any padded call).
Both point at existing ids — the vocabulary is unchanged.
## Reasoning effort
The chat template always injects a `Thinking effort level:` system message (default **0.9**). Control it
with the OpenAI-compatible `reasoning_effort` — `"none"` / `"minimal"` / `"low"` / `"medium"` / `"high"` /
`"max"`, or a float in `[0.0, 0.99]`. `"none"` disables thinking entirely.
When serving over `mlx_vlm.server`, note that Inkling wraps its answer in structural tokens
(`<|message_model|>`, `<|content_text|>`, `<|end_message|>`) which the server's fixed
`_CONTENT_MARKERS` list does not strip, and that its reasoning channel is
`<|content_thinking|>` … `<|end_message|><|message_model|>` rather than one of the built-in marker pairs.
Set `MLX_VLM_THINKING_START_TOKEN` / `MLX_VLM_THINKING_END_TOKEN` accordingly and strip the structural
tokens, or the reasoning and those markers end up in `content`.
## Revision history
- **2026-08-04** — reconverted with mlx-vlm 0.6.9. The previous upload had been converted with 0.6.7,
whose `models/inkling` did not implement the MoE `mlp.global_scale` (50 keys) and `mlp.gate.bias`
(40 keys) present in the official checkpoint, so **those tensors were silently dropped**. It also
shipped a translated config (renamed `intermediate_size` / `dense_intermediate_size`, etc.) that 0.6.9
rejects. If you pulled this repo before this date, re-download it.
|