Inkling-mlx-2bit (2-bit, text backbone, BF16-sourced)

An MLX 2-bit build of the text backbone of Thinking Machines' Inkling (975B-total / 41B-active MoE), quantized directly from the BF16 checkpoint. The most compact build in the ladder - for multi-Mac distributed experiments.

This is created for community using a one Apple Mac Studio M3 Ultra with 512 GB.

Heads up

• Memory: ~329 GB on disk (routed experts at 2-bit, group size 64; attention / shared experts / embeddings / norms kept bf16). Loading needs roughly that much unified memory -> fits a 2x 192 GB Mac Studio distributed setup; does not fit a single Mac.
• 2-bit quality: experts are quantized hard; this is the lowest-quality rung. For better quality see the 3-bit / 4-bit siblings.
• Not verified yet: custom Inkling forward (factorized attention + short-conv + sigmoid MoE) is a from-reference reimplementation; logits not yet checked vs the original.
• Scope: text decoder only (no vision/audio).

Ladder

variant	bits	~size	fits
this	2	329 GB	2 Macs
Inkling-mlx-3bit	3	~454 GB	3 Macs
Inkling-mlx	4 (bf16 src)	~560 GB	3-4 Macs
Inkling-NVFP4-mlx	4 (nvfp4 src)	~581 GB	3-4 Macs

Usage (once a loader is available)

from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Inkling-mlx-2bit")
print(generate(model, tokenizer, prompt="The capital of France is", max_tokens=64))