Inkling 2 Mini — MLX

MLX conversion of thinkingmachines/Inkling-2-Mini, preserving its native mixed precision. The checkpoint accepts text and images and generates text.

The first 20 MoE layers retain NVFP4 expert weights; the last four retain MXFP8. Other parameters retain their original BF16/FP32 dtypes. The 24 physical decoder layers execute 48 layer visits through the model's three-pass middle loop. The original bundled DSpark draft is included as mtp.safetensors.

Target weights: 19.53 GB across four shards. Bundled draft: 0.64 GB. The upstream model recommends Apple silicon with 32 GB or more memory; longer contexts need additional memory.

Required runtime

This release includes a patched MLX-VLM build with Inkling 2 Mini loop support. Use this build for the examples below; the upstream base revision did not yet support inkling2_loop.

python -m pip install --upgrade "https://huggingface.co/nativ-community/Inkling-2-Mini-nvfp4/resolve/main/runtime/mlx_vlm-0.8.0+inkling2mini-py3-none-any.whl"

The wheel is based on MLX-VLM commit 392052b4439c. Its source changes are available in runtime/inkling2-mini.patch. The wheel contains the Python runtime source, and its code retains the MLX-VLM license and upstream notices.

Text generation

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model_id = "nativ-community/Inkling-2-Mini-nvfp4"
model, processor = load(model_id)
prompt = apply_chat_template(
    processor, model.config,
    "Explain in one sentence why ice floats on water.",
    reasoning_effort="none",
)
result = generate(
    model, processor, prompt,
    max_tokens=256, temperature=0, prefill_step_size=128,
)
print(result.text)

The model may still emit reasoning when reasoning_effort="none". Its message tokens distinguish reasoning from the final answer; stripping all special tokens concatenates these sections.

Image input

prompt = apply_chat_template(
    processor, model.config, "Describe this image.",
    num_images=1, reasoning_effort="none",
)
result = generate(
    model, processor, prompt, image=["image.png"],
    max_tokens=256, temperature=0, prefill_step_size=128,
)
print(result.text)

Bundled speculative draft

from mlx_vlm.speculative.drafters import load_drafter

draft, kind = load_drafter(model_id)
result = generate(
    model, processor, prompt, image=["image.png"],
    draft_model=draft, draft_kind=kind,
    max_tokens=256, temperature=0, prefill_step_size=128,
)
print(result.text)

Validation and provenance

  • Strict checkpoint reload succeeded; all 1,271 target tensors are exactly equal to the sanitized source tensors, including packed weights, scales, and floating-point parameters.
  • 67 targeted tests passed, covering native and converted loading, mixed precision, loop caches, image inputs, and speculative generation/reset.
  • The loop message gates and message attention were compared with the upstream Transformers implementation on small FP32 tensors; maximum absolute errors were below 4e-7.
  • Real-weight smoke checks on an Apple M3 Ultra correctly answered arithmetic, explained ice buoyancy, and recognized a red circle. Standard and bundled-draft arithmetic generation produced the same 45 greedy tokens.

These checks establish conversion integrity and basic inference behavior. They do not establish full-model numerical parity with the original runtime or benchmark accuracy. MLX-VLM uses floating-point activations rather than the reference runtime's activation quantization. The advertised 1M context window was not exercised in these checks.

Source revision: 6e7c8248f75265bdd4fbab5273304243b5d8c2cb. See conversion.json, validation.json, and the upstream model card for architecture, training, evaluations, limitations, and licensing.

Reproduce the conversion

With the included runtime installed:

python -m mlx_vlm convert \
  --hf-path thinkingmachines/Inkling-2-Mini \
  --revision 6e7c8248f75265bdd4fbab5273304243b5d8c2cb \
  --mlx-path Inkling-2-Mini-nvfp4

Copy the original mtp.safetensors into the output folder to retain the bundled draft. This conversion does not requantize the source experts.

Downloads last month
-
Safetensors
Model size
26B params
Tensor type
BF16
·
F32
·
U32
·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for nativ-community/Inkling-2-Mini-nvfp4

Quantized
(20)
this model