Inkling-2-Mini-MLX-8bit

MLX conversion of thinkingmachines/Inkling-2-Mini.

  • The routed experts keep the original checkpoint's native NVFP4 weights.
  • In this variant, the attention/embedding/dense layers quantized to 8-bit (affine, group size 64).
  • Converted with mlx-lm from the open Inkling 2 Mini support PR (ml-explore/mlx-lm#2086). Released mlx-lm versions up to 0.32.2 do not load this model yet.
  • Text generation was smoke-tested. The vision tower was not tested.
pip install git+https://github.com/ml-explore/mlx-lm.git@refs/pull/2086/head
python -m mlx_lm generate --model pipenetwork/Inkling-2-Mini-MLX-8bit --prompt "Hello"

License: Apache 2.0, inherited from the base model.

Downloads last month
-
Safetensors
Model size
26B params
Tensor type
U32
·
F32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for pipenetwork/Inkling-2-Mini-MLX-8bit

Quantized
(20)
this model

Collection including pipenetwork/Inkling-2-Mini-MLX-8bit