Molmo2-8B-mxfp4

This model was converted to MLX format from allenai/Molmo2-8B using mxlfp converter version 0.3.x.

Refer to the original model card for more details on the model.

Quantization

This is a 4-bit MXFP4 conversion:

• Quantization mode: mxfp4
• Bits: 4
• Group size: 32
• Converted with dtype float16

MXFP4 is a 4-bit floating-point format that uses a shared power-of-two scale per group of 32 weights. It differs from affine (integer) 4-bit quantization and from NVIDIA's NVFP4 format.

The quantization is applied per-module; the vision tower and multimodal projector are kept at higher precision, so the effective size is ~5.7 bits per weight (5.8 GB on disk) rather than exactly 4 bits per parameter.

Use with MLX

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "mlx-community/Molmo2-8B-mxfp4"

model, processor = load(model_path)
config = load_config(model_path)

image = ["path/to/image.jpg"]
prompt = "Describe this image."

formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=len(image)
)

output = generate(model, processor, formatted_prompt, image, verbose=False)
print(output.text)

CLI

mlx_vlm generate \
  --model mlx-community/Molmo2-8B-mxfp4 \
  --image path/to/image.jpg \
  --prompt "Describe this image." \
  --max-tokens 150