majentik/gemma-4-e4b-it-MLX-bf16

Gemma-4-E4B-it MLX BF16

Unquantized bfloat16 MLX conversion of google/gemma-4-E4B-it for Apple Silicon inference with mlx-lm.

This repo is the plain 16-bit reference variant: no 8-bit, 4-bit, RotorQuant, TurboQuant, AWQ, GPTQ, or GGUF quantization is applied.

Provenance

Field	Value
Source model	google/gemma-4-E4B-it
Format	MLX safetensors
Weight dtype	bfloat16
Tensor check	665 tensors, all mlx.core.bfloat16
Local conversion tool	mlx-lm
License	Apache 2.0 / Gemma license terms from upstream

Conversion command:

mlx_lm.convert \
  --hf-path google/gemma-4-E4B-it \
  --mlx-path gemma-4-e4b-it-MLX-bf16 \
  --dtype bfloat16

Why BF16?

Gemma-4 is distributed natively in bfloat16. Keeping BF16 preserves the upstream numerical format while avoiding the quality/runtime tradeoffs of weight quantization.

Use with MLX

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("majentik/gemma-4-e4b-it-MLX-bf16")

messages = [{"role": "user", "content": "Explain Singapore's MRT system in one paragraph."}]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_dict=False,
)

response = generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True)
print(response)

Relationship to quantized variants

Use this repo when you want the unquantized BF16 reference decoder. For smaller/faster variants, use the existing quantized MLX repos under majentik, such as:

• majentik/gemma-4-E4B-RotorQuant-MLX-8bit
• majentik/gemma-4-e4b-it-mlx-4bit

Notes

• This is a format conversion of the upstream Gemma-4 E4B instruct model, not a fine-tune.
• The weights remain unquantized BF16.
• For licensing and acceptable use, follow the upstream Gemma terms linked above.