README.md

---
base_model: google/gemma-4-E2B-it-qat-q4_0-unquantized
library_name: mlx
pipeline_tag: image-text-to-text
license: apache-2.0
quantized_by: MichaelAnthony
tags:
- gemma4
- mlx
- mlx-vlm
- 6-bit
- quantized
- affine
- snowfox
---

# Gemma 4 E2B SnowFox MLX 6-bit (affine, group 64)

Standard MLX-VLM 6-bit affine weight quantization of the SnowFox model —
the MLX equivalent of GGUF `Q6_K`. This is a genuine MLX-VLM package
(quantized `safetensors` + `config.json` carrying a `quantization` field),
not a GGUF file or a renamed HF checkpoint.

SnowFox is a language-only LoRA merge based on Google's Gemma 4 E2B
instruction QAT-derived checkpoint. The image and audio towers were frozen
during fine-tuning and are retained here, together with the processor and
tokenizer needed by MLX-VLM.

## Exact lineage

- Base: [`google/gemma-4-E2B-it-qat-q4_0-unquantized`](https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-unquantized)
- Quantized from: [`MichaelAnthony/gemma4-e2b-Snowfox-hf`](https://huggingface.co/MichaelAnthony/gemma4-e2b-Snowfox-hf) (the canonical merged BF16 source)
- FP16 reference: [`MichaelAnthony/gemma4-e2b-Snowfox-MLX`](https://huggingface.co/MichaelAnthony/gemma4-e2b-Snowfox-MLX)
- Quantization: MLX affine, 6-bit, group size 64 (`{"group_size": 64, "bits": 6, "mode": "affine"}`)

## What is quantized

- **280 language-model layers** (`q/k/v/o` projections, MLP gate/up/down,
  the multimodal embedding projections, and the large embeddings) are 6-bit
  affine quantized: packed `uint32` `weight` (4 values per 3 bytes, low bits
  first) + float16 `scales`/`biases`.
- **The vision tower and audio tower are left in float16 (dense)** — matching
  MLX-VLM's `convert --quantize`, which skips multimodal modules. Their QAT
  `ClippableLinear` layers carry input/output clipping parameters
  (`input_max`/`input_min`/`output_max`/`output_min`) that must not be
  affine-quantized, so they stay dense and are loaded as regular `nn.Linear`.
- The dense per-layer input embedding (`embed_tokens_per_layer`) **is**
  quantized here, so the language model stays compact without exceeding the
  Metal buffer cap.

## Package contents

- `model-00001-of-00001.safetensors` (4,708,626,430 bytes): the 6-bit MLX model
  in a single shard (~4.71 GB total).
- `model.safetensors.index.json`: complete shard map.
- `config.json` (with `quantization` + `quantization_config`), `generation_config.json`,
  `processor_config.json`, tokenizer files, and `chat_template.jinja`.

## Model size vs HF parameter display

This is a **~5.1B-parameter** model (2.3B effective), identical to the source
SnowFox checkpoint. Hugging Face's model page reports ~1.49B because the 6-bit
weights are stored as *packed* `uint32` words (each holds multiple 6-bit values)
and HF counts each packed word as one parameter. The packed word count is a
storage detail, not the parameter count.

## Verification performed

The conversion host has no Apple-Silicon MLX runtime, so the quantized package
was structurally validated before upload:

- 1,951 source tensors mapped with no missing or extra keys; 280 language-model
  layers quantized; vision/audio towers left dense.
- Quantized weight format matches the MLX affine contract: 6-bit values packed
  4-per-3-bytes (24-bit little-endian word), dequantization `scale * q + bias`,
  group 64.
- Round-trip dequantization of sampled layers reproduces the source weights to
  within 6-bit precision.

**Apple-Silicon MLX-VLM inference has not been run.** Treat this as a
structurally validated quantization pending a real Apple-Silicon text / image /
audio smoke test.

## Run on Apple Silicon

Use full MLX-VLM (not text-only MLX-LM) — Gemma 4 E2B includes image and audio:

```bash
python -m pip install "mlx-vlm==0.6.13"

python -m mlx_vlm.generate \
  --model MichaelAnthony/gemma4-e2b-Snowfox-MLX-6bit \
  --max-tokens 128 \
  --temperature 0.0 \
  --prompt "Explain what SnowFox is in one sentence."
```

Add `--image /path/to/image.png` for image prompting.

## License

Gemma 4 is Apache-2.0. This derivative package uses the Apache-2.0 license
declared by the pinned base model.