README.md

---
base_model: google/gemma-4-12B
license: gemma
library_name: mlx
pipeline_tag: image-text-to-text
tags:
  - mlx
  - quantized
  - gemma
  - gemma4
  - image-text-to-text
  - apple-silicon
  - rotorquant
---

# gemma-4-12B — RotorQuant MLX 8bit

[`google/gemma-4-12B`](https://huggingface.co/google/gemma-4-12B) @ `023679ed352de9bb66cc873c9009ce3482585c08` quantized pack, published as `majentik/gemma-4-12B-RotorQuant-MLX-8bit`.

## Method

MLX quantization via mlx-vlm 0.6.3 (8bit, group_size 64); vision + audio towers retained in BF16 (not quantized).

## Release line

Released under the **RotorQuant** line. RotorQuant and TurboQuant are this project's release labels for this pack, not distinct quantization algorithms — both brand repos for a given tier carry byte-identical weights, produced once and published under two names. No brand-specific speedup is claimed or measured for either label.

## Modality

This pack is `image-text-to-text` capable: the vision and audio towers ship in BF16 alongside the quantized text tower, so image (and audio) inputs are supported end to end via `mlx-vlm`.

**Base-model note:** this is the raw completion model (not instruction-tuned). It is not instruction-aligned for multimodal Q&A — expect free-form continuation behavior rather than chat-style image description, even though the vision/audio towers are present.

**mlx-vlm suppress-tokens note:** the base model's `generation_config.json` carries `suppress_tokens` for six multimodal placeholder token ids so that `transformers` generation is clean. `mlx-vlm` does not honor `suppress_tokens` automatically, so text-only generation with `mlx-vlm` can leak placeholder tokens (observed as a degenerate `A<image|>A<image|>...` loop in smoke testing) unless you suppress them yourself:

```python
from mlx_vlm import load, generate

model, processor = load("majentik/gemma-4-12B-RotorQuant-MLX-8bit")
suppress = {255999: -1e9, 256000: -1e9, 258880: -1e9, 258881: -1e9, 258882: -1e9, 258883: -1e9}
output = generate(
    model, processor, prompt,
    max_tokens=256,
    logit_bias=suppress,
)
```

## License

Governed by the [Gemma Terms of Use](https://ai.google.dev/gemma/terms). See the upstream repo for the full license text.