File size: 2,703 Bytes
ee7f971 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | ---
license: apache-2.0
library_name: mlx
pipeline_tag: text-generation
base_model:
- google/gemma-4-E4B-it
tags:
- mlx
- gemma4
- text-only
- 4-bit
language:
- en
- ar
- ckb
---
# Gemma 4 E4B Text for MLX (4-bit)
[google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) as a **text-only** model for Apple's [MLX](https://github.com/ml-explore/mlx), compressed to 4 bits. It runs on device on Apple silicon: Mac, and iPhone or iPad with 8 GB of memory.
Gemma 4 E4B can also read images and audio. This version keeps only the language model, so it is **4.2 GB instead of 5.2 GB** and writes exactly the same text.
All credit for the model goes to Google. This repository only changes its file format.
## At a glance
| Detail | Value |
|---|---|
| Base model | google/gemma-4-E4B-it (instruction-tuned) |
| Input and output | Text only |
| Version | 4-bit: weights compressed from 16 to 4 bits (affine, groups of 64) |
| Size | 4.2 GB (the multimodal 4-bit MLX version is 5.2 GB) |
| Model type | `gemma4_text` |
| Runtime | [mlx-lm](https://github.com/ml-explore/mlx-lm) (Python) or [mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm) |
| License | Apache 2.0, the same as the original model |
## Usage
```bash
pip install -U mlx-lm
```
```python
from mlx_lm import load, generate
model, tokenizer = load("UIDUser-NSB/Gemma4-E4B-Text-MLX")
messages = [{"role": "user", "content": "Summarize this meeting in three points: ..."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```
Pass `enable_thinking=False` for a direct answer. Without it, Gemma 4 writes its reasoning first.
## Check
With greedy decoding, this model and [mlx-community/gemma-4-e4b-it-4bit](https://huggingface.co/mlx-community/gemma-4-e4b-it-4bit) (the multimodal version) write the same text for the same prompt.
## How it was converted
1. **Source:** google/gemma-4-E4B-it, revision `ee0ef6023621cff504d758262d4e04895a5af4a2`.
2. **Text only:** the language model's weights are kept and renamed from `model.language_model.*` to `model.*`. The vision and audio encoders and their embedders are removed.
3. **Config:** `model_type` is `gemma4_text`. The text model's settings (`text_config`) are also written at the top level, where mlx-lm and mlx-swift-lm read them for this model type.
4. **Compression:** `python -m mlx_lm convert -q --q-bits 4 --q-group-size 64` with mlx-lm 0.32.0.
## Credits
Gemma 4 by Google DeepMind. See the [original model card](https://huggingface.co/google/gemma-4-E4B-it) for training data, evaluations, intended use and limitations.
|