README.md

---
tags:
- mlx
- apple-silicon
- text-generation
- gemma3
library_name: mlx-lm
pipeline_tag: text-generation
base_model: Daizee/Gemma3-Callous-Calla-4B
---

# Gemma3-Callous-Calla-4B — **MLX** builds (Apple Silicon)

This repo hosts **MLX-converted** variants of **Daizee/Gemma3-Callous-Calla-4B** for fast, local inference on Apple Silicon (M-series).  
Tokenizer/config are included at the repo root. MLX weight folders live under `mlx/`.

> **Note on vocab padding:** For MLX compatibility, the tokenizer/embeddings were padded to the next multiple of 64 tokens.  
> In this build: **262,208 tokens** (added 64 placeholder tokens named `<pad_ex_*>`).

## Variants

| Path         | Bits | Group Size | Notes                              |
|--------------|------|------------|------------------------------------|
| `mlx/g128/`  | int4 | 128        | Smallest & fastest                 |
| `mlx/g64/`   | int4 | 64         | Slightly larger, often steadier    |
| `mlx/int8/`  | 8    | —          | Closest to fp16 quality (slower)   |

## Quickstart (MLX-LM)

### Run from Hugging Face (no cloning needed)
```bash
python -m mlx_lm.generate \
  --model hf://Daizee/Gemma3-Callous-Calla-4B-mlx/mlx/g64 \
  --prompt "Summarize the Bill of Rights for 7th graders in 4 bullet points." \
  --max-tokens 180 --temp 0.3 --top-p 0.92