Agisight/tyv-gemma4-e4b-mlx-4bit

Gemma 4 E4B — Russian ↔ Tuvan translation, MLX 4-bit

google/gemma-4-E4B-it fine-tuned for bidirectional Russian ↔ Tuvan translation (LoRA adapter: Agisight/tyv-gemma4-e4b-lora), merged into the base weights and quantized to 4 bits for Apple Silicon with mlx-vlm. Runs fully offline on a Mac.

• Size: 4.9 GB (5.2 bits per weight on average; some layers are kept at higher precision).
• Speed on a MacBook Air M4 (16 GB): 0.53 s per sentence (median), 5.1 GB peak memory.
• Other variant: Agisight/tyv-gemma4-e4b-mlx-8bit.

Results

Clean held-out test set, 1,369 pairs (NLLB v3 test split of Agisight/tyv-rus-200k with training duplicates removed). chrF++, greedy decoding.

Model	ru→tyv	tyv→ru
NLLB v3 (Agisight/nllb-rus-tyv-v3-dict_16.5k)	49.2	49.1
Fine-tuned Gemma 4 E4B, bf16 (adapter)	50.5	50.9
This 4-bit MLX model (first 100 pairs only; bf16 on the same pairs: 51.1 / 49.6)	48.2	44.3

Not recommended. 4-bit quantization noticeably hurts this model: on the first 100 test pairs it loses 2.9 / 5.3 chrF++ against bf16. Use Agisight/tyv-gemma4-e4b-pruned-mlx-6bit — 3.2 GB with the full model's quality.

The bf16 adapter significantly outperforms NLLB v3 in both directions (paired bootstrap, p < 0.05) and also on BLEU, normalized chrF++ and COMET — see the adapter card for the full table.

Prompt format

Single-turn chat message with a short prefix, exactly as in training:

• Russian → Tuvan: ru→tyv: <Russian text>
• Tuvan → Russian: tyv→ru: <Tuvan text>

Usage

pip install -U mlx-vlm
mlx_vlm.generate --model Agisight/tyv-gemma4-e4b-mlx-4bit \
  --prompt "ru→tyv: Завтра я поеду в Кызыл к родителям." --max-tokens 64 --temperature 0.0

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("Agisight/tyv-gemma4-e4b-mlx-4bit")
prompt = apply_chat_template(processor, model.config, "ru→tyv: Завтра я поеду в Кызыл к родителям.")
print(generate(model, processor, prompt, max_tokens=64, temperature=0.0))

Notes

• The vision and audio towers of Gemma 4 are kept, but the model was trained for text translation only.
• On iPhone, 4.9 GB exceeds Apple's ~2 GB guidance for on-device models; a pruned-vocabulary version for iOS is planned.

Code

Training notebook (Colab), data preparation, conversion and evaluation scripts: Agisight/tyv-translation-mlx.

Credits

• Base model: Google, Gemma 4 E4B.
• Data, fine-tuning, evaluation and MLX conversion: Ali Kuzhuget.