mixer3d/translategemma-4b-it-gguf

translategemma-4b-it-q5_k_m.gguf

This repo contains GGUF weights for google/translategemma-4b-it.

Quantization Details

• Method: llama-quantize
• Llama.cpp Version: 7770 (fe44d3557)
• Original Model Precision: BF16

Files Provided

File	Quant Method	Size	Description
translategemma-4b-it-q5_k_m.gguf	Q5_K_M	3.2 GB	High quality, recommended for most uses.

Usage

You can use these models with llama.cpp

./llama-server -m translategemma-4b-it-q5_k_m.gguf -no-mmap -ngl 99 --port 8080 -c 8192 -fa 1 --jinja