---
tags:
- gemma3
- xnnpack
---

# Gemma 3 4B-IT (ExecuTorch, XNNPACK, 8da4w)

This folder contains an ExecuTorch .pte export of https://huggingface.co/google/gemma-3-4b-it for CPU inference via the XNNPACK backend, with post-training quantization enabled. Gemma 3 is a multimodal vision-language model     
supporting both text and image inputs.

Quantization

- --qlinear 8da4w: linear layers (text decoder) use 8-bit dynamic activations + 4-bit weights
- --qlinear_encoder 8da4w: linear layers (vision encoder) use 8-bit dynamic activations + 4-bit weights
- --qembedding 8w: embeddings use 8-bit weights

## Export
```
optimum-cli export executorch \
  --model "google/gemma-3-4b-it" \
  --task "multimodal-text-to-text" \
  --recipe "xnnpack" \
  --use_custom_sdpa \
  --use_custom_kv_cache \
  --qlinear "8da4w" \
  --qlinear_encoder "8da4w" \
  --qembedding "8w" \
  --output_dir "<output_dir>"
```

## Run
Build the runner from the ExecuTorch repo root
```
make gemma3-cpu
```
Download tokenizer
```
curl -L https://huggingface.co/google/gemma-3-4b-it/resolve/main/tokenizer.json -o tokenizer.json
```
Run model
```
./cmake-out/examples/models/gemma3/gemma3_e2e_runner \
  --model_path "model.pte" \
  --tokenizer_path "tokenizer.json" \
  --image_path "docs/source/_static/img/et-logo.png" \
  --temperature 0
```