README.md

metadata

pipeline_tag: image-text-to-text
license: gemma
base_model: google/gemma-3n-E4B
library_name: kerasformers
extra_gated_heading: Access Gemma on Hugging Face
extra_gated_prompt: >-
  To access Gemma on Hugging Face, you're required to review and agree to
  Google's usage license. To do this, please ensure you're logged in to Hugging
  Face and click below. Requests are processed immediately.
extra_gated_button_content: Acknowledge license
license_link: https://ai.google.dev/gemma/terms
language:
  - en
tags:
  - keras
  - kerasformers
  - gemma3n
  - gemma-3n
  - image-text-to-text
  - audio-text-to-text
  - multimodal
  - pytorch
  - jax
  - tf

See our collection for all versions of Gemma 3n.

Run Gemma 3n with Keras 3: JAX, PyTorch, or TensorFlow

kerasformers/gemma-3n-e4b

Gemma 3n is Google's on-device multimodal (image + audio + text) model. Its decoder layers several on-device innovations on the Gemma shape: AltUp (alternating updates over parallel hidden streams), LAuReL (learned augmented residuals), MatFormer (nested per-layer widths), Per-Layer Embeddings, and activation sparsity, with tail KV-sharing and a 5:1 sliding/global attention schedule. Vision is a MobileNet-V5 encoder and audio a USM conformer, both feeding soft tokens into the decoder. Base checkpoints are for completion; -it variants are instruction-tuned.

For more details, see Google's original model card.

Pure-Keras 3 conversion of google/gemma-3n-E4B for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a base (pretrained) checkpoint, for completion / fine-tuning.

✨ Quick start

Text-only

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from kerasformers.models.gemma3n import Gemma3nTextGenerate, Gemma3nTokenizer

model = Gemma3nTextGenerate.from_weights("kerasformers/gemma-3n-e4b")
tokenizer = Gemma3nTokenizer.from_weights("kerasformers/gemma-3n-e4b")

inputs = tokenizer([{"role": "user", "content": "Hello, who are you?"}])
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0]))

Image + text

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from kerasformers.models.gemma3n import Gemma3nConditionalGenerate, Gemma3nProcessor

model = Gemma3nConditionalGenerate.from_weights("kerasformers/gemma-3n-e4b")
processor = Gemma3nProcessor.from_weights("kerasformers/gemma-3n-e4b")

inputs = processor(conversation=[
    {"role": "user", "content": [
        {"type": "image", "image": Image.open("cat.jpg")},
        {"type": "text", "text": "Describe this image in one sentence."},
    ]}
])
outputs = model.generate(**inputs, max_new_tokens=64)
print(processor.decode(outputs[0]))

All Gemma 3n variants load the same way with from_weights("kerasformers/<variant>"):

Variant	Hub	Type
gemma-3n-e2b	kerasformers/gemma-3n-e2b	multimodal / base
gemma-3n-e2b-it	kerasformers/gemma-3n-e2b-it	multimodal / instruct
gemma-3n-e4b	kerasformers/gemma-3n-e4b	multimodal / base
gemma-3n-e4b-it	kerasformers/gemma-3n-e4b-it	multimodal / instruct

Tips

• Set KERAS_BACKEND before importing Keras / kerasformers.
• Loads in bfloat16 by default (the weights are bf16). Pass load_dtype="float32" for full precision, or quantization="int8" to shrink further.
• Gemma 3n is audio-capable too: pass audio content items in the conversation to transcribe / reason over speech.
• See Gemma 3n docs and Loading Weights.
• Upstream safetensors still work via the hf: prefix, e.g. Gemma3nConditionalGenerate.from_weights("hf:google/gemma-3n-E4B").

Special Thanks

A huge thank you to the Google Gemma authors for creating and releasing these models.

License: Gemma (gated). Accept the license on the upstream Hub card before downloading.