Buckets:

hf-doc-build
/

doc-dev

Files

xet

hf-doc-build/doc-dev / transformers /pr_43265 /en /model_doc /t5gemma2.md

HuggingFaceDocBuilder

about 2 months ago

preview code

download

raw

78.6 kB

T5Gemma 2

T5Gemma 2 is a family of pretrained encoder-decoder large language models with strong multilingual, multimodal and long-context capability, available in 270M-270M, 1B-1B and 4B-4B parameters. Following T5Gemma, it is built via model adaptation (based on Gemma 3) using UL2. The architecture is similar to T5Gemma and Gemma 3, enhanced with tied word embeddings and merged self- and cross-attention to save model parameters.

Click on the T5Gemma 2 models in the right sidebar for more examples of how to apply T5Gemma 2 to different language tasks.

The example below demonstrates how to chat with the model with Pipeline or the AutoModel class, and from the command line.

from transformers import pipeline

generator = pipeline(
    "image-text-to-text",
    model="google/t5gemma-2-270m-270m",
    device_map="auto",
)

generator(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg",
    text="<start_of_image> in this image, there is",
    generate_kwargs={"do_sample": False, "max_new_tokens": 50},
)

import requests
from PIL import Image

from transformers import AutoModelForSeq2SeqLM, AutoProcessor

processor = AutoProcessor.from_pretrained("google/t5gemma-2-270m-270m")
model = AutoModelForSeq2SeqLM.from_pretrained(
    "google/t5gemma-2-270m-270m",
    device_map="auto",
)

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"
image = Image.open(requests.get(url, stream=True).raw)
prompt = "<start_of_image> in this image, there is"

model_inputs = processor(text=prompt, images=image, return_tensors="pt").to(model.device)
generation = model.generate(**model_inputs, max_new_tokens=20, do_sample=False)
print(processor.decode(generation[0]))

T5Gemma2Config[[transformers.T5Gemma2Config]]

transformers.T5Gemma2Config[[transformers.T5Gemma2Config]]

Source

This is the configuration class to store the configuration of a T5Gemma2Model. It is used to instantiate a T5Gemma2 model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the google/t5gemma-2-270m-270m

Configuration objects inherit from PreTrainedConfig and can be used to control the model outputs. Read the documentation from PreTrainedConfig for more information.

>>> from transformers import T5Gemma2Config, T5Gemma2Model
>>> t5gemma2_config = T5Gemma2Config.from_pretrained("google/t5gemma-270m-270m")
>>> model = T5Gemma2Model(t5gemma2_config)

Parameters:

is_encoder_decoder (bool, optional, defaults to True) : Whether the model is used as an encoder/decoder or not.

encoder (Union[T5Gemma2EncoderConfig, dict], optional, optional) : Configuration for the encoder.

decoder (Union[T5Gemma2DecoderConfig, dict], optional, optional) : Configuration for the decoder.

dropout_rate (Union[float, int], optional, defaults to 0.0) : The ratio for all dropout layers.

attention_dropout (Union[float, int], optional, defaults to 0.0) : The dropout ratio for the attention probabilities.

classifier_dropout_rate (Union[float, int], optional, defaults to 0.0) : The dropout ratio for classifier.

initializer_range (float, optional, defaults to 0.02) : The standard deviation of the truncated_normal_initializer for initializing all weight matrices.

image_token_index (int, optional, defaults to 256001) : The image token index used as a placeholder for input images.

eoi_token_index (int, optional) : The end-of-image token index to wrap the image prompt. Will be same as self.encoder.eoi_token_index

tie_word_embeddings (bool, optional, defaults to True) : Whether to tie weight embeddings according to model's tied_weights_keys mapping.

T5Gemma2TextConfig[[transformers.T5Gemma2TextConfig]]

transformers.T5Gemma2TextConfig[[transformers.T5Gemma2TextConfig]]

Source

Configuration objects inherit from PreTrainedConfig and can be used to control the model outputs. Read the documentation from PreTrainedConfig for more information.

Parameters:

vocab_size (int, optional, defaults to 262208) : Vocabulary size of the model. Defines the number of different tokens that can be represented by the input_ids.

hidden_size (int, optional, defaults to 2304) : Dimension of the hidden representations.

intermediate_size (int, optional, defaults to 9216) : Dimension of the MLP representations.

num_hidden_layers (int, optional, defaults to 26) : Number of hidden layers in the Transformer decoder.

num_attention_heads (int, optional, defaults to 8) : Number of attention heads for each attention layer in the Transformer decoder.

num_key_value_heads (int, optional, defaults to 4) : This is the number of key_value heads that should be used to implement Grouped Query Attention. If num_key_value_heads=num_attention_heads, the model will use Multi Head Attention (MHA), if num_key_value_heads=1 the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out this paper. If it is not specified, will default to num_attention_heads.

head_dim (int, optional, defaults to 256) : The attention head dimension. If None, it will default to hidden_size // num_attention_heads

hidden_activation (str, optional, defaults to gelu_pytorch_tanh) : The non-linear activation function (function or string) in the decoder. For example, "gelu", "relu", "silu", etc.

max_position_embeddings (int, optional, defaults to 131072) : The maximum sequence length that this model might ever be used with.

initializer_range (float, optional, defaults to 0.02) : The standard deviation of the truncated_normal_initializer for initializing all weight matrices.

rms_norm_eps (float, optional, defaults to 1e-06) : The epsilon used by the rms normalization layers.

use_cache (bool, optional, defaults to True) : Whether or not the model should return the last key/values attentions (not used by all models). Only relevant if config.is_decoder=True or when the model is a decoder-only generative model.

pad_token_id (int, optional, defaults to 0) : Token id used for padding in the vocabulary.

eos_token_id (Union[int, list[int]], optional, defaults to 1) : Token id used for end-of-stream in the vocabulary.

bos_token_id (int, optional, defaults to 2) : Token id used for beginning-of-stream in the vocabulary.

tie_word_embeddings (bool, optional, defaults to True) : Whether to tie weight embeddings according to model's tied_weights_keys mapping.

rope_parameters (dict, optional) : Dictionary containing the configuration parameters for the RoPE embeddings. The dictionary should contain a value for rope_theta and optionally parameters used for scaling in case you want to use RoPE with longer max_position_embeddings.

attention_bias (bool, optional, defaults to False) : Whether to use a bias in the query, key, value and output projection layers during self-attention.

attention_dropout (Union[int, float], optional, defaults to 0.0) : The dropout ratio for the attention probabilities.

query_pre_attn_scalar (float, optional, defaults to 256) : Scaling factor used on the attention scores

sliding_window (int, optional, defaults to 4096) : Sliding window attention window size. If None, no sliding window is applied.

layer_types (list[str], optional) : A list that explicitly maps each layer index with its layer type. If not provided, it will be automatically generated based on config values.

final_logit_softcapping (float, optional) : Scaling factor when applying tanh softcapping on the logits.

attn_logit_softcapping (float, optional) : Scaling factor when applying tanh softcapping on the attention scores.

T5Gemma2EncoderConfig[[transformers.T5Gemma2EncoderConfig]]

transformers.T5Gemma2EncoderConfig[[transformers.T5Gemma2EncoderConfig]]

Source

Configuration objects inherit from PreTrainedConfig and can be used to control the model outputs. Read the documentation from PreTrainedConfig for more information.

Example:

>>> from transformers import T5Gemma2EncoderForConditionalGeneration, T5Gemma2EncoderConfig, SiglipVisionConfig, T5Gemma2EncoderTextConfig

>>> # Initializing a Siglip-like vision config
>>> vision_config = SiglipVisionConfig()

>>> # Initializing a T5Gemma2Encoder Text config
>>> text_config = T5Gemma2EncoderTextConfig()

>>> # Initializing a T5Gemma2Encoder gemma-3-4b style configuration
>>> configuration = T5Gemma2EncoderConfig(vision_config, text_config)

>>> # Initializing a model from the gemma-3-4b style configuration
>>> model = T5Gemma2EncoderTextConfig(configuration)

>>> # Accessing the model configuration
>>> configuration = model.config

Parameters:

text_config (Union[~models.t5gemma2.configuration_t5gemma2.T5Gemma2TextConfig, dict[str, Any]], optional) : The config object or dictionary of the text backbone.

vision_config (Union[~models.siglip.configuration_siglip.SiglipVisionConfig, dict[str, Any]], optional) : The config object or dictionary of the vision backbone.

mm_tokens_per_image (int, optional, defaults to 256) : The number of tokens per image embedding.

boi_token_index (int, optional, defaults to 255999) : The begin-of-image token index to wrap the image prompt.

eoi_token_index (int, optional, defaults to 256000) : The end-of-image token index to wrap the image prompt.

image_token_index (int, optional, defaults to 262144) : The image token index used as a placeholder for input images.

initializer_range (float, optional, defaults to 0.02) : The standard deviation of the truncated_normal_initializer for initializing all weight matrices.

tie_word_embeddings (bool, optional, defaults to True) : Whether to tie weight embeddings according to model's tied_weights_keys mapping.

T5Gemma2DecoderConfig[[transformers.T5Gemma2DecoderConfig]]

transformers.T5Gemma2DecoderConfig[[transformers.T5Gemma2DecoderConfig]]