README.md

---
pipeline_tag: translation
license: apache-2.0
base_model: google-t5/t5-11b
library_name: kerasformers
language:
- en
- fr
- de
- ro
tags:
- keras
- kerasformers
- t5
- text2text-generation
- pytorch
- jax
- tf
---

# T5-11b in Keras 3: JAX, PyTorch, or TensorFlow

[![GitHub](https://img.shields.io/badge/GitHub-KerasFormers-181717?logo=github)](https://github.com/IMvision12/KerasFormers) [![Docs](https://img.shields.io/badge/Docs-T5-1f6feb)](https://imvision12.github.io/KerasFormers/t5/) [![HuggingFace](https://img.shields.io/badge/HuggingFace-T5-ffd21e?logo=huggingface&logoColor=black)](https://huggingface.co/collections/kerasformers/t5-6a85056935f438653698c56f)

# kerasformers/t5_11b

Pure-**Keras 3** conversion of [`google-t5/t5-11b`](https://huggingface.co/google-t5/t5-11b) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**, bit-exact with the Hugging Face original. T5 is a text-to-text encoder-decoder; this repo hosts the full backbone (`kf_config.json` declares `T5Model`), and every T5 class (`T5ConditionalGenerate`, `T5EncoderModel`, and the classification / QA heads) loads its subset from the one `model.weights.h5`.

For model details, license, and usage terms, see the upstream [model card](https://huggingface.co/google-t5/t5-11b).

Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/abs/1910.10683)

## Quick start

```python
import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from kerasformers.models.t5 import T5ConditionalGenerate, T5Tokenizer

model = T5ConditionalGenerate.from_weights("kerasformers/t5_11b")
tokenizer = T5Tokenizer.from_weights("kerasformers/t5_11b")

inputs = tokenizer("translate English to German: The house is wonderful.")
output_ids = model.generate(
    inputs["input_ids"], inputs["attention_mask"], max_new_tokens=40
)
print(tokenizer.decode(output_ids[0]))
```

Load any T5 variant the same way with `from_weights("kerasformers/<variant>")`. Browse them all in the [T5 collection](https://huggingface.co/collections/kerasformers/t5-6a85056935f438653698c56f).

## Available classes

Load any of these from this repo with `from_weights("kerasformers/t5_11b")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).

| Class | Task |
|---|---|
| `T5Model` | Encoder backbone |
| `T5ConditionalGenerate` | Text-to-text generation |
| `T5EncoderModel` | Encoder-only features |
| `T5SequenceClassify` | Sequence classification |
| `T5TokenClassify` | Token classification (NER / POS) |
| `T5QnA` | Extractive question answering |

```python
from kerasformers.models.t5 import T5SequenceClassify
model = T5SequenceClassify.from_weights("kerasformers/t5_11b")
```

## Special Thanks

Thank you to the Google T5 team for creating and releasing the T5 models.