File size: 3,016 Bytes
bf4d254 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
pipeline_tag: translation
license: apache-2.0
base_model: google-t5/t5-11b
library_name: kerasformers
language:
- en
- fr
- de
- ro
tags:
- keras
- kerasformers
- t5
- text2text-generation
- pytorch
- jax
- tf
---
# T5-11b in Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/KerasFormers) [](https://imvision12.github.io/KerasFormers/t5/) [](https://huggingface.co/collections/kerasformers/t5-6a85056935f438653698c56f)
# kerasformers/t5_11b
Pure-**Keras 3** conversion of [`google-t5/t5-11b`](https://huggingface.co/google-t5/t5-11b) for [kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on **TensorFlow / Torch / JAX**, bit-exact with the Hugging Face original. T5 is a text-to-text encoder-decoder; this repo hosts the full backbone (`kf_config.json` declares `T5Model`), and every T5 class (`T5ConditionalGenerate`, `T5EncoderModel`, and the classification / QA heads) loads its subset from the one `model.weights.h5`.
For model details, license, and usage terms, see the upstream [model card](https://huggingface.co/google-t5/t5-11b).
Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/abs/1910.10683)
## Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from kerasformers.models.t5 import T5ConditionalGenerate, T5Tokenizer
model = T5ConditionalGenerate.from_weights("kerasformers/t5_11b")
tokenizer = T5Tokenizer.from_weights("kerasformers/t5_11b")
inputs = tokenizer("translate English to German: The house is wonderful.")
output_ids = model.generate(
inputs["input_ids"], inputs["attention_mask"], max_new_tokens=40
)
print(tokenizer.decode(output_ids[0]))
```
Load any T5 variant the same way with `from_weights("kerasformers/<variant>")`. Browse them all in the [T5 collection](https://huggingface.co/collections/kerasformers/t5-6a85056935f438653698c56f).
## Available classes
Load any of these from this repo with `from_weights("kerasformers/t5_11b")` (or on the fly via the `hf:` prefix). The pretrained backbone is shared; task heads not stored in this checkpoint start randomly initialized, ready for fine-tuning (or load a `hf:` fine-tune).
| Class | Task |
|---|---|
| `T5Model` | Encoder backbone |
| `T5ConditionalGenerate` | Text-to-text generation |
| `T5EncoderModel` | Encoder-only features |
| `T5SequenceClassify` | Sequence classification |
| `T5TokenClassify` | Token classification (NER / POS) |
| `T5QnA` | Extractive question answering |
```python
from kerasformers.models.t5 import T5SequenceClassify
model = T5SequenceClassify.from_weights("kerasformers/t5_11b")
```
## Special Thanks
Thank you to the Google T5 team for creating and releasing the T5 models.
|