File size: 2,404 Bytes
e04b684 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
pipeline_tag: text-generation
license: mit
base_model: openai-community/openai-gpt
library_name: kerasformers
language:
- en
tags:
- keras
- kerasformers
- gpt
- openai-gpt
- text-generation
- pytorch
- jax
- tf
---
# Run GPT with Keras 3: JAX, PyTorch, or TensorFlow
[](https://github.com/IMvision12/KerasFormers) [](https://imvision12.github.io/KerasFormers/gpt/)
# kerasformers/gpt
Paper: [Improving Language Understanding by Generative Pre-Training (Radford et al., 2018)](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf)
GPT (the original GPT-1) is OpenAI's first generative pre-trained transformer: a
12-layer decoder-only model with learned position embeddings (512-token context),
`gelu_new` activations, and a byte-pair-encoding tokenizer, trained on BookCorpus.
This is the **117M** base completion model (no chat template).
For more details, see the upstream [model card](https://huggingface.co/openai-community/openai-gpt).
Pure-**Keras 3** conversion of [`openai-community/openai-gpt`](https://huggingface.co/openai-community/openai-gpt) for
[kerasformers](https://github.com/IMvision12/KerasFormers). One implementation runs unmodified on
**TensorFlow / Torch / JAX**.
## Quick start
```python
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from kerasformers.models.gpt import GptTextGenerate, GptTokenizer
model = GptTextGenerate.from_weights("kerasformers/gpt")
tokenizer = GptTokenizer.from_weights("kerasformers/gpt")
inputs = tokenizer("the meaning of life is")
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0]))
```
## Tips
- Set `KERAS_BACKEND` **before** importing Keras / kerasformers.
- Context length is 512 tokens; this is a base completion model, not
instruction-tuned.
- See the [GPT docs](https://imvision12.github.io/KerasFormers/gpt/) and [Loading Weights](https://imvision12.github.io/KerasFormers/loading_weights/).
- Upstream safetensors still work via the `hf:` prefix, e.g.
`GptTextGenerate.from_weights("hf:openai-community/openai-gpt")`.
## Special Thanks
A huge thank you to the OpenAI GPT authors for creating and releasing this model.
License: MIT, inherited from the upstream OpenAI GPT release.
|