Run GPT with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/gpt

Paper: Improving Language Understanding by Generative Pre-Training (Radford et al., 2018)

GPT (the original GPT-1) is OpenAI's first generative pre-trained transformer: a 12-layer decoder-only model with learned position embeddings (512-token context), gelu_new activations, and a byte-pair-encoding tokenizer, trained on BookCorpus. This is the 117M base completion model (no chat template).

For more details, see the upstream model card.

Pure-Keras 3 conversion of openai-community/openai-gpt for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.gpt import GptTextGenerate, GptTokenizer

model = GptTextGenerate.from_weights("zeromodels/gpt")
tokenizer = GptTokenizer.from_weights("zeromodels/gpt")

inputs = tokenizer("the meaning of life is")
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0]))

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Context length is 512 tokens; this is a base completion model, not instruction-tuned.
• See the GPT docs and Loading Weights.
• Upstream safetensors still work via the hf: prefix, e.g. GptTextGenerate.from_weights("hf:openai-community/openai-gpt").

Special Thanks

A huge thank you to the OpenAI GPT authors for creating and releasing this model.

License: MIT, inherited from the upstream OpenAI GPT release.