GPT (causal-LM)
https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
Lucid port of transformers/openai-community/openai-gpt,
converted to Lucid-native safetensors.
Available weights
| Tag | Params | GFLOPs | Size | Source |
|---|---|---|---|---|
BOOKCORPUS (default) |
116.5M | — | 563.15 MB | transformers |
Usage
import lucid
import lucid.models as models
from lucid.models.weights import GptLmWeights
# default tag
model = models.gpt_lm(pretrained=True)
# explicit tag (enum or string)
model = models.gpt_lm(weights=GptLmWeights.BOOKCORPUS)
model = models.gpt_lm(pretrained="BOOKCORPUS")
# feed token ids (tokenize with the matching lucid.utils.tokenizer)
input_ids = lucid.tensor([[101, 7592, 2088, 102]], dtype=lucid.int64)
out = model(input_ids)
logits = out.logits # (B, T, vocab_size)
Conversion
Converted from transformers/openai-community/openai-gpt via
python -m tools.convert_weights gpt_lm --tag BOOKCORPUS.
Key mapping + numerical parity verified against the source.
License
mit — inherited from the original weights.
Citation
Radford et al., "Improving Language Understanding by Generative Pre-Training", 2018 (GPT).