README.md

install

pip install torch numpy transformers datasets tiktoken wandb tqdm

Dependencies:

• pytorch
• numpy
• transformers for huggingface transformers
• datasets for huggingface datasets (preprocess OpenWebText test and validation bin)
• tiktoken for OpenAI's fast BPE code
• wandb for optional logging
• tqdm for progress bars

quick start

Inference:

$ python inference.py

Thanks

Zero To Hero series. Specifically, the GPT video is popular if you have some prior language modeling context.