Uploading the logging phase format FindWebEDU dataset. This is a 163 million parameter model because the weights are untied here.

Uploading the model in Hugging Face format trained on Open Web Text data. It's a 124M parameter model.

Upload tinygpt_pretrained_weights_tied_openwebtext.pt

Delete pretraining/tinygpt huggingface

Rename pretraining/checkpoint/tinygpt_pretrained_checkpoint_438k.pt to pretraining/checkpoint/tinygpt_pretrained_checkpoint_untied_fineweb.pt

pretrained and finetuned tinyGPT dataset