ashrafs1/shakespeare

GPT-2 124M Speedrun

• Step: 100
• Val Loss: 5.3181

Load model

import torch, json
from model.gpt import GPT, GPTConfig

with open('config.json') as f:
    cfg = json.load(f)
config = GPTConfig(**{k: cfg[k] for k in ['context_length','vocab_size','num_layers','embd_size','num_heads']})
model = GPT(config)
model.load_state_dict(torch.load('model.pt', map_location='cpu'))