triton329
/

gpt2

Model card Files Files and versions

gpt2 / src /data /__init__.py

triton329's picture

Upload folder using huggingface_hub

c21e887 verified 13 days ago

History Blame Contribute Delete

125 Bytes

	from src.data.dataset import create_gpt_dataloader
	from src.data.tokenizer import SimpleTokenizer, TikTokenizer, build_vocab