bad-gpt / encoder.py

Commit History

Switch from custom tokenizer to tiktoken
e3fc82f

shamashel commited on

Retrieve dataset from dataset instead of text file
2a3cf64

shamashel commited on

Clone from github
3c8279d

shamashel commited on