Update tokenizer_config.json
#7
by abhipn - opened
Configuring bos, eos, pad and unk tokens helps load the tokenizer successfully, otherwise getting a recursion error "RecursionError: maximum recursion depth exceeded while getting the str of an object."
Thanks!
kbressem changed pull request status to merged