tokenizer-fineweb_tr / tokenizer_config.json

Commit History

Register tokenizer as HF SentencePiece tokenizer
36565f0
verified

suchirsalhan commited on