smartcore-v1 / tokenizer /tokenizer.model

Commit History

Faz 0: 48K EN+TR SentencePiece tokenizer (frozen)
505e70d
verified

kdirgul commited on