Hugging Face's logo

bashyaldhiraj2067
/
NepaliTokenizerBPE

Transformers
Model card Files
xet
Community
NepaliTokenizerBPE / tokenizer.json

Commit History

Add custom BPE tokenizer with special tokens
6ad1496
verified

bashyaldhiraj2067 commited on