bashyaldhiraj2067
/

NepaliBPE_Tokenizer

Model card Files Files and versions

NepaliBPE_Tokenizer / tokenizer.json

Commit History

Add custom BPE tokenizer with special tokens

a7e2d11
verified

bashyaldhiraj2067 commited on Apr 8, 2025

Upload tokenizer.json with huggingface_hub

3eb1a82
verified

bashyaldhiraj2067 commited on Apr 8, 2025

Upload tokenizer

aeed873
verified

bashyaldhiraj2067 commited on Apr 8, 2025