bashyaldhiraj2067
/

NepaliBPE_Tokenizer

Model card Files Files and versions

NepaliBPE_Tokenizer / tokenizer_config.json

Commit History

Add custom BPE tokenizer with special tokens

a7e2d11
verified

bashyaldhiraj2067 commited on Apr 8, 2025

Upload tokenizer_config.json with huggingface_hub

738b26c
verified

bashyaldhiraj2067 commited on Apr 8, 2025

Upload tokenizer

aeed873
verified

bashyaldhiraj2067 commited on Apr 8, 2025