atharvanighot
/

hindi-tokenizer

Model card Files Files and versions

Model Card for Model ID

Llama 2 Tokenizer with Additional 25104 Hindi Tokens which makes it much better on Hindi Data

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using atharvanighot/hindi-tokenizer 1