atharvanighot
/

hindi-tokenizer

Model card Files Files and versions

hindi-tokenizer / README.md

atharvanighot's picture

Update README.md

8721db5 verified over 1 year ago

|

history blame contribute delete

226 Bytes

library_name: transformers
tags: []

Model Card for Model ID

Llama 2 Tokenizer with Additional 25104 Hindi Tokens which makes it much better on Hindi Data