Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Hardware
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
junaid008
/
hindko-tokenizer
Like
2
Northern Hindko
Southern Hindko
tokenizers
tokenizer
sentencepiece
unigram
hindko
low-resource
urdu-script
perso-arabic
License:
cc-by-4.0
Model card
Files
Files and versions
xet
Community
Copy to bucket
new
main
hindko-tokenizer
6.36 MB
Ctrl+K
Ctrl+K
1 contributor
History:
10 commits
junaid008
Update README.md
5eaa6de
verified
2 days ago
benchmarks
Add benchmark images (benchmarks/)
3 days ago
eval
Add evaluation tables and charts
3 days ago
examples
Upload quickstart.py
3 days ago
.gitattributes
Safe
2.21 kB
Add benchmark images (benchmarks/)
3 days ago
LICENSE
Safe
19.3 kB
License: CC BY 4.0; add benchmark gallery to the model card; refresh SHA256SUMS
3 days ago
README.md
Safe
48.4 kB
Update README.md
2 days ago
SHA256SUMS
Safe
2.85 kB
License: CC BY 4.0; add benchmark gallery to the model card; refresh SHA256SUMS
3 days ago
gauge.json
Safe
6.91 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files)
3 days ago
sp.model
Safe
669 kB
xet
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files)
3 days ago
sp.vocab
Safe
951 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files)
3 days ago
special_tokens_map.json
Safe
1.61 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files)
3 days ago
tokenizer.json
Safe
2.16 MB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files)
3 days ago
tokenizer_config.json
Safe
14.6 kB
Hindko tokenizer v1.0.0: SentencePiece Unigram 32k (tokenizer files)
3 days ago
