Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Log In
Sign Up
Remeinium
/
WWHO
like
0
Follow
Remeinium AI
1
Feature Extraction
Transformers
Remeinium/WWHO_30m
Sinhala
Hindi
English
tokenizer
WWHO
SGPE
linguis_trie
token
tokenization
Syllable
remeinium
transformer
linguistics
NLP
sinhala
hindi
english
BPE
GPE
Eval Results (legacy)
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
Deploy
Use this model
main
WWHO
18.5 MB
1 contributor
History:
7 commits
thekusaldarshana
Update README.md
b3a398c
verified
11 days ago
.gitattributes
Safe
1.52 kB
initial commit
23 days ago
EVALUATION.md
Safe
18.9 kB
Seperate Before you Compress
11 days ago
LICENSE
Safe
9.14 kB
Syllable is the Token
23 days ago
README.md
5.93 kB
Update README.md
11 days ago
encoder.py
Safe
13.1 kB
Seperate Before you Compress
11 days ago
gpe_trainer.py
Safe
28.4 kB
Seperate Before you Compress
11 days ago
linguis_trie.py
Safe
11.1 kB
WWHO
13 days ago
router.py
Safe
5.75 kB
Seperate Before you Compress
11 days ago
tokenizer.json
Safe
8.07 MB
WWHO
13 days ago
vocab.json
Safe
10.4 MB
WWHO
13 days ago
