Hugging Face
Search models, datasets, users...
Ctrl K
Models
Datasets
Spaces
Buckets
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Hardware
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
somast
/
somali-tokenizer
Like
4
Follow
Somast
20
Feature Extraction
Somali
tokenizers
tokenizer
somali
bpe
byte-level
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
Copy to bucket
main
somali-tokenizer
Commit History
Nov 19
2 days ago
03:07
Update model card: new id, Python package, benchmark against general tokenizers
sharafdin
Verified signature
7420ad3
Oct 19
22:35
Add Somali ByteLevel BPE v2
sharafdin
Verified signature
e7d6827
22:35
initial commit
sharafdin
Verified signature
438c6b6