Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
gsaltintas
/
tokenizer-comparison
like
0
Sleeping
App
Files
Files
Community
Fetching metadata from the HF Docker repository...
0c7d05e
tokenizer-comparison
30.3 kB
2 contributors
History:
7 commits
Gül Sena Altıntaş
subworsd mostly fixed, tokenmonster and byt5 still wrong [WIP]
0c7d05e
6 months ago
.gitattributes
1.52 kB
initial commit
6 months ago
.gitignore
43 Bytes
Refactoring, and visual improvements
6 months ago
README.md
263 Bytes
initial commit
6 months ago
app.py
21.5 kB
Added sample texts
6 months ago
mappings.py
1.67 kB
Refactoring, and visual improvements
6 months ago
requirements.txt
48 Bytes
Added additional tokenizers
6 months ago
utils.py
5.22 kB
subworsd mostly fixed, tokenmonster and byt5 still wrong [WIP]
6 months ago
