Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
toksuite
/
tokenizer-comparison
like
0
Running
App
Files
Files
Community
Fetching metadata from the HF Docker repository...
37a99cb
tokenizer-comparison
36.5 kB
2 contributors
History:
8 commits
Gül Sena Altıntaş
Add normalization
37a99cb
7 months ago
.gitattributes
1.52 kB
initial commit
7 months ago
.gitignore
43 Bytes
Refactoring, and visual improvements
7 months ago
README.md
263 Bytes
initial commit
7 months ago
app.py
26.2 kB
Add normalization
7 months ago
mappings.py
1.67 kB
Refactoring, and visual improvements
7 months ago
requirements.txt
48 Bytes
Added additional tokenizers
7 months ago
utils.py
6.73 kB
Add normalization
7 months ago
