Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
tspersian
/
mana_tokenizer
like
0
Persian
English
Arabic
License:
mit
Model card
Files
Files and versions
xet
Community
main
mana_tokenizer
14.1 MB
1 contributor
History:
16 commits
tspersian
control
81c873e
about 1 year ago
.gitattributes
Safe
88 Bytes
Re-add model and vocab files with LFS
over 1 year ago
.gitignore
Safe
13 Bytes
package
about 1 year ago
README.md
Safe
2.28 kB
regex
about 1 year ago
__init__.py
Safe
133 Bytes
package
about 1 year ago
base.py
Safe
14.6 kB
package
about 1 year ago
benchmark.py
Safe
3.49 kB
improve
about 1 year ago
helper.py
Safe
3.27 kB
scripts
about 1 year ago
mana.model
2.92 MB
xet
regex
about 1 year ago
mana.vocab
Safe
11.1 MB
xet
control
about 1 year ago
mana_tokenizer.py
Safe
6.36 kB
improve
about 1 year ago
test.py
Safe
229 Bytes
package
about 1 year ago