Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets new
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

tspersian
/
mana_tokenizer

Persian
English
Arabic
Model card Files Files and versions
xet
Community
mana_tokenizer
Ctrl+K
Ctrl+K
  • 1 contributor
History: 16 commits
tspersian's picture
tspersian
control
81c873e over 1 year ago
  • .gitattributes
    88 Bytes
    Re-add model and vocab files with LFS over 1 year ago
  • .gitignore
    13 Bytes
    package over 1 year ago
  • README.md
    2.28 kB
    regex over 1 year ago
  • __init__.py
    133 Bytes
    package over 1 year ago
  • base.py
    14.6 kB
    package over 1 year ago
  • benchmark.py
    3.49 kB
    improve over 1 year ago
  • helper.py
    3.27 kB
    scripts over 1 year ago
  • mana.model
    2.92 MB
    xet
    regex over 1 year ago
  • mana.vocab
    11.1 MB
    xet
    control over 1 year ago
  • mana_tokenizer.py
    6.36 kB
    improve over 1 year ago
  • test.py
    229 Bytes
    package over 1 year ago