Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

camenduru
/
NeMo

NeMo
Model card Files Files and versions
xet
Community
NeMo / nemo /collections /common /tokenizers
158 kB
  • 1 contributor
History: 1 commit
camenduru's picture
camenduru
thanks to NVIDIA ❤
7934b29 almost 3 years ago
  • huggingface
    thanks to NVIDIA ❤ almost 3 years ago
  • text_to_speech
    thanks to NVIDIA ❤ almost 3 years ago
  • __init__.py
    1.36 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • aggregate_tokenizer.py
    8.72 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • bytelevel_tokenizers.py
    3.23 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • char_tokenizer.py
    25.7 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • chinese_tokenizers.py
    2.72 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • column_coder.py
    11.3 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • en_ja_tokenizers.py
    2.96 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • fairseq_tokenizer.py
    4.16 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • indic_tokenizers.py
    1.54 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • moses_tokenizers.py
    1.6 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • regex_tokenizer.py
    10 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • sentencepiece_tokenizer.py
    14.6 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • tabular_tokenizer.py
    6.94 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • tokenizer_spec.py
    1.4 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • word_tokenizer.py
    2.48 kB
    thanks to NVIDIA ❤ almost 3 years ago
  • youtokentome_tokenizer.py
    2.61 kB
    thanks to NVIDIA ❤ almost 3 years ago