Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets new
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

2ira
/
Byte-lingua-code

Model card Files Files and versions
xet
Community
Byte-lingua-code / superbpe /tokenizers_superbpe /tokenizers /src /pre_tokenizers
154 kB
Ctrl+K
Ctrl+K
  • 1 contributor
History: 1 commit
2ira's picture
2ira
offline_compression_graph_code
72c0672 verified 4 months ago
  • unicode_scripts
    offline_compression_graph_code 4 months ago
  • bert.rs
    2.64 kB
    offline_compression_graph_code 4 months ago
  • byte_level.rs
    19.8 kB
    offline_compression_graph_code 4 months ago
  • delimiter.rs
    769 Bytes
    offline_compression_graph_code 4 months ago
  • digits.rs
    3.18 kB
    offline_compression_graph_code 4 months ago
  • metaspace.rs
    12.5 kB
    offline_compression_graph_code 4 months ago
  • mod.rs
    13.1 kB
    offline_compression_graph_code 4 months ago
  • punctuation.rs
    2.35 kB
    offline_compression_graph_code 4 months ago
  • sequence.rs
    2.12 kB
    offline_compression_graph_code 4 months ago
  • split.rs
    7.63 kB
    offline_compression_graph_code 4 months ago
  • whitespace.rs
    3.06 kB
    offline_compression_graph_code 4 months ago