Hugging Face's logo

ByteSpanTokenisers
/
tokenizers

Model card Files
xet
 Community
tokenizers

Commit History

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c5daf73

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    596f9ba

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    1562447

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d415def

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d9d2be1

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    bf6af83

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    162d166

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    67cc952

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    47b2f9e

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0e739a7

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ff2e0a8

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f1580b2

  11. Fix tokenizers

    Zeb
    6b0beee

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    2e4b4eb

  13. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4d7f813

  14. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    eed0fd0

  15. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    b1001e7

  16. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6b0e22b

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    cb6ec62

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    2063827

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4ea427e

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    683f4d3

  5. Remove faulty tokenizers

    Zeb
    7005c51

  6. Update frequency

    Zeb
    ad1ff6d

  7. Fix spaceless tokenizers

    Zeb
    d69e013

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    b08c86b

  9. Fix spaceless tokenizers

    Zeb
    c58de58

  10. Remove ngram tokenizers

    Zeb
    7487a89

  11. Delete fw57M_Entropy_thresholdM_500

    codebyzeb
    Verified signature
    a802c9a

  12. Fix multilingual frequency

    Zeb
    d30eee2

  13. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    29dda54

  14. Delete fw57M_Entropy_thresholdM_500

    codebyzeb
    Verified signature
    b3b5b2d

  15. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ad3d889

  16. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7e118b6

  17. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    62a0567

  18. Fix tokenizers

    Zeb
    f9c3417

  19. Separate frequency tokenizers

    Zeb
    f25f2a5

  20. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f9c85cd

  21. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    280544f

  22. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    3bd5966

  23. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    77097ea

  24. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    67b58c4

  25. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f0fd6ae

  26. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    72c0e05

  27. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    114a45f

  28. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    06adb6a

  29. Upload folder using huggingface_hub

    codebyzeb
    1b719ca

  30. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    853f739

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    60e26ce

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4763edc