Hugging Face's logo

ByteSpanTokenisers
/
tokenizers

Model card Files
xet
 Community
tokenizers

Commit History

  1. Fix tokenizer names

    Zeb
    84b51c9

  2. Remove tokenisers not analysed

    Zeb
    dcaf536

  3. Rename all tokenizers

    Zeb
    583e43f

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8b5bd01

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    daf4462

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    960adb1

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    e4a5353

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    294a2d0

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    64e4613

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    1fe38fe

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    1a2b660

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d77eeef

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    77dc68e

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    3365b58

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    83bd1f5

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    3a16124

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    9f8661a

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    2d81082

  13. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6f50d0e

  14. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    2a5fb1b

  15. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ffa7675

  16. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c983585

  17. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    b510387

  18. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    a218539

  19. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    36f0710

  20. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    9870c9e

  21. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    11f160e

  22. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7db89b2

  23. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    590737e

  24. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6f2eb6e

  25. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    5c4cfbf

  26. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    539cb50

  27. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    58f43a1

  28. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7b015fa

  29. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    27249a1

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    84d254c

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    e76cb7f

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    fbaaea7

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6508d77

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    180eaec

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    1a60b3c

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    1f716b2

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0966175

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c9eab65

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7def990

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    60429c2

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    88b7b77

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7f9bd40

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    9d702d6

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8ac5d80