Hugging Face's logo

ByteSpanTokenisers
/
tokenizers

Model card Files
xet
 Community
tokenizers

Commit History

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0290e96

  2. Bytelevel without normalizer

    Zeb
    d842b80

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    30b14ac

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f1c1fba

  5. Remove normalizers

    Zeb
    64b3309

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    38928ba

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    847dbbd

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    625b01f

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4637696

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    aa577b2

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    bfc2a21

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f1e8d54

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    fa83c21

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4ab1b41

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4b9bf06

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    46e77f8

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    2527407

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4ed123d

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7e062f7

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    a02f082

  13. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6e5822f

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    038fc30

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    14a8d5d

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8ea2e94

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    382f45b

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    9399389

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    e6c0cea

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    2bdfab0

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d7c0993

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    defd90e

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    200b2b1

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    00f1ccd

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c8471f1

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7688dda

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0c02895

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    785e99f

  3. Remove faulty tokenizers

    Zeb
    26f9c53

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    85c3781

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f46fd0f

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c4fcb17

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    39ede7a

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    bd5babd

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    cbd3433

  5. Temporarily add tokenizer

    Zeb
    cab11c3

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    3303ec4

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    062a1df

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    278d102

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f0c169c

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0c342cf

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8943763