Hugging Face's logo

ByteSpanTokenisers
/
tokenizers

Model card Files
xet
 Community
tokenizers

Commit History

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d2ac827

  2. Delete fw57M_Entropy_thresholdBX_600

    codebyzeb
    Verified signature
    a4fb558

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    53333c9

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d000429

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0322999

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c404f26

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    58beac7

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    57b71d2

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d7e180f

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7c9b416

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    77f8c2e

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ed6a20c

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    006a6ab

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    36edef8

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8d254aa

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    420e742

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4a56766

  8. Remove faulty tokenizers

    Zeb
    faf7f42

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8994231

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c8158ef

  11. Rename and remove old tokenizers

    Zeb
    24ade47

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f554e97

  13. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0f89be7

  14. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ccf1210

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8a22353

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    161dd71

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    371260a

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    bf6c8e4

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    a9d704d

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    e649e4e

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7b7b585

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    c0c59ff

  9. Rename mutual information tokenizers

    Zeb
    46d4bdf

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    154e15d

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    b763610

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    be3a680

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    57c37a4

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    a3aca41

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    cc8d9b1

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    9e2eb56

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    afd7ac4

  8. Rename frequency tokenizers

    Zeb
    fde09f8

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ce169aa

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4e4ca7d

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    47c71b1

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6cbba06

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    82fe54b

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    137aa67

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ccb0f87

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    a3843f5