Hugging Face's logo

ASTERIZER
/
LUNA-Tokenizer

Token Classification
English
gpt_neox
gpt-neox
tokenizer
pythia
luna
text-generation
Model card Files
xet
 Community
LUNA-Tokenizer

Commit History

  1. Add CC BY-NC-ND 4.0 license

    REZIRETSA
    Verified signature
    b948acc

  2. Apply CC BY-NC-ND 4.0 license

    REZIRETSA
    Verified signature
    d35070a

  3. Add shared LUNA tokenizer (GPT-NeoX, vocab 50304)

    REZIRETSA
    Verified signature
    99bdaa8

  4. Add shared LUNA tokenizer (GPT-NeoX, vocab 50304)

    REZIRETSA
    Verified signature
    31bc007

  5. Add shared LUNA tokenizer (GPT-NeoX, vocab 50304)

    REZIRETSA
    Verified signature
    be9a0fa

  6. Add shared LUNA tokenizer (GPT-NeoX, vocab 50304)

    REZIRETSA
    Verified signature
    196d102

  7. initial commit

    REZIRETSA
    Verified signature
    36643cf