ASTERIZER
/

LUNA-Tokenizer

Token Classification

text-generation

Model card Files Files and versions

LUNA-Tokenizer / tokenizer.json

REZIRETSA's picture

Add shared LUNA tokenizer (GPT-NeoX, vocab 50304)

31bc007 verified 3 days ago

History Contribute Delete

2.11 MB

File too large to display, you can check the raw version instead.