openeurollm
/

tokenizer-128k

european-languages

Model card Files Files and versions

Commit History

Add model card with description, usage, and evaluation results

a89eaa2
verified

timpal0l commited on 1 day ago

Upload 128k vocab SentencePiece BPE tokenizer (trained on 175GB)

43f50d5
verified

timpal0l commited on 1 day ago

initial commit

cde2ef4
verified

timpal0l commited on 1 day ago