Hugging Face's logo

openeurollm
/
tokenizer-128k-v2

sentencepiece
tokenizer
bpe
multilingual
openeurollm
Model card Files
xet
 Community
tokenizer-128k-v2
4.33 MB
  • 1 contributor
 History: 5 commits
timpal0l's picture
timpal0l
Multi-domain eval: 5 domains (FLORES, code, math, chat, PDFs) — v2-256k tops the SOTA leaderboard
026aa54 verified