---
license: mit
language:
- en
- ru
- he
- de
- ar
---

This is a BPE tokenizer that is used in Prome models starting from the second version