somali-tokenizer / tokenizer_config.json

Commit History

  1. Add Somali ByteLevel BPE v2

    Verified signature
    e7d6827