ItsMaxNorm
/

bpess

chess,

tiktoken,

tokenizer

Model card Files Files and versions

xet

Community

bpess / config.json

ItsMaxNorm

Upload folder using huggingface_hub

643e9f6 verified 18 days ago

raw

history blame

160 Bytes

{"pattern": "'(?i:[sdmt]|ll|ve|re)|[^\\r\\n\\p{L}\\p{N}]?+\\p{L}+|\\p{N}{1,3}| ?[^\\s\\p{L}\\p{N}]++[\\r\\n]*|\\s*[\\r\\n]|\\s+(?!\\S)|\\s+", "vocab_size": 388}