Commit History

feat: update tokenizer artifacts with 41.4M character corpus
39ad643

janakhpon commited on

feat: restructure and upgrade to 32k vocab model (v2)
90115a5

janakhpon commited on

feat: restructure and upgrade to 32k vocab model (v2)
e909ce1

janakhpon commited on

feat: simplified mon tokenizer in hf format, updated tags, resolve the legacy issue
9d74fcf

janakhpon commited on

feat: simplified mon tokenizer in hf format, updated tags, resolve the legacy issue
1f1b899

janakhpon commited on

feat: simplified mon tokenizer in hf format, updated tags, resolve the legacy issue
e9d0f85

janakhpon commited on

feat: simplified mon tokenizer in hf format, updated tags, resolve the legacy issue
81cf36d

janakhpon commited on

feat: simplified mon tokenizer in hf format, updated tags
9ed3203

janakhpon commited on

feat: simplified mon tokenizer in hf format, updated tags
bece7ec

janakhpon commited on

feat: simplified mon tokenizer in hf format
f51150c

janakhpon commited on

feat: simplified mon tokenizer in hf format
2ded9d3

janakhpon commited on

initial commit
3f73eda
verified

janakhpon commited on