chengzl18
/

cctokenizer

Model card Files Files and versions

cctokenizer / replace.json

Commit History

A Chinese Character Tokenizer with a Many-to-one Mapping.

9836307

chengzl18 commited on Mar 24, 2023