ByteSpanTokenisers
/

tokenizers

Model card Files Files and versions

tokenizers / frequencymulti_64000

8.9 MB

Ctrl+K

Ctrl+K

3 contributors

History: 1 commit

Zeb

Fix multilingual frequency

d30eee2 11 months ago