Thaitokenizer / tokenizer.json

Commit History

Implement feature X to enhance user experience and fix bug Y in module Z
7c0f077

Nattapong Tapachoom commited on

Update tokenizer.json and vocab.json for improved formatting and added new token
bf83b0d

Nattapong Tapachoom commited on

Add tokenizer configuration file with special tokens and settings
12f0d60

Nattapong Tapachoom commited on

Upload folder using huggingface_hub
55f2143
verified

JonusNattapong commited on