ThisenEkanayake
/

sinhala-tokenizer

Model card Files Files and versions

sinhala-tokenizer / tokenizer_config.json

Commit History

tokenizer class updated

e337c70

Thisen Ekanayake commited on Oct 1, 2025

updated tokenizer class

3841b6e

Thisen Ekanayake commited on Sep 30, 2025

Add custom Sinhala SentencePiece tokenizer

01e5ddf

Thisen Ekanayake commited on Sep 30, 2025