aakashMeghwar01
/

SindhiLM-Tokenizer-v1

morphology-aware

low-resource-nlp

Model card Files Files and versions

SindhiLM-Tokenizer-v1 / tokenizer.json

Commit History

Upgraded architecture: Unigram + Metaspace fix + 32k Vocab limit + Morpheme splitting

be8d37f
verified

aakashMeghwar01 commited on 25 days ago

Upgraded architecture: Unigram + Metaspace fix + 32k Vocab limit + Morpheme splitting

f82968f
verified

aakashMeghwar01 commited on 25 days ago

Upload tokenizer

619611c
verified

aakashMeghwar01 commited on 26 days ago

Upload tokenizer

f397ed2
verified

aakashMeghwar01 commited on 26 days ago